2024-03-07T06:35:10.520 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-07T06:35:10.520 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/strays} --archive /home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926 --owner scheduled_leonidus@teuthology --name leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi --verbose -- /home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926/orig.config.yaml 2024-03-07T06:35:10.567 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T06:35:10.653 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926 branch: wip-lusov-qdb description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/strays} email: null first_in_suite: false job_id: '7583926' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-qdb ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 kclient: syntax: v2 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-qdb sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5072 sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 sleep_before_teardown: 0 subset: 246/10000 suite: fs suite_branch: wip-lusov-qdb suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNnQT/fxUBs5Q0elCOxjRtpReucIfdQ7XZT28EvF4NqbJJWKkBq9sNyVNk0kzyXIw+udH5hrcAWgun/AQdgp6tc= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMTx5NOlXi6N3VtHmq0Qr1XTvtJezbU/EhMYstAczFirOyXNqu+yRMicxrcRDOcv8elfpQq53o9nBxC/te7xEos= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_strays teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-07_05:44:13 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-07T06:35:10.653 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa; will attempt to use it 2024-03-07T06:35:10.654 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks 2024-03-07T06:35:10.654 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-07T06:35:10.656 INFO:teuthology.task.internal:Checking packages... 2024-03-07T06:35:10.681 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8b896a9e145796119f0451201dc4d53ddac97db0' 2024-03-07T06:35:10.681 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-07T06:35:10.682 INFO:teuthology.packaging:ref: None 2024-03-07T06:35:10.682 INFO:teuthology.packaging:tag: None 2024-03-07T06:35:10.682 INFO:teuthology.packaging:branch: wip-lusov-qdb 2024-03-07T06:35:10.682 INFO:teuthology.packaging:sha1: 8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T06:35:10.682 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-qdb 2024-03-07T06:35:10.828 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1848.g8b896a9e 2024-03-07T06:35:10.830 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-07T06:35:10.837 INFO:teuthology.task.internal:no buildpackages task found 2024-03-07T06:35:10.837 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-07T06:35:10.846 INFO:teuthology.task.internal:Saving configuration 2024-03-07T06:35:10.864 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-07T06:35:10.872 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-07T06:35:10.900 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 06:29:08.180308', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNnQT/fxUBs5Q0elCOxjRtpReucIfdQ7XZT28EvF4NqbJJWKkBq9sNyVNk0kzyXIw+udH5hrcAWgun/AQdgp6tc='} 2024-03-07T06:35:10.929 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 06:29:08.181504', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMTx5NOlXi6N3VtHmq0Qr1XTvtJezbU/EhMYstAczFirOyXNqu+yRMicxrcRDOcv8elfpQq53o9nBxC/te7xEos='} 2024-03-07T06:35:10.929 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-07T06:35:10.936 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-07T06:35:10.936 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-07T06:35:10.936 INFO:teuthology.run_tasks:Running task console_log... 2024-03-07T06:35:11.016 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f96a46dff70>, signals=[15]) 2024-03-07T06:35:11.016 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-07T06:35:11.023 INFO:teuthology.task.internal:Opening connections... 2024-03-07T06:35:11.023 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:35:11.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T06:35:11.144 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2024-03-07T06:35:11.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T06:35:11.244 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-07T06:35:11.252 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-03-07T06:35:11.269 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-03-07T06:35:11.270 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-03-07T06:35:11.327 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2024-03-07T06:35:11.327 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9" 2024-03-07T06:35:11.327 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2024-03-07T06:35:11.327 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2024-03-07T06:35:11.327 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9" 2024-03-07T06:35:11.327 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9" 2024-03-07T06:35:11.328 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-07T06:35:11.328 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2024-03-07T06:35:11.328 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon" 2024-03-07T06:35:11.328 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-07T06:35:11.328 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2024-03-07T06:35:11.328 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-07T06:35:11.328 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-07T06:35:11.328 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-07T06:35:11.329 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-03-07T06:35:11.368 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2024-03-07T06:35:11.387 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2024-03-07T06:35:11.387 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2024-03-07T06:35:11.444 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2024-03-07T06:35:11.444 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="9" 2024-03-07T06:35:11.444 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2024-03-07T06:35:11.444 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2024-03-07T06:35:11.444 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="9" 2024-03-07T06:35:11.445 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el9" 2024-03-07T06:35:11.445 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-07T06:35:11.445 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2024-03-07T06:35:11.445 INFO:teuthology.orchestra.run.smithi161.stdout:LOGO="fedora-logo-icon" 2024-03-07T06:35:11.445 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-07T06:35:11.445 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2024-03-07T06:35:11.445 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-07T06:35:11.445 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-07T06:35:11.445 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-07T06:35:11.445 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2024-03-07T06:35:11.476 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-07T06:35:11.489 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-07T06:35:11.497 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-07T06:35:11.498 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-03-07T06:35:11.501 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2024-03-07T06:35:11.517 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-07T06:35:11.560 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-07T06:35:11.561 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-03-07T06:35:11.563 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2024-03-07T06:35:11.667 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-07T06:35:11.723 INFO:teuthology.run_tasks:Running task kernel... 2024-03-07T06:35:11.741 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-07T06:35:11.741 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-03-07T06:35:11.741 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-03-07T06:35:11.741 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-07T06:35:11.742 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-07T06:35:11.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:35:11.758 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-07T06:35:11.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:35:11.762 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-07T06:35:12.115 DEBUG:teuthology.task.kernel:sha1 for client.1 is dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-07T06:35:12.115 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412"... 2024-03-07T06:35:12.115 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-03-07T06:35:12.120 DEBUG:teuthology.task.kernel:sha1 for client.0 is dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-07T06:35:12.120 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412"... 2024-03-07T06:35:12.120 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-03-07T06:35:12.132 INFO:teuthology.orchestra.run.smithi161.stdout:5.14.0-361.el9.x86_64 2024-03-07T06:35:12.133 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-07T06:35:12.133 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-07T06:35:12.134 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum upgrade -y linux-firmware 2024-03-07T06:35:12.137 INFO:teuthology.orchestra.run.smithi133.stdout:5.14.0-361.el9.x86_64 2024-03-07T06:35:12.137 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-07T06:35:12.137 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-07T06:35:12.138 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum upgrade -y linux-firmware 2024-03-07T06:35:14.396 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-9 - AppStream 13 MB/s | 8.1 MB 00:00 2024-03-07T06:35:14.447 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS-9 - AppStream 12 MB/s | 8.1 MB 00:00 2024-03-07T06:35:16.113 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-9 - BaseOS 6.2 MB/s | 2.5 MB 00:00 2024-03-07T06:35:16.231 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.5 MB 00:00 2024-03-07T06:35:17.762 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.0 MB 00:01 2024-03-07T06:35:19.350 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.0 MB 00:02 2024-03-07T06:35:25.645 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 2.8 MB/s | 19 MB 00:06 2024-03-07T06:35:37.778 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 754 kB/s | 6.1 MB 00:08 2024-03-07T06:35:40.079 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 15 kB 00:01 2024-03-07T06:35:41.488 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-03-07T06:35:42.319 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 880 kB/s | 19 MB 00:21 2024-03-07T06:35:46.521 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-03-07T06:35:47.503 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.1 MB 00:01 2024-03-07T06:35:48.777 INFO:teuthology.orchestra.run.smithi133.stderr:Modular dependency problems: 2024-03-07T06:35:48.777 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-03-07T06:35:48.777 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 1: conflicting requests 2024-03-07T06:35:48.777 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240227110532:20125149.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.777 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 2: conflicting requests 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 3: conflicting requests 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 4: conflicting requests 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 5: conflicting requests 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 6: conflicting requests 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 7: conflicting requests 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240222150954:49cc9d1b.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 8: conflicting requests 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 9: conflicting requests 2024-03-07T06:35:48.778 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 10: conflicting requests 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 11: conflicting requests 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 12: conflicting requests 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 13: conflicting requests 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 14: conflicting requests 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 15: conflicting requests 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 16: conflicting requests 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 17: conflicting requests 2024-03-07T06:35:48.779 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 18: conflicting requests 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 19: conflicting requests 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 20: conflicting requests 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 21: conflicting requests 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 22: conflicting requests 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 23: conflicting requests 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 24: conflicting requests 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.780 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 25: conflicting requests 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 26: conflicting requests 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 27: conflicting requests 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 28: conflicting requests 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 29: conflicting requests 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 30: conflicting requests 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 31: conflicting requests 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 32: conflicting requests 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.781 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 33: conflicting requests 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 34: conflicting requests 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 35: conflicting requests 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 36: conflicting requests 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 37: conflicting requests 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 38: cannot install the best candidate for the job 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.782 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 39: conflicting requests 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 40: cannot install the best candidate for the job 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 41: cannot install the best candidate for the job 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.783 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 42: cannot install the best candidate for the job 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 43: cannot install the best candidate for the job 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 44: cannot install the best candidate for the job 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.784 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.785 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.785 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.785 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 45: cannot install the best candidate for the job 2024-03-07T06:35:48.785 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.785 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.785 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.785 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-03-07T06:35:48.854 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 15 kB 00:00 2024-03-07T06:35:49.263 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-03-07T06:35:49.270 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-03-07T06:35:49.271 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2024-03-07T06:35:49.271 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-03-07T06:35:49.271 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2024-03-07T06:35:49.271 INFO:teuthology.orchestra.run.smithi133.stdout: linux-firmware noarch 20240219-143.el9 baseos 376 M 2024-03-07T06:35:49.271 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:35:49.271 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-03-07T06:35:49.271 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2024-03-07T06:35:49.271 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 1 Package 2024-03-07T06:35:49.271 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:35:49.271 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 376 M 2024-03-07T06:35:49.271 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-03-07T06:35:51.772 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 7.4 MB/s | 21 MB 00:02 2024-03-07T06:35:56.730 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-03-07T06:35:58.918 INFO:teuthology.orchestra.run.smithi161.stderr:Modular dependency problems: 2024-03-07T06:35:58.918 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 1: conflicting requests 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240227110532:20125149.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 2: conflicting requests 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 3: conflicting requests 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 4: conflicting requests 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 5: conflicting requests 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 6: conflicting requests 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 7: conflicting requests 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240222150954:49cc9d1b.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.919 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 8: conflicting requests 2024-03-07T06:35:58.920 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.920 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 9: conflicting requests 2024-03-07T06:35:58.920 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.920 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 10: conflicting requests 2024-03-07T06:35:58.920 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.920 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 11: conflicting requests 2024-03-07T06:35:58.920 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.920 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 12: conflicting requests 2024-03-07T06:35:58.920 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.920 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 13: conflicting requests 2024-03-07T06:35:58.920 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.920 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 14: conflicting requests 2024-03-07T06:35:58.920 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 15: conflicting requests 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 16: conflicting requests 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 17: conflicting requests 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 18: conflicting requests 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 19: conflicting requests 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 20: conflicting requests 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 21: conflicting requests 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 22: conflicting requests 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.921 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 23: conflicting requests 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 24: conflicting requests 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 25: conflicting requests 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 26: conflicting requests 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 27: conflicting requests 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 28: conflicting requests 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 29: conflicting requests 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 30: conflicting requests 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 31: conflicting requests 2024-03-07T06:35:58.922 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 32: conflicting requests 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 33: conflicting requests 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 34: conflicting requests 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 35: conflicting requests 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 36: conflicting requests 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 37: conflicting requests 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 38: cannot install the best candidate for the job 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.923 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.924 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.924 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.924 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 39: conflicting requests 2024-03-07T06:35:58.924 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.924 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.924 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.924 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.924 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.924 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.924 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 40: cannot install the best candidate for the job 2024-03-07T06:35:58.924 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 41: cannot install the best candidate for the job 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 42: cannot install the best candidate for the job 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 43: cannot install the best candidate for the job 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.925 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.926 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.926 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 44: cannot install the best candidate for the job 2024-03-07T06:35:58.926 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.926 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.926 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.926 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.926 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 45: cannot install the best candidate for the job 2024-03-07T06:35:58.926 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.926 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.926 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-03-07T06:35:58.926 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-03-07T06:35:59.423 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-03-07T06:35:59.435 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-03-07T06:35:59.436 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2024-03-07T06:35:59.436 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-03-07T06:35:59.436 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-03-07T06:35:59.436 INFO:teuthology.orchestra.run.smithi161.stdout: linux-firmware noarch 20240219-143.el9 baseos 376 M 2024-03-07T06:35:59.436 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:35:59.436 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-03-07T06:35:59.436 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2024-03-07T06:35:59.436 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 1 Package 2024-03-07T06:35:59.436 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:35:59.436 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 376 M 2024-03-07T06:35:59.436 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-03-07T06:37:18.819 INFO:teuthology.orchestra.run.smithi133.stdout:linux-firmware-20240219-143.el9.noarch.rpm 4.2 MB/s | 376 MB 01:29 2024-03-07T06:37:18.819 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-03-07T06:37:18.819 INFO:teuthology.orchestra.run.smithi133.stdout:Total 4.2 MB/s | 376 MB 01:29 2024-03-07T06:37:21.860 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-03-07T06:37:22.020 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-03-07T06:37:22.020 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-03-07T06:37:25.233 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-03-07T06:37:25.234 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-03-07T06:37:30.344 INFO:teuthology.orchestra.run.smithi161.stdout:linux-firmware-20240219-143.el9.noarch.rpm 4.1 MB/s | 376 MB 01:30 2024-03-07T06:37:30.345 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-03-07T06:37:30.345 INFO:teuthology.orchestra.run.smithi161.stdout:Total 4.1 MB/s | 376 MB 01:30 2024-03-07T06:37:32.404 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-03-07T06:37:32.762 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-07T06:37:32.844 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-07T06:37:33.378 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-03-07T06:37:33.473 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-03-07T06:37:33.473 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-03-07T06:37:33.904 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-07T06:37:33.905 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-07T06:37:34.363 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-07T06:37:34.363 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:37:34.363 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2024-03-07T06:37:34.363 INFO:teuthology.orchestra.run.smithi133.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-07T06:37:34.363 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:37:34.363 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-03-07T06:37:34.677 INFO:teuthology.task.kernel:Downloading kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.0... 2024-03-07T06:37:34.677 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-07T06:37:34.903 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ 2024-03-07T06:37:34.903 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-07T06:37:34.905 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi133.front.sepia.ceph.com to complete... 2024-03-07T06:37:36.680 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-03-07T06:37:36.681 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-03-07T06:37:43.866 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-03-07T06:37:44.250 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-07T06:37:44.339 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-07T06:37:45.300 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-07T06:37:45.300 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-07T06:37:45.758 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-07T06:37:45.759 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:37:45.759 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-03-07T06:37:45.759 INFO:teuthology.orchestra.run.smithi161.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-07T06:37:45.759 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:37:45.759 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-03-07T06:37:46.083 INFO:teuthology.task.kernel:Downloading kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.1... 2024-03-07T06:37:46.083 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 2024-03-07T06:37:46.243 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ 2024-03-07T06:37:46.243 DEBUG:teuthology.orchestra.run.smithi161:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-07T06:37:46.246 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi161.front.sepia.ceph.com to complete... 2024-03-07T06:37:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07 06:37:46 URL:https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm [336216379/336216379] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-07T06:37:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:FINISHED --2024-03-07 06:37:46-- 2024-03-07T06:37:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:Total wall clock time: 11s 2024-03-07T06:37:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:Downloaded: 1 files, 321M in 11s (28.8 MB/s) 2024-03-07T06:37:46.329 INFO:teuthology.task.kernel:Installing kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.0... 2024-03-07T06:37:46.329 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-07T06:37:48.269 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying... ######################################## 2024-03-07T06:37:48.323 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing... ######################################## 2024-03-07T06:37:48.404 INFO:teuthology.orchestra.run.smithi133.stdout:Updating / installing... 2024-03-07T06:37:55.947 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-6.8.0_rc7_gdd5c1cd17167-1 ######################################## 2024-03-07T06:38:11.143 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07 06:38:11 URL:https://2.chacra.ceph.com/r/kernel/testing/dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412/centos/9/flavors/default/x86_64/kernel-6.8.0_rc7_gdd5c1cd17167-1.x86_64.rpm [336216379/336216379] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-07T06:38:11.144 INFO:teuthology.orchestra.run.smithi161.stderr:FINISHED --2024-03-07 06:38:11-- 2024-03-07T06:38:11.144 INFO:teuthology.orchestra.run.smithi161.stderr:Total wall clock time: 25s 2024-03-07T06:38:11.144 INFO:teuthology.orchestra.run.smithi161.stderr:Downloaded: 1 files, 321M in 24s (13.1 MB/s) 2024-03-07T06:38:11.147 INFO:teuthology.task.kernel:Installing kernel dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 on client.1... 2024-03-07T06:38:11.147 DEBUG:teuthology.orchestra.run.smithi161:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-07T06:38:11.642 INFO:teuthology.orchestra.run.smithi133.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-07T06:38:13.078 INFO:teuthology.orchestra.run.smithi161.stdout:Verifying... ######################################## 2024-03-07T06:38:13.165 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing... ######################################## 2024-03-07T06:38:13.246 INFO:teuthology.orchestra.run.smithi161.stdout:Updating / installing... 2024-03-07T06:38:20.839 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-6.8.0_rc7_gdd5c1cd17167-1 ######################################## 2024-03-07T06:38:33.996 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi133.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-07T06:38:33.996 DEBUG:teuthology.orchestra.run.smithi133:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-07T06:38:34.078 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:38:34.078 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/System.map 2024-03-07T06:38:34.078 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/config 2024-03-07T06:38:34.078 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel 2024-03-07T06:38:34.078 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch 2024-03-07T06:38:34.078 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86 2024-03-07T06:38:34.078 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto 2024-03-07T06:38:34.078 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-07T06:38:34.078 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-07T06:38:34.078 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-07T06:38:34.078 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-07T06:38:34.079 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-07T06:38:34.079 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-07T06:38:34.079 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-07T06:38:34.079 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-07T06:38:34.079 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-07T06:38:34.079 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-07T06:38:34.079 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-07T06:38:34.079 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-07T06:38:34.079 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-07T06:38:34.079 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-07T06:38:34.079 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-07T06:38:34.079 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-07T06:38:34.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-07T06:38:34.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-07T06:38:34.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-07T06:38:34.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-07T06:38:34.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel 2024-03-07T06:38:34.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu 2024-03-07T06:38:34.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce 2024-03-07T06:38:34.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-07T06:38:34.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm 2024-03-07T06:38:34.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-07T06:38:34.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-07T06:38:34.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm.ko 2024-03-07T06:38:34.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto 2024-03-07T06:38:34.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ansi_cprng.ko 2024-03-07T06:38:34.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/anubis.ko 2024-03-07T06:38:34.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/arc4.ko 2024-03-07T06:38:34.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx 2024-03-07T06:38:34.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_memcpy.ko 2024-03-07T06:38:34.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_pq.ko 2024-03-07T06:38:34.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-07T06:38:34.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_tx.ko 2024-03-07T06:38:34.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_xor.ko 2024-03-07T06:38:34.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/raid6test.ko 2024-03-07T06:38:34.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authenc.ko 2024-03-07T06:38:34.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authencesn.ko 2024-03-07T06:38:34.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blake2b_generic.ko 2024-03-07T06:38:34.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_common.ko 2024-03-07T06:38:34.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_generic.ko 2024-03-07T06:38:34.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/camellia_generic.ko 2024-03-07T06:38:34.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast5_generic.ko 2024-03-07T06:38:34.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast6_generic.ko 2024-03-07T06:38:34.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast_common.ko 2024-03-07T06:38:34.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ccm.ko 2024-03-07T06:38:34.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cmac.ko 2024-03-07T06:38:34.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crc32_generic.ko 2024-03-07T06:38:34.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_engine.ko 2024-03-07T06:38:34.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_user.ko 2024-03-07T06:38:34.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cts.ko 2024-03-07T06:38:34.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/des_generic.ko 2024-03-07T06:38:34.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/dh_generic.ko 2024-03-07T06:38:34.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/echainiv.ko 2024-03-07T06:38:34.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/essiv.ko 2024-03-07T06:38:34.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/fcrypt.ko 2024-03-07T06:38:34.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/khazad.ko 2024-03-07T06:38:34.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4.ko 2024-03-07T06:38:34.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4hc.ko 2024-03-07T06:38:34.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/md4.ko 2024-03-07T06:38:34.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/michael_mic.ko 2024-03-07T06:38:34.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcbc.ko 2024-03-07T06:38:34.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcrypt.ko 2024-03-07T06:38:34.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/rmd160.ko 2024-03-07T06:38:34.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/seed.ko 2024-03-07T06:38:34.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/serpent_generic.ko 2024-03-07T06:38:34.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tcrypt.ko 2024-03-07T06:38:34.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tea.ko 2024-03-07T06:38:34.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_common.ko 2024-03-07T06:38:34.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_generic.ko 2024-03-07T06:38:34.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/vmac.ko 2024-03-07T06:38:34.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/wp512.ko 2024-03-07T06:38:34.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xcbc.ko 2024-03-07T06:38:34.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xor.ko 2024-03-07T06:38:34.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xxhash_generic.ko 2024-03-07T06:38:34.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers 2024-03-07T06:38:34.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi 2024-03-07T06:38:34.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-07T06:38:34.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_pad.ko 2024-03-07T06:38:34.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/custom_method.ko 2024-03-07T06:38:34.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/ec_sys.ko 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/platform_profile.ko 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbs.ko 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbshc.ko 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/video.ko 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/acard-ahci.ko 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ahci_platform.ko 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ata_generic.ko 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/libahci_platform.ko 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_acpi.ko 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ali.ko 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_amd.ko 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_artop.ko 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atiixp.ko 2024-03-07T06:38:34.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atp867x.ko 2024-03-07T06:38:34.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd640.ko 2024-03-07T06:38:34.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd64x.ko 2024-03-07T06:38:34.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cypress.ko 2024-03-07T06:38:34.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_efar.ko 2024-03-07T06:38:34.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt366.ko 2024-03-07T06:38:34.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt37x.ko 2024-03-07T06:38:34.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-07T06:38:34.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-07T06:38:34.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it8213.ko 2024-03-07T06:38:34.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it821x.ko 2024-03-07T06:38:34.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_jmicron.ko 2024-03-07T06:38:34.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_marvell.ko 2024-03-07T06:38:34.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_mpiix.ko 2024-03-07T06:38:34.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_netcell.ko 2024-03-07T06:38:34.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ninja32.ko 2024-03-07T06:38:34.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87410.ko 2024-03-07T06:38:34.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87415.ko 2024-03-07T06:38:34.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_oldpiix.ko 2024-03-07T06:38:34.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_opti.ko 2024-03-07T06:38:34.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_optidma.ko 2024-03-07T06:38:34.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pcmcia.ko 2024-03-07T06:38:34.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-07T06:38:34.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-07T06:38:34.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_piccolo.ko 2024-03-07T06:38:34.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_rdc.ko 2024-03-07T06:38:34.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sch.ko 2024-03-07T06:38:34.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_serverworks.ko 2024-03-07T06:38:34.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sil680.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sis.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sl82c105.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_triflex.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_via.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pdc_adma.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_inic162x.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_mv.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_nv.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_promise.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_qstor.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil24.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sis.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_svw.ko 2024-03-07T06:38:34.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sx4.ko 2024-03-07T06:38:34.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_uli.ko 2024-03-07T06:38:34.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_via.ko 2024-03-07T06:38:34.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_vsc.ko 2024-03-07T06:38:34.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay 2024-03-07T06:38:34.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay/ks0108.ko 2024-03-07T06:38:34.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base 2024-03-07T06:38:34.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap 2024-03-07T06:38:34.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-07T06:38:34.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-07T06:38:34.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma 2024-03-07T06:38:34.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma/bcma.ko 2024-03-07T06:38:34.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block 2024-03-07T06:38:34.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe 2024-03-07T06:38:34.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe/aoe.ko 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/brd.ko 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd/drbd.ko 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/floppy.ko 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/loop.ko 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/nbd.ko 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk/null_blk.ko 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/pktcdvd.ko 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/rbd.ko 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/virtio_blk.ko 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback 2024-03-07T06:38:34.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-07T06:38:34.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkfront.ko 2024-03-07T06:38:34.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram 2024-03-07T06:38:34.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram/zram.ko 2024-03-07T06:38:34.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char 2024-03-07T06:38:34.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hangcheck-timer.ko 2024-03-07T06:38:34.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random 2024-03-07T06:38:34.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-07T06:38:34.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-07T06:38:34.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-07T06:38:34.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/via-rng.ko 2024-03-07T06:38:34.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-07T06:38:34.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi 2024-03-07T06:38:34.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-07T06:38:34.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-07T06:38:34.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-07T06:38:34.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-07T06:38:34.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-07T06:38:34.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-07T06:38:34.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/lp.ko 2024-03-07T06:38:34.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave 2024-03-07T06:38:34.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave/mwave.ko 2024-03-07T06:38:34.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ppdev.ko 2024-03-07T06:38:34.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tlclk.ko 2024-03-07T06:38:34.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm 2024-03-07T06:38:34.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm.ko 2024-03-07T06:38:34.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-07T06:38:34.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-07T06:38:34.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-07T06:38:34.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-07T06:38:34.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-07T06:38:34.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/uv_mmtimer.ko 2024-03-07T06:38:34.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/virtio_console.ko 2024-03-07T06:38:34.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq 2024-03-07T06:38:34.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-07T06:38:34.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-07T06:38:34.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-07T06:38:34.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-07T06:38:34.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-07T06:38:34.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-07T06:38:34.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto 2024-03-07T06:38:34.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp 2024-03-07T06:38:34.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp.ko 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-aes.ko 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-sha.ko 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca/dca.ko 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq 2024-03-07T06:38:34.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/amd64_edac.ko 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/e752x_edac.ko 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/edac_mce_amd.ko 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3000_edac.ko 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3200_edac.ko 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5100_edac.ko 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5400_edac.ko 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7300_edac.ko 2024-03-07T06:38:34.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7core_edac.ko 2024-03-07T06:38:34.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i82975x_edac.ko 2024-03-07T06:38:34.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/ie31200_edac.ko 2024-03-07T06:38:34.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/sb_edac.ko 2024-03-07T06:38:34.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/x38_edac.ko 2024-03-07T06:38:34.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware 2024-03-07T06:38:34.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/edd.ko 2024-03-07T06:38:34.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-07T06:38:34.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio 2024-03-07T06:38:34.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-ich.ko 2024-03-07T06:38:34.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-07T06:38:34.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid 2024-03-07T06:38:34.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-appleir.ko 2024-03-07T06:38:34.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-aureal.ko 2024-03-07T06:38:34.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-axff.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-dr.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elecom.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elo.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-emsff.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gaff.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gt683r.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gyration.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtekff.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-hyperv.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-icade.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-keytouch.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-kye.ko 2024-03-07T06:38:34.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lcpower.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-led.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lenovo.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-multitouch.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-ortek.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-penmount.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-petalynx.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-picolcd.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-pl.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-plantronics.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-primax.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-rmi.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-07T06:38:34.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-common.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-saitek.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-samsung.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sjoy.ko 2024-03-07T06:38:34.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sony.ko 2024-03-07T06:38:34.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-speedlink.ko 2024-03-07T06:38:34.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-steelseries.ko 2024-03-07T06:38:34.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sunplus.ko 2024-03-07T06:38:34.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-07T06:38:34.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tivo.ko 2024-03-07T06:38:34.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tmff.ko 2024-03-07T06:38:34.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-topseed.ko 2024-03-07T06:38:34.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-twinhan.ko 2024-03-07T06:38:34.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-uclogic.ko 2024-03-07T06:38:34.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-waltop.ko 2024-03-07T06:38:34.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-wiimote.ko 2024-03-07T06:38:34.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-xinmo.ko 2024-03-07T06:38:34.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zpff.ko 2024-03-07T06:38:34.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zydacron.ko 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/uhid.ko 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/wacom.ko 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_balloon.ko 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_utils.ko 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_vmbus.ko 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru.ko 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru3.ko 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7414.ko 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7418.ko 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adc128d818.ko 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1025.ko 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1026.ko 2024-03-07T06:38:34.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1029.ko 2024-03-07T06:38:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1031.ko 2024-03-07T06:38:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm9240.ko 2024-03-07T06:38:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ads7828.ko 2024-03-07T06:38:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7410.ko 2024-03-07T06:38:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7411.ko 2024-03-07T06:38:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7462.ko 2024-03-07T06:38:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7470.ko 2024-03-07T06:38:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7475.ko 2024-03-07T06:38:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7x10.ko 2024-03-07T06:38:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/amc6821.ko 2024-03-07T06:38:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/applesmc.ko 2024-03-07T06:38:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asb100.ko 2024-03-07T06:38:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asc7621.ko 2024-03-07T06:38:34.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/atxp1.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/coretemp.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dme1737.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds1621.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds620.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc1403.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc6w201.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71805f.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71882fg.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f75375s.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fam15h_power.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fschmd.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g760a.ko 2024-03-07T06:38:34.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g762.ko 2024-03-07T06:38:34.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl518sm.ko 2024-03-07T06:38:34.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl520sm.ko 2024-03-07T06:38:34.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-07T06:38:34.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5500_temp.ko 2024-03-07T06:38:34.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5k_amb.ko 2024-03-07T06:38:34.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmaem.ko 2024-03-07T06:38:34.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmpex.ko 2024-03-07T06:38:34.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina209.ko 2024-03-07T06:38:34.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina2xx.ko 2024-03-07T06:38:34.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/it87.ko 2024-03-07T06:38:34.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k10temp.ko 2024-03-07T06:38:34.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k8temp.ko 2024-03-07T06:38:34.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lineage-pem.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm63.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm73.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm75.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm77.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm78.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm80.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm83.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm85.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm87.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm90.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm92.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm93.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95234.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95241.ko 2024-03-07T06:38:34.105 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95245.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc2945.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4151.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4215.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4222.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4245.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4260.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4261.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max16065.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1619.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1668.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max197.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6639.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6650.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6697.ko 2024-03-07T06:38:34.106 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/mcp3021.ko 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6683.ko 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775-core.ko 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775.ko 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct7802.ko 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87360.ko 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87427.ko 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pcf8591.ko 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-07T06:38:34.107 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/powr1220.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5627.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5636.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht15.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht21.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/shtc1.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sis5595.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47b397.ko 2024-03-07T06:38:34.108 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m1.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m192.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/thmc50.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp102.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp103.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp401.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp421.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via-cputemp.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via686a.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt1211.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt8231.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627ehf.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627hf.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83781d.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83791d.ko 2024-03-07T06:38:34.109 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83792d.ko 2024-03-07T06:38:34.110 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83793.ko 2024-03-07T06:38:34.110 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83795.ko 2024-03-07T06:38:34.110 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l785ts.ko 2024-03-07T06:38:34.110 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l786ng.ko 2024-03-07T06:38:34.110 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c 2024-03-07T06:38:34.110 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos 2024-03-07T06:38:34.110 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-07T06:38:34.110 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-07T06:38:34.110 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses 2024-03-07T06:38:34.110 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-07T06:38:34.110 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-07T06:38:34.110 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-07T06:38:34.110 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-07T06:38:34.110 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-07T06:38:34.111 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-dev.ko 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-mux.ko 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-smbus.ko 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-stub.ko 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_core.ko 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-07T06:38:34.112 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-07T06:38:34.113 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/gameport.ko 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/joydev.ko 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard 2024-03-07T06:38:34.114 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/appletouch.ko 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/bcm5974.ko 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/cyapatp.ko 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/sermouse.ko 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/altera_ps2.ko 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/arc_ps2.ko 2024-03-07T06:38:34.115 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-07T06:38:34.116 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/serio_raw.ko 2024-03-07T06:38:34.116 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/sparse-keymap.ko 2024-03-07T06:38:34.116 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds 2024-03-07T06:38:34.116 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-blinkm.ko 2024-03-07T06:38:34.116 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lm3530.ko 2024-03-07T06:38:34.116 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lp3944.ko 2024-03-07T06:38:34.116 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-ss4200.ko 2024-03-07T06:38:34.116 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger 2024-03-07T06:38:34.116 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-07T06:38:34.116 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-07T06:38:34.116 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-07T06:38:34.116 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-07T06:38:34.116 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-07T06:38:34.116 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache/bcache.ko 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-bio-prison.ko 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache-smq.ko 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache.ko 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-crypt.ko 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-delay.ko 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-flakey.ko 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-log-userspace.ko 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-multipath.ko 2024-03-07T06:38:34.117 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-queue-length.ko 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-raid.ko 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-round-robin.ko 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-service-time.ko 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-switch.ko 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-thin-pool.ko 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-verity.ko 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid0.ko 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid1.ko 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid10.ko 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid456.ko 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion 2024-03-07T06:38:34.118 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptbase.ko 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptctl.ko 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptfc.ko 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptlan.ko 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptsas.ko 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptscsih.ko 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptspi.ko 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_ich.ko 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_sch.ko 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/mfd-core.ko 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/sm501.ko 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/viperboard.ko 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/vx855.ko 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/wl1273-core.ko 2024-03-07T06:38:34.119 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc 2024-03-07T06:38:34.120 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl 2024-03-07T06:38:34.120 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-07T06:38:34.120 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds9802als.ko 2024-03-07T06:38:34.120 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds990x.ko 2024-03-07T06:38:34.120 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/bh1770glc.ko 2024-03-07T06:38:34.120 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710 2024-03-07T06:38:34.120 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710/cb710.ko 2024-03-07T06:38:34.120 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom 2024-03-07T06:38:34.120 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/at24.ko 2024-03-07T06:38:34.120 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-07T06:38:34.120 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/max6875.ko 2024-03-07T06:38:34.120 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/enclosure.ko 2024-03-07T06:38:34.120 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/hpilo.ko 2024-03-07T06:38:34.120 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29003.ko 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29020.ko 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-me.ko 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-txe.ko 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei.ko 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-07T06:38:34.121 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_7xx1.ko 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_core.ko 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tsl2550.ko 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_balloon.ko 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding/bonding.ko 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-07T06:38:34.122 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dummy.ko 2024-03-07T06:38:34.123 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/eql.ko 2024-03-07T06:38:34.123 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet 2024-03-07T06:38:34.123 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com 2024-03-07T06:38:34.123 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-07T06:38:34.123 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-07T06:38:34.123 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-07T06:38:34.123 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-07T06:38:34.123 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390 2024-03-07T06:38:34.123 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-07T06:38:34.123 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-07T06:38:34.123 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-07T06:38:34.123 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-07T06:38:34.123 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec 2024-03-07T06:38:34.123 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-07T06:38:34.124 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-07T06:38:34.125 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-07T06:38:34.125 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-07T06:38:34.125 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx 2024-03-07T06:38:34.125 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-07T06:38:34.125 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-07T06:38:34.125 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom 2024-03-07T06:38:34.125 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-07T06:38:34.125 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-07T06:38:34.125 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-07T06:38:34.125 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-07T06:38:34.125 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-07T06:38:34.125 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet 2024-03-07T06:38:34.125 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-07T06:38:34.125 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-07T06:38:34.126 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dnet.ko 2024-03-07T06:38:34.127 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ethoc.ko 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/fealnx.ko 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-07T06:38:34.128 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/jme.ko 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-07T06:38:34.129 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-07T06:38:34.130 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-07T06:38:34.131 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek 2024-03-07T06:38:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-07T06:38:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-07T06:38:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-07T06:38:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-07T06:38:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker 2024-03-07T06:38:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-07T06:38:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc 2024-03-07T06:38:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-07T06:38:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan 2024-03-07T06:38:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-07T06:38:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis 2024-03-07T06:38:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-07T06:38:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-07T06:38:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti 2024-03-07T06:38:34.133 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/geneve.ko 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154 2024-03-07T06:38:34.134 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ifb.ko 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvlan.ko 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvtap.ko 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio.ko 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mii.ko 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/net_failover.ko 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/netconsole.ko 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/nlmon.ko 2024-03-07T06:38:34.135 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/amd.ko 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm87xx.ko 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/broadcom.ko 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/cicada.ko 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/davicom.ko 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/dp83640.ko 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/et1011c.ko 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/icplus.ko 2024-03-07T06:38:34.136 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/lxt.ko 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/marvell.ko 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/micrel.ko 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/national.ko 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/phylink.ko 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/qsemi.ko 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/realtek.ko 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/smsc.ko 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/ste10Xp.ko 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/vitesse.ko 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_async.ko 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-07T06:38:34.137 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppoe.ko 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppox.ko 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pptp.ko 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slhc.ko 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slip.ko 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/sungem_phy.ko 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tap.ko 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team.ko 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-07T06:38:34.138 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_random.ko 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tun.ko 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/veth.ko 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/virtio_net.ko 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan/vxlan.ko 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netfront.ko 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb/ntb.ko 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host 2024-03-07T06:38:34.139 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme-core.ko 2024-03-07T06:38:34.140 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme.ko 2024-03-07T06:38:34.140 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport 2024-03-07T06:38:34.140 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport.ko 2024-03-07T06:38:34.140 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_cs.ko 2024-03-07T06:38:34.140 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_pc.ko 2024-03-07T06:38:34.140 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_serial.ko 2024-03-07T06:38:34.140 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci 2024-03-07T06:38:34.140 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug 2024-03-07T06:38:34.140 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-07T06:38:34.140 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie 2024-03-07T06:38:34.140 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-07T06:38:34.140 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/xen-pcifront.ko 2024-03-07T06:38:34.140 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia 2024-03-07T06:38:34.140 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/i82092.ko 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/pd6729.ko 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acerhdf.ko 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-07T06:38:34.141 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/hdaps.ko 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel_ips.ko 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-07T06:38:34.142 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi.ko 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps_parport.ko 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/pps_core.ko 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp.ko 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp_kvm.ko 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc 2024-03-07T06:38:34.143 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-em3027.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-07T06:38:34.144 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-max6900.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-07T06:38:34.145 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-x1205.ko 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-9xxx.ko 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-sas.ko 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-xxxx.ko 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/BusLogic.ko 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/a100u2w.ko 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/advansys.ko 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/am53c974.ko 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-07T06:38:34.146 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/atp870u.ko 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa/bfa.ko 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ch.ko 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-07T06:38:34.147 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-07T06:38:34.148 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-07T06:38:34.148 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-07T06:38:34.148 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dc395x.ko 2024-03-07T06:38:34.148 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler 2024-03-07T06:38:34.148 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-07T06:38:34.148 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-07T06:38:34.148 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-07T06:38:34.148 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-07T06:38:34.148 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dmx3191d.ko 2024-03-07T06:38:34.148 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r 2024-03-07T06:38:34.148 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-07T06:38:34.148 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esp_scsi.ko 2024-03-07T06:38:34.148 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe 2024-03-07T06:38:34.148 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic/fnic.ko 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hpsa.ko 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hptiop.ko 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hv_storvsc.ko 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/initio.ko 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ipr.ko 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ips.ko 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci/isci.ko 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc 2024-03-07T06:38:34.149 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc/libfc.ko 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi.ko 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas/libsas.ko 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid.ko 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas 2024-03-07T06:38:34.150 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvumi.ko 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pmcraid.ko 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla1280.ko 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/raid_class.ko 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_debug.ko 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-07T06:38:34.151 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ses.ko 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/st.ko 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/stex.ko 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/virtio_scsi.ko 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/wd719x.ko 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb/ssb.ko 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-07T06:38:34.152 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_file.ko 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_iblock.ko 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_mod.ko 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_pscsi.ko 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_user.ko 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-07T06:38:34.153 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-07T06:38:34.154 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_gsm.ko 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_hdlc.ko 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/nozomi.ko 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/arc_uart.ko 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-07T06:38:34.155 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host 2024-03-07T06:38:34.156 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-07T06:38:34.156 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio 2024-03-07T06:38:34.156 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio.ko 2024-03-07T06:38:34.156 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_aec.ko 2024-03-07T06:38:34.156 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_cif.ko 2024-03-07T06:38:34.156 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_pci_generic.ko 2024-03-07T06:38:34.156 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_sercos3.ko 2024-03-07T06:38:34.156 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb 2024-03-07T06:38:34.156 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class 2024-03-07T06:38:34.156 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-acm.ko 2024-03-07T06:38:34.156 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-07T06:38:34.156 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usblp.ko 2024-03-07T06:38:34.156 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usbtmc.ko 2024-03-07T06:38:34.156 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host 2024-03-07T06:38:34.157 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-07T06:38:34.157 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image 2024-03-07T06:38:34.157 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/mdc800.ko 2024-03-07T06:38:34.157 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/microtek.ko 2024-03-07T06:38:34.157 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc 2024-03-07T06:38:34.157 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/adutux.ko 2024-03-07T06:38:34.157 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/appledisplay.ko 2024-03-07T06:38:34.157 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi26.ko 2024-03-07T06:38:34.157 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi62.ko 2024-03-07T06:38:34.157 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ezusb.ko 2024-03-07T06:38:34.157 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/idmouse.ko 2024-03-07T06:38:34.157 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/iowarrior.ko 2024-03-07T06:38:34.157 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-07T06:38:34.157 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ldusb.ko 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/legousbtower.ko 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usb3503.ko 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usblcd.ko 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/uss720.ko 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/yurex.ko 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy/phy-generic.ko 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/aircable.ko 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ark3116.ko 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-07T06:38:34.158 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ch341.ko 2024-03-07T06:38:34.159 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cp210x.ko 2024-03-07T06:38:34.159 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cyberjack.ko 2024-03-07T06:38:34.159 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-07T06:38:34.159 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-07T06:38:34.159 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/empeg.ko 2024-03-07T06:38:34.159 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-07T06:38:34.159 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-07T06:38:34.159 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-07T06:38:34.159 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_ti.ko 2024-03-07T06:38:34.159 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipaq.ko 2024-03-07T06:38:34.159 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipw.ko 2024-03-07T06:38:34.159 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ir-usb.ko 2024-03-07T06:38:34.159 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-07T06:38:34.159 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mct_u232.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7720.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7840.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/navman.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/omninet.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/opticon.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/option.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/oti6858.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/pl2303.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcaux.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcserial.ko 2024-03-07T06:38:34.160 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/quatech2.ko 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/safe_serial.ko 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/sierra.ko 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ssu100.ko 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/symbolserial.ko 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_debug.ko 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/visor.ko 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/whiteheat.ko 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/uas.ko 2024-03-07T06:38:34.161 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-karma.ko 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/usb-storage.ko 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip 2024-03-07T06:38:34.162 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio.ko 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost.ko 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_net.ko 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_scsi.ko 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video 2024-03-07T06:38:34.163 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/apple_bl.ko 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/lcd.ko 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio.ko 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_balloon.ko 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_mmio.ko 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci.ko 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_ring.ko 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2482.ko 2024-03-07T06:38:34.164 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2490.ko 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/wire.ko 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog 2024-03-07T06:38:34.165 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/hpwdt.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/i6300esb.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ib700wdt.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ibmasr.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it87_wdt.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/machzwd.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/nv_tco.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-07T06:38:34.166 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/softdog.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/via_wdt.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/wdt_pci.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/xen_wdt.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-evtchn.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntalloc.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntdev.ko 2024-03-07T06:38:34.167 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback 2024-03-07T06:38:34.168 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-07T06:38:34.168 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-privcmd.ko 2024-03-07T06:38:34.168 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs 2024-03-07T06:38:34.168 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-07T06:38:34.168 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs 2024-03-07T06:38:34.168 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p 2024-03-07T06:38:34.168 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p/9p.ko 2024-03-07T06:38:34.168 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs 2024-03-07T06:38:34.168 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs/affs.ko 2024-03-07T06:38:34.168 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs 2024-03-07T06:38:34.168 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs/befs.ko 2024-03-07T06:38:34.168 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/binfmt_misc.ko 2024-03-07T06:38:34.168 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs 2024-03-07T06:38:34.168 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs/btrfs.ko 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles/cachefiles.ko 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph/ceph.ko 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda/coda.ko 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs/cramfs.ko 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm/dlm.ko 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs/f2fs.ko 2024-03-07T06:38:34.169 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/fat.ko 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/msdos.ko 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/vfat.ko 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/cuse.ko 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/fuse.ko 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2/gfs2.ko 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs/hfs.ko 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus/hfsplus.ko 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs/isofs.ko 2024-03-07T06:38:34.170 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs/jfs.ko 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd/lockd.ko 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix/minix.ko 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs/netfs.ko 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-07T06:38:34.171 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfs.ko 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv3.ko 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv4.ko 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/grace.ko 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/nfs_acl.ko 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd/nfsd.ko 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2/nilfs2.ko 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-celtic.ko 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-centeuro.ko 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-croatian.ko 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-cyrillic.ko 2024-03-07T06:38:34.172 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-gaelic.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-greek.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-iceland.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-inuit.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-roman.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-romanian.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-turkish.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1250.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1251.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1255.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp737.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp775.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp850.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp852.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp855.ko 2024-03-07T06:38:34.173 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp857.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp860.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp861.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp862.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp863.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp864.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp865.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp866.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp869.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp874.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp932.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp936.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp949.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp950.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_euc-jp.ko 2024-03-07T06:38:34.174 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-1.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-13.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-14.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-15.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-2.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-3.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-4.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-5.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-6.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-7.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-9.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-r.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-ru.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-u.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-07T06:38:34.175 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_utf8.ko 2024-03-07T06:38:34.176 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2 2024-03-07T06:38:34.176 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster 2024-03-07T06:38:34.176 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-07T06:38:34.176 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm 2024-03-07T06:38:34.176 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-07T06:38:34.176 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs 2024-03-07T06:38:34.176 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-07T06:38:34.176 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2.ko 2024-03-07T06:38:34.176 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-07T06:38:34.176 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-07T06:38:34.176 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-07T06:38:34.176 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs 2024-03-07T06:38:34.176 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs/overlay.ko 2024-03-07T06:38:34.176 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore/ramoops.ko 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs/reiserfs.ko 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs/romfs.ko 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client/cifs.ko 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_arc4.ko 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_md4.ko 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs/squashfs.ko 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv/sysv.ko 2024-03-07T06:38:34.177 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf/udf.ko 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs/ufs.ko 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs/xfs.ko 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu/rcutorture.ko 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/torture.ko 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/asn1_encoder.ko 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc-itu-t.ko 2024-03-07T06:38:34.178 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc8.ko 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libarc4.ko 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libdes.ko 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/libcrc32c.ko 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lru_cache.ko 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4_compress.ko 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4hc_compress.ko 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math/cordic.ko 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6/raid6_pq.ko 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-07T06:38:34.179 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_bm.ko 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_fsm.ko 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_kmp.ko 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd/zstd_compress.ko 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm/hwpoison-inject.ko 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/6lowpan.ko 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_dest.ko 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_fragment.ko 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_hop.ko 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_mobility.ko 2024-03-07T06:38:34.180 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_routing.ko 2024-03-07T06:38:34.181 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_udp.ko 2024-03-07T06:38:34.181 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802 2024-03-07T06:38:34.181 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/garp.ko 2024-03-07T06:38:34.181 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/mrp.ko 2024-03-07T06:38:34.181 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/p8022.ko 2024-03-07T06:38:34.181 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/psnap.ko 2024-03-07T06:38:34.181 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/stp.ko 2024-03-07T06:38:34.181 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q 2024-03-07T06:38:34.181 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q/8021q.ko 2024-03-07T06:38:34.181 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p 2024-03-07T06:38:34.181 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet.ko 2024-03-07T06:38:34.181 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_fd.ko 2024-03-07T06:38:34.181 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_rdma.ko 2024-03-07T06:38:34.181 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_virtio.ko 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv/batman-adv.ko 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/br_netfilter.ko 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/bridge.ko 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-07T06:38:34.182 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtables.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-07T06:38:34.183 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph/libceph.ko 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/failover.ko 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/pktgen.ko 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp.ko 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_diag.ko 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv4.ko 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv6.ko 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver/dns_resolver.ko 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/dsa_core.ko 2024-03-07T06:38:34.184 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_brcm.ko 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_dsa.ko 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_none.ko 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_trailer.ko 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154.ko 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ah4.ko 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/esp4.ko 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/fou.ko 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/gre.ko 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/inet_diag.ko 2024-03-07T06:38:34.185 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_gre.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_tunnel.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_vti.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipcomp.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipip.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-07T06:38:34.186 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_bic.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_dctcp.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_diag.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_highspeed.ko 2024-03-07T06:38:34.187 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_htcp.ko 2024-03-07T06:38:34.188 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_hybla.ko 2024-03-07T06:38:34.188 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_illinois.ko 2024-03-07T06:38:34.188 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_lp.ko 2024-03-07T06:38:34.188 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_scalable.ko 2024-03-07T06:38:34.188 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_vegas.ko 2024-03-07T06:38:34.188 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_veno.ko 2024-03-07T06:38:34.188 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_westwood.ko 2024-03-07T06:38:34.188 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_yeah.ko 2024-03-07T06:38:34.188 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tunnel4.ko 2024-03-07T06:38:34.188 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_diag.ko 2024-03-07T06:38:34.188 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_tunnel.ko 2024-03-07T06:38:34.188 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-07T06:38:34.188 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6 2024-03-07T06:38:34.188 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ah6.ko 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/esp6.ko 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/fou6.ko 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_tunnel.ko 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_vti.ko 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ipcomp6.ko 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-07T06:38:34.189 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-07T06:38:34.190 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/sit.ko 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/tunnel6.ko 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key/af_key.ko 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_core.ko 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_eth.ko 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip.ko 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip6.ko 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_netlink.ko 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ppp.ko 2024-03-07T06:38:34.191 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc 2024-03-07T06:38:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc/llc.ko 2024-03-07T06:38:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154 2024-03-07T06:38:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154/mac802154.ko 2024-03-07T06:38:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls 2024-03-07T06:38:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls/mpls_gso.ko 2024-03-07T06:38:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter 2024-03-07T06:38:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset 2024-03-07T06:38:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set.ko 2024-03-07T06:38:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-07T06:38:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-07T06:38:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-07T06:38:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-07T06:38:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-07T06:38:34.192 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-07T06:38:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conncount.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-07T06:38:34.194 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_log_syslog.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-07T06:38:34.195 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_irc.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_sip.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_tables.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_log.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_chain_nat.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_compat.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_ct.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-07T06:38:34.196 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fib.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_hash.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_limit.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_log.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_masq.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_nat.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_numgen.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_queue.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_quota.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_redir.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject_inet.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_socket.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_tunnel.ko 2024-03-07T06:38:34.197 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_AUDIT.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CT.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_DSCP.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HL.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HMARK.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LED.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LOG.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NETMAP.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFLOG.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-07T06:38:34.198 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_RATEEST.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_SECMARK.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TEE.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TPROXY.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TRACE.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_addrtype.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_bpf.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cgroup.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cluster.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_comment.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connbytes.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlabel.ko 2024-03-07T06:38:34.199 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlimit.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connmark.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_conntrack.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cpu.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dccp.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_devgroup.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dscp.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ecn.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_esp.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hashlimit.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_helper.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hl.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipcomp.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_iprange.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipvs.ko 2024-03-07T06:38:34.200 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_l2tp.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_length.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_limit.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mac.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mark.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_multiport.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nat.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nfacct.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_osf.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_owner.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_physdev.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_pkttype.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_policy.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_quota.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_rateest.ko 2024-03-07T06:38:34.201 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_realm.ko 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_recent.ko 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_sctp.ko 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_set.ko 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_socket.ko 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_state.ko 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_statistic.ko 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_string.ko 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_tcpmss.ko 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_time.ko 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_u32.ko 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink/netlink_diag.ko 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh/nsh.ko 2024-03-07T06:38:34.202 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/openvswitch.ko 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-geneve.ko 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-gre.ko 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-vxlan.ko 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet/af_packet_diag.ko 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds.ko 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_rdma.ko 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_tcp.ko 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_csum.ko 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_gact.ko 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_mirred.ko 2024-03-07T06:38:34.203 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_nat.ko 2024-03-07T06:38:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_pedit.ko 2024-03-07T06:38:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_police.ko 2024-03-07T06:38:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_simple.ko 2024-03-07T06:38:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_skbedit.ko 2024-03-07T06:38:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_vlan.ko 2024-03-07T06:38:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_basic.ko 2024-03-07T06:38:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_bpf.ko 2024-03-07T06:38:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_flow.ko 2024-03-07T06:38:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_fw.ko 2024-03-07T06:38:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_route.ko 2024-03-07T06:38:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_u32.ko 2024-03-07T06:38:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_cmp.ko 2024-03-07T06:38:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_ipset.ko 2024-03-07T06:38:34.204 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_meta.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_nbyte.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_text.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_u32.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_choke.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_codel.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_drr.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_fq.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_gred.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hfsc.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hhf.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_htb.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_ingress.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio_lib.ko 2024-03-07T06:38:34.205 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_multiq.ko 2024-03-07T06:38:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_netem.ko 2024-03-07T06:38:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_pie.ko 2024-03-07T06:38:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_plug.ko 2024-03-07T06:38:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_prio.ko 2024-03-07T06:38:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_qfq.ko 2024-03-07T06:38:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_red.ko 2024-03-07T06:38:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfb.ko 2024-03-07T06:38:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfq.ko 2024-03-07T06:38:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_tbf.ko 2024-03-07T06:38:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_teql.ko 2024-03-07T06:38:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp 2024-03-07T06:38:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp.ko 2024-03-07T06:38:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp_diag.ko 2024-03-07T06:38:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/sunrpc.ko 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/diag.ko 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/tipc.ko 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix/unix_diag.ko 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-07T06:38:34.207 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock.ko 2024-03-07T06:38:34.208 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-07T06:38:34.208 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-07T06:38:34.208 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm 2024-03-07T06:38:34.208 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-07T06:38:34.208 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security 2024-03-07T06:38:34.208 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys 2024-03-07T06:38:34.208 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys 2024-03-07T06:38:34.208 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-07T06:38:34.208 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys 2024-03-07T06:38:34.208 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys/trusted.ko 2024-03-07T06:38:34.208 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt 2024-03-07T06:38:34.208 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib 2024-03-07T06:38:34.208 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib/irqbypass.ko 2024-03-07T06:38:34.208 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias 2024-03-07T06:38:34.209 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias.bin 2024-03-07T06:38:34.209 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin 2024-03-07T06:38:34.209 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.alias.bin 2024-03-07T06:38:34.209 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.bin 2024-03-07T06:38:34.209 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.modinfo 2024-03-07T06:38:34.209 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep 2024-03-07T06:38:34.209 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep.bin 2024-03-07T06:38:34.209 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.devname 2024-03-07T06:38:34.209 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.order 2024-03-07T06:38:34.209 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.softdep 2024-03-07T06:38:34.209 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols 2024-03-07T06:38:34.209 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols.bin 2024-03-07T06:38:34.209 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-07T06:38:34.213 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:38:34.214 DEBUG:teuthology.orchestra.run.smithi133:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-07T06:38:34.235 INFO:teuthology.orchestra.run.smithi133.stdout:postinstall scriptlet (using /bin/sh): 2024-03-07T06:38:34.235 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-07T06:38:34.235 INFO:teuthology.orchestra.run.smithi133.stdout: /usr/bin/kernel-install add 6.8.0-rc7-gdd5c1cd17167 /lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-07T06:38:34.235 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2024-03-07T06:38:34.235 INFO:teuthology.orchestra.run.smithi133.stdout:for file in vmlinuz System.map config; do 2024-03-07T06:38:34.235 INFO:teuthology.orchestra.run.smithi133.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167"; then 2024-03-07T06:38:34.235 INFO:teuthology.orchestra.run.smithi133.stdout: cp "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167" 2024-03-07T06:38:34.235 INFO:teuthology.orchestra.run.smithi133.stdout: fi 2024-03-07T06:38:34.235 INFO:teuthology.orchestra.run.smithi133.stdout:done 2024-03-07T06:38:34.235 INFO:teuthology.orchestra.run.smithi133.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-07T06:38:34.235 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-07T06:38:34.236 INFO:teuthology.orchestra.run.smithi133.stdout:new-kernel-pkg --remove 6.8.0-rc7-gdd5c1cd17167 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-gdd5c1cd17167.img 2024-03-07T06:38:34.236 INFO:teuthology.orchestra.run.smithi133.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-07T06:38:34.236 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-install remove 6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:38:34.236 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2024-03-07T06:38:34.236 INFO:teuthology.orchestra.run.smithi133.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-07T06:38:34.236 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-07T06:38:34.236 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:38:34.236 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2024-03-07T06:38:34.237 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -qi grub2-tools 2024-03-07T06:38:34.311 INFO:teuthology.orchestra.run.smithi133.stdout:Name : grub2-tools 2024-03-07T06:38:34.312 INFO:teuthology.orchestra.run.smithi133.stdout:Epoch : 1 2024-03-07T06:38:34.312 INFO:teuthology.orchestra.run.smithi133.stdout:Version : 2.06 2024-03-07T06:38:34.312 INFO:teuthology.orchestra.run.smithi133.stdout:Release : 68.el9 2024-03-07T06:38:34.312 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: x86_64 2024-03-07T06:38:34.312 INFO:teuthology.orchestra.run.smithi133.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-07T06:38:34.312 INFO:teuthology.orchestra.run.smithi133.stdout:Group : Unspecified 2024-03-07T06:38:34.312 INFO:teuthology.orchestra.run.smithi133.stdout:Size : 8264254 2024-03-07T06:38:34.312 INFO:teuthology.orchestra.run.smithi133.stdout:License : GPLv3+ 2024-03-07T06:38:34.312 INFO:teuthology.orchestra.run.smithi133.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-07T06:38:34.312 INFO:teuthology.orchestra.run.smithi133.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-07T06:38:34.312 INFO:teuthology.orchestra.run.smithi133.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-07T06:38:34.312 INFO:teuthology.orchestra.run.smithi133.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-07T06:38:34.313 INFO:teuthology.orchestra.run.smithi133.stdout:Packager : builder@centos.org 2024-03-07T06:38:34.313 INFO:teuthology.orchestra.run.smithi133.stdout:Vendor : CentOS 2024-03-07T06:38:34.313 INFO:teuthology.orchestra.run.smithi133.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-07T06:38:34.313 INFO:teuthology.orchestra.run.smithi133.stdout:Summary : Support tools for GRUB. 2024-03-07T06:38:34.313 INFO:teuthology.orchestra.run.smithi133.stdout:Description : 2024-03-07T06:38:34.313 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:38:34.313 INFO:teuthology.orchestra.run.smithi133.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-07T06:38:34.313 INFO:teuthology.orchestra.run.smithi133.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-07T06:38:34.313 INFO:teuthology.orchestra.run.smithi133.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-07T06:38:34.313 INFO:teuthology.orchestra.run.smithi133.stdout:hardware devices. 2024-03-07T06:38:34.313 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:38:34.313 INFO:teuthology.orchestra.run.smithi133.stdout:This subpackage provides tools for support of all platforms. 2024-03-07T06:38:34.314 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-07T06:38:34.315 INFO:teuthology.task.kernel:Updating grub on smithi133 to boot 6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:38:34.315 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-07T06:38:34.991 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2024-03-07T06:38:36.335 INFO:teuthology.orchestra.run.smithi133.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-07T06:38:36.375 INFO:teuthology.orchestra.run.smithi133.stderr:done 2024-03-07T06:38:36.381 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-03-07T06:38:36.396 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.j0cCEsm2hA 2024-03-07T06:38:36.397 DEBUG:teuthology.orchestra.run.smithi133:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.j0cCEsm2hA 2024-03-07T06:38:36.399 INFO:teuthology.orchestra.run.smithi161.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-07T06:38:36.465 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.j0cCEsm2hA 2024-03-07T06:38:36.607 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.j0cCEsm2hA is 6KB 2024-03-07T06:38:36.655 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.j0cCEsm2hA 2024-03-07T06:38:36.669 DEBUG:teuthology.orchestra.run.smithi133:> sudo /bin/ls /boot/loader/entries || true 2024-03-07T06:38:36.734 INFO:teuthology.orchestra.run.smithi133.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-07T06:38:36.734 INFO:teuthology.orchestra.run.smithi133.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-07T06:38:36.734 INFO:teuthology.orchestra.run.smithi133.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-07T06:38:36.734 INFO:teuthology.orchestra.run.smithi133.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167.conf 2024-03-07T06:38:36.735 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:38:36.908 DEBUG:teuthology.orchestra.run.smithi133:> sudo shutdown -r now 2024-03-07T06:38:58.830 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi161.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-07T06:38:58.830 DEBUG:teuthology.orchestra.run.smithi161:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-07T06:38:58.905 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:38:58.905 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/System.map 2024-03-07T06:38:58.905 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/config 2024-03-07T06:38:58.905 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel 2024-03-07T06:38:58.905 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch 2024-03-07T06:38:58.905 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86 2024-03-07T06:38:58.905 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto 2024-03-07T06:38:58.905 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-07T06:38:58.906 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/arch/x86/kvm/kvm.ko 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ansi_cprng.ko 2024-03-07T06:38:58.907 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/anubis.ko 2024-03-07T06:38:58.908 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/arc4.ko 2024-03-07T06:38:58.908 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx 2024-03-07T06:38:58.908 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_memcpy.ko 2024-03-07T06:38:58.908 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_pq.ko 2024-03-07T06:38:58.908 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-07T06:38:58.908 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_tx.ko 2024-03-07T06:38:58.908 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/async_xor.ko 2024-03-07T06:38:58.908 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/async_tx/raid6test.ko 2024-03-07T06:38:58.908 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authenc.ko 2024-03-07T06:38:58.908 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/authencesn.ko 2024-03-07T06:38:58.908 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blake2b_generic.ko 2024-03-07T06:38:58.908 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_common.ko 2024-03-07T06:38:58.908 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/blowfish_generic.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/camellia_generic.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast5_generic.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast6_generic.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cast_common.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/ccm.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cmac.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crc32_generic.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_engine.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/crypto_user.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/cts.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/des_generic.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/dh_generic.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/echainiv.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/essiv.ko 2024-03-07T06:38:58.909 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/fcrypt.ko 2024-03-07T06:38:58.910 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/khazad.ko 2024-03-07T06:38:58.910 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4.ko 2024-03-07T06:38:58.910 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/lz4hc.ko 2024-03-07T06:38:58.910 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/md4.ko 2024-03-07T06:38:58.910 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/michael_mic.ko 2024-03-07T06:38:58.910 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcbc.ko 2024-03-07T06:38:58.910 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/pcrypt.ko 2024-03-07T06:38:58.910 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/rmd160.ko 2024-03-07T06:38:58.910 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/seed.ko 2024-03-07T06:38:58.910 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/serpent_generic.ko 2024-03-07T06:38:58.910 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tcrypt.ko 2024-03-07T06:38:58.910 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/tea.ko 2024-03-07T06:38:58.910 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_common.ko 2024-03-07T06:38:58.910 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/twofish_generic.ko 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/vmac.ko 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/wp512.ko 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xcbc.ko 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xor.ko 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/crypto/xxhash_generic.ko 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/acpi_pad.ko 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/custom_method.ko 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/ec_sys.ko 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/platform_profile.ko 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbs.ko 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/sbshc.ko 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/acpi/video.ko 2024-03-07T06:38:58.911 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata 2024-03-07T06:38:58.912 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/acard-ahci.ko 2024-03-07T06:38:58.912 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ahci_platform.ko 2024-03-07T06:38:58.912 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/ata_generic.ko 2024-03-07T06:38:58.912 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/libahci_platform.ko 2024-03-07T06:38:58.912 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_acpi.ko 2024-03-07T06:38:58.912 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ali.ko 2024-03-07T06:38:58.912 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_amd.ko 2024-03-07T06:38:58.912 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_artop.ko 2024-03-07T06:38:58.912 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atiixp.ko 2024-03-07T06:38:58.912 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_atp867x.ko 2024-03-07T06:38:58.912 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd640.ko 2024-03-07T06:38:58.912 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cmd64x.ko 2024-03-07T06:38:58.912 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_cypress.ko 2024-03-07T06:38:58.912 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_efar.ko 2024-03-07T06:38:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt366.ko 2024-03-07T06:38:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt37x.ko 2024-03-07T06:38:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-07T06:38:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-07T06:38:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it8213.ko 2024-03-07T06:38:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_it821x.ko 2024-03-07T06:38:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_jmicron.ko 2024-03-07T06:38:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_marvell.ko 2024-03-07T06:38:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_mpiix.ko 2024-03-07T06:38:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_netcell.ko 2024-03-07T06:38:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ninja32.ko 2024-03-07T06:38:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87410.ko 2024-03-07T06:38:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_ns87415.ko 2024-03-07T06:38:58.913 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_oldpiix.ko 2024-03-07T06:38:58.914 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_opti.ko 2024-03-07T06:38:58.914 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_optidma.ko 2024-03-07T06:38:58.914 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pcmcia.ko 2024-03-07T06:38:58.914 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-07T06:38:58.914 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-07T06:38:58.914 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_piccolo.ko 2024-03-07T06:38:58.914 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_rdc.ko 2024-03-07T06:38:58.914 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sch.ko 2024-03-07T06:38:58.914 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_serverworks.ko 2024-03-07T06:38:58.914 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sil680.ko 2024-03-07T06:38:58.914 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sis.ko 2024-03-07T06:38:58.914 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_sl82c105.ko 2024-03-07T06:38:58.914 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_triflex.ko 2024-03-07T06:38:58.914 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pata_via.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/pdc_adma.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_inic162x.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_mv.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_nv.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_promise.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_qstor.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sil24.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sis.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_svw.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_sx4.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_uli.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_via.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ata/sata_vsc.ko 2024-03-07T06:38:58.915 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay 2024-03-07T06:38:58.916 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/auxdisplay/ks0108.ko 2024-03-07T06:38:58.916 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base 2024-03-07T06:38:58.916 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap 2024-03-07T06:38:58.916 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-07T06:38:58.916 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-07T06:38:58.916 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma 2024-03-07T06:38:58.916 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/bcma/bcma.ko 2024-03-07T06:38:58.916 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block 2024-03-07T06:38:58.916 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe 2024-03-07T06:38:58.916 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/aoe/aoe.ko 2024-03-07T06:38:58.916 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/brd.ko 2024-03-07T06:38:58.916 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd 2024-03-07T06:38:58.916 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/drbd/drbd.ko 2024-03-07T06:38:58.916 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/floppy.ko 2024-03-07T06:38:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/loop.ko 2024-03-07T06:38:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx 2024-03-07T06:38:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-07T06:38:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/nbd.ko 2024-03-07T06:38:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk 2024-03-07T06:38:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/null_blk/null_blk.ko 2024-03-07T06:38:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/pktcdvd.ko 2024-03-07T06:38:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/rbd.ko 2024-03-07T06:38:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/virtio_blk.ko 2024-03-07T06:38:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback 2024-03-07T06:38:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-07T06:38:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/xen-blkfront.ko 2024-03-07T06:38:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram 2024-03-07T06:38:58.917 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/block/zram/zram.ko 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hangcheck-timer.ko 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/via-rng.ko 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-07T06:38:58.918 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-07T06:38:58.919 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/lp.ko 2024-03-07T06:38:58.919 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave 2024-03-07T06:38:58.919 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/mwave/mwave.ko 2024-03-07T06:38:58.919 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/ppdev.ko 2024-03-07T06:38:58.919 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tlclk.ko 2024-03-07T06:38:58.919 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm 2024-03-07T06:38:58.919 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm.ko 2024-03-07T06:38:58.919 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-07T06:38:58.919 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-07T06:38:58.919 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-07T06:38:58.919 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-07T06:38:58.919 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-07T06:38:58.919 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/uv_mmtimer.ko 2024-03-07T06:38:58.919 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/char/virtio_console.ko 2024-03-07T06:38:58.920 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq 2024-03-07T06:38:58.920 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-07T06:38:58.920 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-07T06:38:58.920 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-07T06:38:58.920 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-07T06:38:58.920 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-07T06:38:58.920 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-07T06:38:58.920 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto 2024-03-07T06:38:58.920 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp 2024-03-07T06:38:58.920 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-07T06:38:58.920 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/ccp/ccp.ko 2024-03-07T06:38:58.920 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel 2024-03-07T06:38:58.920 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-aes.ko 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/padlock-sha.ko 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dca/dca.ko 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw 2024-03-07T06:38:58.921 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-07T06:38:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat 2024-03-07T06:38:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-07T06:38:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac 2024-03-07T06:38:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/amd64_edac.ko 2024-03-07T06:38:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/e752x_edac.ko 2024-03-07T06:38:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/edac_mce_amd.ko 2024-03-07T06:38:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3000_edac.ko 2024-03-07T06:38:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i3200_edac.ko 2024-03-07T06:38:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5100_edac.ko 2024-03-07T06:38:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i5400_edac.ko 2024-03-07T06:38:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7300_edac.ko 2024-03-07T06:38:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i7core_edac.ko 2024-03-07T06:38:58.922 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/i82975x_edac.ko 2024-03-07T06:38:58.923 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/ie31200_edac.ko 2024-03-07T06:38:58.923 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/sb_edac.ko 2024-03-07T06:38:58.923 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/edac/x38_edac.ko 2024-03-07T06:38:58.923 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware 2024-03-07T06:38:58.923 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/edd.ko 2024-03-07T06:38:58.923 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-07T06:38:58.923 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio 2024-03-07T06:38:58.923 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-ich.ko 2024-03-07T06:38:58.923 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-07T06:38:58.923 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid 2024-03-07T06:38:58.923 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-appleir.ko 2024-03-07T06:38:58.923 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-aureal.ko 2024-03-07T06:38:58.923 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-axff.ko 2024-03-07T06:38:58.923 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-dr.ko 2024-03-07T06:38:58.924 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elecom.ko 2024-03-07T06:38:58.924 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-elo.ko 2024-03-07T06:38:58.924 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-emsff.ko 2024-03-07T06:38:58.924 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gaff.ko 2024-03-07T06:38:58.924 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gt683r.ko 2024-03-07T06:38:58.924 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-gyration.ko 2024-03-07T06:38:58.924 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-07T06:38:58.924 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-07T06:38:58.924 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-holtekff.ko 2024-03-07T06:38:58.924 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-hyperv.ko 2024-03-07T06:38:58.924 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-icade.ko 2024-03-07T06:38:58.924 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-keytouch.ko 2024-03-07T06:38:58.924 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-kye.ko 2024-03-07T06:38:58.924 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lcpower.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-led.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-lenovo.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-multitouch.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-ortek.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-penmount.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-petalynx.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-picolcd.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-pl.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-plantronics.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-primax.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-rmi.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-07T06:38:58.925 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-common.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-roccat.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-saitek.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-samsung.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sjoy.ko 2024-03-07T06:38:58.926 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sony.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-speedlink.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-steelseries.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-sunplus.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tivo.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-tmff.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-topseed.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-twinhan.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-uclogic.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-waltop.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-wiimote.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-xinmo.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zpff.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/hid-zydacron.ko 2024-03-07T06:38:58.927 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/uhid.ko 2024-03-07T06:38:58.928 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hid/wacom.ko 2024-03-07T06:38:58.928 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv 2024-03-07T06:38:58.928 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_balloon.ko 2024-03-07T06:38:58.928 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_utils.ko 2024-03-07T06:38:58.928 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hv/hv_vmbus.ko 2024-03-07T06:38:58.928 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon 2024-03-07T06:38:58.928 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru.ko 2024-03-07T06:38:58.928 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/abituguru3.ko 2024-03-07T06:38:58.928 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-07T06:38:58.928 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7414.ko 2024-03-07T06:38:58.928 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ad7418.ko 2024-03-07T06:38:58.928 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adc128d818.ko 2024-03-07T06:38:58.928 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1025.ko 2024-03-07T06:38:58.928 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1026.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1029.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm1031.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adm9240.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ads7828.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7410.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7411.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7462.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7470.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7475.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/adt7x10.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/amc6821.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/applesmc.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asb100.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asc7621.ko 2024-03-07T06:38:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/atxp1.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/coretemp.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/dme1737.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds1621.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ds620.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc1403.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/emc6w201.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71805f.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f71882fg.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/f75375s.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fam15h_power.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/fschmd.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g760a.ko 2024-03-07T06:38:58.930 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/g762.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl518sm.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/gl520sm.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5500_temp.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/i5k_amb.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmaem.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ibmpex.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina209.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ina2xx.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/it87.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k10temp.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/k8temp.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lineage-pem.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm63.ko 2024-03-07T06:38:58.931 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm73.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm75.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm77.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm78.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm80.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm83.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm85.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm87.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm90.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm92.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm93.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95234.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95241.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/lm95245.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc2945.ko 2024-03-07T06:38:58.932 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4151.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4215.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4222.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4245.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4260.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/ltc4261.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max16065.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1619.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max1668.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max197.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6639.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6650.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/max6697.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/mcp3021.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6683.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775-core.ko 2024-03-07T06:38:58.933 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct6775.ko 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/nct7802.ko 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87360.ko 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pc87427.ko 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pcf8591.ko 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-07T06:38:58.934 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/powr1220.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5627.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch5636.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht15.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sht21.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/shtc1.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/sis5595.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47b397.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m1.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/smsc47m192.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/thmc50.ko 2024-03-07T06:38:58.935 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp102.ko 2024-03-07T06:38:58.936 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp103.ko 2024-03-07T06:38:58.936 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp401.ko 2024-03-07T06:38:58.936 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/tmp421.ko 2024-03-07T06:38:58.936 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via-cputemp.ko 2024-03-07T06:38:58.936 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/via686a.ko 2024-03-07T06:38:58.936 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt1211.ko 2024-03-07T06:38:58.936 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/vt8231.ko 2024-03-07T06:38:58.936 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627ehf.ko 2024-03-07T06:38:58.936 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83627hf.ko 2024-03-07T06:38:58.936 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83781d.ko 2024-03-07T06:38:58.936 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83791d.ko 2024-03-07T06:38:58.936 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83792d.ko 2024-03-07T06:38:58.936 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83793.ko 2024-03-07T06:38:58.936 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83795.ko 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l785ts.ko 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/hwmon/w83l786ng.ko 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-07T06:38:58.937 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-07T06:38:58.938 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-dev.ko 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-mux.ko 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-smbus.ko 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/i2c/i2c-stub.ko 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_core.ko 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-07T06:38:58.939 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw 2024-03-07T06:38:58.940 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4 2024-03-07T06:38:58.940 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-07T06:38:58.940 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4 2024-03-07T06:38:58.940 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-07T06:38:58.940 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5 2024-03-07T06:38:58.940 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-07T06:38:58.940 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca 2024-03-07T06:38:58.940 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-07T06:38:58.940 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp 2024-03-07T06:38:58.940 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib 2024-03-07T06:38:58.940 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-07T06:38:58.940 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser 2024-03-07T06:38:58.940 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/gameport/gameport.ko 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/joydev.ko 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse 2024-03-07T06:38:58.941 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/appletouch.ko 2024-03-07T06:38:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/bcm5974.ko 2024-03-07T06:38:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/cyapatp.ko 2024-03-07T06:38:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-07T06:38:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/sermouse.ko 2024-03-07T06:38:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-07T06:38:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-07T06:38:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-07T06:38:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4 2024-03-07T06:38:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-07T06:38:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio 2024-03-07T06:38:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/altera_ps2.ko 2024-03-07T06:38:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/arc_ps2.ko 2024-03-07T06:38:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-07T06:38:58.942 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/serio/serio_raw.ko 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/input/sparse-keymap.ko 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-blinkm.ko 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lm3530.ko 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-lp3944.ko 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/leds-ss4200.ko 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-07T06:38:58.943 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-07T06:38:58.944 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-07T06:38:58.944 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md 2024-03-07T06:38:58.944 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache 2024-03-07T06:38:58.944 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/bcache/bcache.ko 2024-03-07T06:38:58.944 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-bio-prison.ko 2024-03-07T06:38:58.944 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache-smq.ko 2024-03-07T06:38:58.944 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-cache.ko 2024-03-07T06:38:58.944 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-crypt.ko 2024-03-07T06:38:58.944 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-delay.ko 2024-03-07T06:38:58.944 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-flakey.ko 2024-03-07T06:38:58.944 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-log-userspace.ko 2024-03-07T06:38:58.944 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-multipath.ko 2024-03-07T06:38:58.944 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-queue-length.ko 2024-03-07T06:38:58.944 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-raid.ko 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-round-robin.ko 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-service-time.ko 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-switch.ko 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-thin-pool.ko 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/dm-verity.ko 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid0.ko 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid1.ko 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid10.ko 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/md/raid456.ko 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptbase.ko 2024-03-07T06:38:58.945 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptctl.ko 2024-03-07T06:38:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptfc.ko 2024-03-07T06:38:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptlan.ko 2024-03-07T06:38:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptsas.ko 2024-03-07T06:38:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptscsih.ko 2024-03-07T06:38:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/message/fusion/mptspi.ko 2024-03-07T06:38:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd 2024-03-07T06:38:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_ich.ko 2024-03-07T06:38:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/lpc_sch.ko 2024-03-07T06:38:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/mfd-core.ko 2024-03-07T06:38:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/sm501.ko 2024-03-07T06:38:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/viperboard.ko 2024-03-07T06:38:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/vx855.ko 2024-03-07T06:38:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/mfd/wl1273-core.ko 2024-03-07T06:38:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc 2024-03-07T06:38:58.947 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl 2024-03-07T06:38:58.947 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-07T06:38:58.947 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds9802als.ko 2024-03-07T06:38:58.947 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/apds990x.ko 2024-03-07T06:38:58.947 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/bh1770glc.ko 2024-03-07T06:38:58.947 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710 2024-03-07T06:38:58.947 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/cb710/cb710.ko 2024-03-07T06:38:58.947 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom 2024-03-07T06:38:58.947 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/at24.ko 2024-03-07T06:38:58.947 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-07T06:38:58.947 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/eeprom/max6875.ko 2024-03-07T06:38:58.947 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/enclosure.ko 2024-03-07T06:38:58.947 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/hpilo.ko 2024-03-07T06:38:58.947 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29003.ko 2024-03-07T06:38:58.948 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/isl29020.ko 2024-03-07T06:38:58.948 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d 2024-03-07T06:38:58.948 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-07T06:38:58.948 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-07T06:38:58.948 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei 2024-03-07T06:38:58.948 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-me.ko 2024-03-07T06:38:58.948 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei-txe.ko 2024-03-07T06:38:58.948 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/mei/mei.ko 2024-03-07T06:38:58.948 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru 2024-03-07T06:38:58.948 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-07T06:38:58.948 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp 2024-03-07T06:38:58.948 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-07T06:38:58.948 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-07T06:38:58.948 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_7xx1.ko 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tifm_core.ko 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/tsl2550.ko 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_balloon.ko 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/bonding/bonding.ko 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-07T06:38:58.949 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/dummy.ko 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/eql.ko 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-07T06:38:58.950 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec 2024-03-07T06:38:58.951 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-07T06:38:58.951 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere 2024-03-07T06:38:58.951 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-07T06:38:58.951 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon 2024-03-07T06:38:58.951 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-07T06:38:58.951 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera 2024-03-07T06:38:58.951 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-07T06:38:58.951 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd 2024-03-07T06:38:58.951 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-07T06:38:58.951 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-07T06:38:58.951 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-07T06:38:58.951 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros 2024-03-07T06:38:58.951 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx 2024-03-07T06:38:58.951 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-07T06:38:58.952 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-07T06:38:58.952 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-07T06:38:58.952 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-07T06:38:58.952 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-07T06:38:58.952 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx 2024-03-07T06:38:58.952 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-07T06:38:58.952 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-07T06:38:58.952 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom 2024-03-07T06:38:58.952 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-07T06:38:58.952 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-07T06:38:58.952 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-07T06:38:58.952 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-07T06:38:58.952 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-07T06:38:58.952 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet 2024-03-07T06:38:58.953 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-07T06:38:58.953 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-07T06:38:58.953 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade 2024-03-07T06:38:58.953 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna 2024-03-07T06:38:58.953 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-07T06:38:58.953 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio 2024-03-07T06:38:58.953 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-07T06:38:58.953 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-07T06:38:58.953 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-07T06:38:58.953 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-07T06:38:58.953 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-07T06:38:58.953 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-07T06:38:58.953 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-07T06:38:58.953 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink 2024-03-07T06:38:58.954 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/dnet.ko 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ethoc.ko 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/fealnx.ko 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k 2024-03-07T06:38:58.955 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-07T06:38:58.956 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e 2024-03-07T06:38:58.956 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-07T06:38:58.956 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf 2024-03-07T06:38:58.956 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-07T06:38:58.956 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb 2024-03-07T06:38:58.956 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-07T06:38:58.956 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf 2024-03-07T06:38:58.956 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-07T06:38:58.956 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-07T06:38:58.956 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-07T06:38:58.956 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-07T06:38:58.956 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-07T06:38:58.956 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/jme.ko 2024-03-07T06:38:58.956 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-07T06:38:58.957 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-07T06:38:58.958 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi 2024-03-07T06:38:58.958 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-07T06:38:58.958 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-07T06:38:58.958 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion 2024-03-07T06:38:58.958 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-07T06:38:58.958 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia 2024-03-07T06:38:58.958 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-07T06:38:58.958 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines 2024-03-07T06:38:58.958 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-07T06:38:58.958 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-07T06:38:58.958 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic 2024-03-07T06:38:58.958 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-07T06:38:58.958 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-07T06:38:58.958 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan 2024-03-07T06:38:58.959 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-07T06:38:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis 2024-03-07T06:38:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-07T06:38:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-07T06:38:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc 2024-03-07T06:38:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-07T06:38:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-07T06:38:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-07T06:38:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-07T06:38:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro 2024-03-07T06:38:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-07T06:38:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-07T06:38:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-07T06:38:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun 2024-03-07T06:38:58.960 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom 2024-03-07T06:38:58.961 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-07T06:38:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/geneve.ko 2024-03-07T06:38:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv 2024-03-07T06:38:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-07T06:38:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154 2024-03-07T06:38:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-07T06:38:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ifb.ko 2024-03-07T06:38:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan 2024-03-07T06:38:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-07T06:38:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvlan.ko 2024-03-07T06:38:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/macvtap.ko 2024-03-07T06:38:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio 2024-03-07T06:38:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio.ko 2024-03-07T06:38:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-07T06:38:58.962 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/mii.ko 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/net_failover.ko 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/netconsole.ko 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/nlmon.ko 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/amd.ko 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/bcm87xx.ko 2024-03-07T06:38:58.963 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/broadcom.ko 2024-03-07T06:38:58.964 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/cicada.ko 2024-03-07T06:38:58.964 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/davicom.ko 2024-03-07T06:38:58.964 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/dp83640.ko 2024-03-07T06:38:58.964 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/et1011c.ko 2024-03-07T06:38:58.964 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/icplus.ko 2024-03-07T06:38:58.964 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/lxt.ko 2024-03-07T06:38:58.964 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/marvell.ko 2024-03-07T06:38:58.964 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/micrel.ko 2024-03-07T06:38:58.964 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/national.ko 2024-03-07T06:38:58.964 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/phylink.ko 2024-03-07T06:38:58.964 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/qsemi.ko 2024-03-07T06:38:58.964 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/realtek.ko 2024-03-07T06:38:58.964 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/smsc.ko 2024-03-07T06:38:58.964 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/ste10Xp.ko 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/phy/vitesse.ko 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_async.ko 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppoe.ko 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pppox.ko 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/ppp/pptp.ko 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slhc.ko 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/slip/slip.ko 2024-03-07T06:38:58.965 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/sungem_phy.ko 2024-03-07T06:38:58.966 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tap.ko 2024-03-07T06:38:58.966 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team 2024-03-07T06:38:58.966 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team.ko 2024-03-07T06:38:58.966 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-07T06:38:58.966 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-07T06:38:58.966 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-07T06:38:58.966 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_random.ko 2024-03-07T06:38:58.966 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-07T06:38:58.966 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/tun.ko 2024-03-07T06:38:58.966 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/veth.ko 2024-03-07T06:38:58.966 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/virtio_net.ko 2024-03-07T06:38:58.966 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3 2024-03-07T06:38:58.966 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-07T06:38:58.966 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan 2024-03-07T06:38:58.967 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/vxlan/vxlan.ko 2024-03-07T06:38:58.967 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback 2024-03-07T06:38:58.967 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-07T06:38:58.967 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/net/xen-netfront.ko 2024-03-07T06:38:58.967 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb 2024-03-07T06:38:58.967 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ntb/ntb.ko 2024-03-07T06:38:58.967 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme 2024-03-07T06:38:58.967 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host 2024-03-07T06:38:58.967 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme-core.ko 2024-03-07T06:38:58.967 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/nvme/host/nvme.ko 2024-03-07T06:38:58.967 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport 2024-03-07T06:38:58.967 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport.ko 2024-03-07T06:38:58.967 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_cs.ko 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_pc.ko 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/parport/parport_serial.ko 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pci/xen-pcifront.ko 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/i82092.ko 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/pd6729.ko 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel 2024-03-07T06:38:58.968 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-07T06:38:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform 2024-03-07T06:38:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86 2024-03-07T06:38:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-07T06:38:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/acerhdf.ko 2024-03-07T06:38:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-07T06:38:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-07T06:38:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-07T06:38:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-07T06:38:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-07T06:38:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-07T06:38:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-07T06:38:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-07T06:38:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-07T06:38:58.969 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/hdaps.ko 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/intel_ips.ko 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/platform/x86/wmi.ko 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap 2024-03-07T06:38:58.970 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/clients/pps_parport.ko 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/pps/pps_core.ko 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp.ko 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ptp/ptp_kvm.ko 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-07T06:38:58.971 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-em3027.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-max6900.ko 2024-03-07T06:38:58.972 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/rtc/rtc-x1205.ko 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-9xxx.ko 2024-03-07T06:38:58.973 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-sas.ko 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/3w-xxxx.ko 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/BusLogic.ko 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/a100u2w.ko 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/advansys.ko 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/am53c974.ko 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/atp870u.ko 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi 2024-03-07T06:38:58.974 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-07T06:38:58.975 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa 2024-03-07T06:38:58.975 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bfa/bfa.ko 2024-03-07T06:38:58.975 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc 2024-03-07T06:38:58.975 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-07T06:38:58.975 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i 2024-03-07T06:38:58.975 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-07T06:38:58.975 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ch.ko 2024-03-07T06:38:58.975 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor 2024-03-07T06:38:58.975 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-07T06:38:58.975 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi 2024-03-07T06:38:58.975 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-07T06:38:58.975 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-07T06:38:58.975 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-07T06:38:58.976 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-07T06:38:58.976 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-07T06:38:58.976 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dc395x.ko 2024-03-07T06:38:58.976 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler 2024-03-07T06:38:58.976 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-07T06:38:58.976 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-07T06:38:58.976 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-07T06:38:58.976 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-07T06:38:58.976 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/dmx3191d.ko 2024-03-07T06:38:58.976 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r 2024-03-07T06:38:58.976 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-07T06:38:58.976 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/esp_scsi.ko 2024-03-07T06:38:58.976 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe 2024-03-07T06:38:58.976 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/fnic/fnic.ko 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hpsa.ko 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hptiop.ko 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/hv_storvsc.ko 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/initio.ko 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ipr.ko 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ips.ko 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/isci/isci.ko 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc 2024-03-07T06:38:58.977 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libfc/libfc.ko 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi.ko 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/libsas/libsas.ko 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid.ko 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas 2024-03-07T06:38:58.978 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-07T06:38:58.979 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/mvumi.ko 2024-03-07T06:38:58.979 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001 2024-03-07T06:38:58.979 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-07T06:38:58.979 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/pmcraid.ko 2024-03-07T06:38:58.979 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla1280.ko 2024-03-07T06:38:58.979 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx 2024-03-07T06:38:58.979 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-07T06:38:58.979 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-07T06:38:58.979 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx 2024-03-07T06:38:58.979 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-07T06:38:58.979 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/raid_class.ko 2024-03-07T06:38:58.979 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_debug.ko 2024-03-07T06:38:58.979 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-07T06:38:58.979 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/ses.ko 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/st.ko 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/stex.ko 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/virtio_scsi.ko 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/scsi/wd719x.ko 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ssb/ssb.ko 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target 2024-03-07T06:38:58.980 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_file.ko 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_iblock.ko 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_mod.ko 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_pscsi.ko 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/target_core_user.ko 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-07T06:38:58.981 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-07T06:38:58.982 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-07T06:38:58.983 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty 2024-03-07T06:38:58.983 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless 2024-03-07T06:38:58.983 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-07T06:38:58.983 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_gsm.ko 2024-03-07T06:38:58.983 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/n_hdlc.ko 2024-03-07T06:38:58.983 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/nozomi.ko 2024-03-07T06:38:58.983 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial 2024-03-07T06:38:58.983 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250 2024-03-07T06:38:58.983 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-07T06:38:58.983 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/arc_uart.ko 2024-03-07T06:38:58.983 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm 2024-03-07T06:38:58.983 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-07T06:38:58.983 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio.ko 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_aec.ko 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_cif.ko 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_pci_generic.ko 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/uio/uio_sercos3.ko 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-acm.ko 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-07T06:38:58.984 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usblp.ko 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/class/usbtmc.ko 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/mdc800.ko 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/image/microtek.ko 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/adutux.ko 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/appledisplay.ko 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi26.ko 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/emi62.ko 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ezusb.ko 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/idmouse.ko 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/iowarrior.ko 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-07T06:38:58.985 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/ldusb.ko 2024-03-07T06:38:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/legousbtower.ko 2024-03-07T06:38:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga 2024-03-07T06:38:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-07T06:38:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-07T06:38:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usb3503.ko 2024-03-07T06:38:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usblcd.ko 2024-03-07T06:38:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-07T06:38:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/uss720.ko 2024-03-07T06:38:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/misc/yurex.ko 2024-03-07T06:38:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy 2024-03-07T06:38:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/phy/phy-generic.ko 2024-03-07T06:38:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial 2024-03-07T06:38:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/aircable.ko 2024-03-07T06:38:58.986 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ark3116.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ch341.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cp210x.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cyberjack.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/empeg.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/io_ti.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipaq.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ipw.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ir-usb.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-07T06:38:58.987 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan.ko 2024-03-07T06:38:58.988 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-07T06:38:58.988 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-07T06:38:58.988 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-07T06:38:58.988 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mct_u232.ko 2024-03-07T06:38:58.988 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7720.ko 2024-03-07T06:38:58.988 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/mos7840.ko 2024-03-07T06:38:58.988 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/navman.ko 2024-03-07T06:38:58.988 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/omninet.ko 2024-03-07T06:38:58.988 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/opticon.ko 2024-03-07T06:38:58.988 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/option.ko 2024-03-07T06:38:58.988 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/oti6858.ko 2024-03-07T06:38:58.988 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/pl2303.ko 2024-03-07T06:38:58.988 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcaux.ko 2024-03-07T06:38:58.988 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/qcserial.ko 2024-03-07T06:38:58.989 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/quatech2.ko 2024-03-07T06:38:58.989 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/safe_serial.ko 2024-03-07T06:38:58.989 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/sierra.ko 2024-03-07T06:38:58.989 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-07T06:38:58.989 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ssu100.ko 2024-03-07T06:38:58.989 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/symbolserial.ko 2024-03-07T06:38:58.989 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-07T06:38:58.989 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-07T06:38:58.989 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_debug.ko 2024-03-07T06:38:58.989 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-07T06:38:58.989 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/visor.ko 2024-03-07T06:38:58.989 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/whiteheat.ko 2024-03-07T06:38:58.989 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-07T06:38:58.989 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/uas.ko 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-karma.ko 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-07T06:38:58.990 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/storage/usb-storage.ko 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio.ko 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost.ko 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_net.ko 2024-03-07T06:38:58.991 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/vhost/vhost_scsi.ko 2024-03-07T06:38:58.992 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video 2024-03-07T06:38:58.992 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight 2024-03-07T06:38:58.992 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/apple_bl.ko 2024-03-07T06:38:58.992 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/lcd.ko 2024-03-07T06:38:58.992 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-07T06:38:58.992 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio 2024-03-07T06:38:58.992 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio.ko 2024-03-07T06:38:58.992 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_balloon.ko 2024-03-07T06:38:58.992 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_mmio.ko 2024-03-07T06:38:58.992 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci.ko 2024-03-07T06:38:58.992 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-07T06:38:58.992 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-07T06:38:58.992 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/virtio/virtio_ring.ko 2024-03-07T06:38:58.992 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2482.ko 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/masters/ds2490.ko 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-07T06:38:58.993 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/w1/wire.ko 2024-03-07T06:38:58.994 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog 2024-03-07T06:38:58.994 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-07T06:38:58.994 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-07T06:38:58.994 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-07T06:38:58.994 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/hpwdt.ko 2024-03-07T06:38:58.994 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/i6300esb.ko 2024-03-07T06:38:58.994 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-07T06:38:58.994 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-07T06:38:58.994 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ib700wdt.ko 2024-03-07T06:38:58.994 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ibmasr.ko 2024-03-07T06:38:58.994 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-07T06:38:58.994 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-07T06:38:58.994 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/it87_wdt.ko 2024-03-07T06:38:58.994 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/machzwd.ko 2024-03-07T06:38:58.995 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/nv_tco.ko 2024-03-07T06:38:58.995 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-07T06:38:58.995 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-07T06:38:58.995 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-07T06:38:58.995 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-07T06:38:58.995 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/softdog.ko 2024-03-07T06:38:58.995 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-07T06:38:58.995 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/via_wdt.ko 2024-03-07T06:38:58.995 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-07T06:38:58.995 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-07T06:38:58.995 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-07T06:38:58.995 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/wdt_pci.ko 2024-03-07T06:38:58.995 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/watchdog/xen_wdt.ko 2024-03-07T06:38:58.995 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-evtchn.ko 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntalloc.ko 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-gntdev.ko 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xen-privcmd.ko 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/9p/9p.ko 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/affs/affs.ko 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs 2024-03-07T06:38:58.996 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/befs/befs.ko 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/binfmt_misc.ko 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/btrfs/btrfs.ko 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cachefiles/cachefiles.ko 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ceph/ceph.ko 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/coda/coda.ko 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/cramfs/cramfs.ko 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/dlm/dlm.ko 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs 2024-03-07T06:38:58.997 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/f2fs/f2fs.ko 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/fat.ko 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/msdos.ko 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fat/vfat.ko 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/cuse.ko 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/fuse/fuse.ko 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/gfs2/gfs2.ko 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfs/hfs.ko 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus 2024-03-07T06:38:58.998 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/hfsplus/hfsplus.ko 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/isofs/isofs.ko 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/jfs/jfs.ko 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/lockd/lockd.ko 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/minix/minix.ko 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/netfs/netfs.ko 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout 2024-03-07T06:38:58.999 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-07T06:38:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout 2024-03-07T06:38:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-07T06:38:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfs.ko 2024-03-07T06:38:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv3.ko 2024-03-07T06:38:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs/nfsv4.ko 2024-03-07T06:38:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common 2024-03-07T06:38:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/grace.ko 2024-03-07T06:38:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfs_common/nfs_acl.ko 2024-03-07T06:38:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd 2024-03-07T06:38:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nfsd/nfsd.ko 2024-03-07T06:38:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2 2024-03-07T06:38:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nilfs2/nilfs2.ko 2024-03-07T06:38:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls 2024-03-07T06:38:59.000 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-celtic.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-centeuro.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-croatian.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-cyrillic.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-gaelic.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-greek.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-iceland.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-inuit.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-roman.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-romanian.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/mac-turkish.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1250.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1251.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp1255.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp737.ko 2024-03-07T06:38:59.001 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp775.ko 2024-03-07T06:38:59.002 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp850.ko 2024-03-07T06:38:59.002 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp852.ko 2024-03-07T06:38:59.002 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp855.ko 2024-03-07T06:38:59.002 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp857.ko 2024-03-07T06:38:59.002 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp860.ko 2024-03-07T06:38:59.002 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp861.ko 2024-03-07T06:38:59.002 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp862.ko 2024-03-07T06:38:59.002 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp863.ko 2024-03-07T06:38:59.002 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp864.ko 2024-03-07T06:38:59.002 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp865.ko 2024-03-07T06:38:59.002 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp866.ko 2024-03-07T06:38:59.002 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp869.ko 2024-03-07T06:38:59.002 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp874.ko 2024-03-07T06:38:59.002 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp932.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp936.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp949.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_cp950.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_euc-jp.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-1.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-13.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-14.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-15.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-2.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-3.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-4.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-5.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-6.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-7.ko 2024-03-07T06:38:59.003 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_iso8859-9.ko 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-r.ko 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-ru.ko 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_koi8-u.ko 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/nls/nls_utf8.ko 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2.ko 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-07T06:38:59.004 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/overlayfs/overlay.ko 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/pstore/ramoops.ko 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/reiserfs/reiserfs.ko 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/romfs/romfs.ko 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/client/cifs.ko 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_arc4.ko 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/smb/common/cifs_md4.ko 2024-03-07T06:38:59.005 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/squashfs/squashfs.ko 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/sysv/sysv.ko 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/udf/udf.ko 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/ufs/ufs.ko 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/fs/xfs/xfs.ko 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/rcu/rcutorture.ko 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/torture.ko 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-07T06:38:59.006 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/asn1_encoder.ko 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc-itu-t.ko 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crc8.ko 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libarc4.ko 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/crypto/libdes.ko 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/libcrc32c.ko 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lru_cache.ko 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4_compress.ko 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/lz4/lz4hc_compress.ko 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/math/cordic.ko 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/raid6/raid6_pq.ko 2024-03-07T06:38:59.007 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_bm.ko 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_fsm.ko 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/ts_kmp.ko 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/lib/zstd/zstd_compress.ko 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/mm/hwpoison-inject.ko 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/6lowpan.ko 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_dest.ko 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_fragment.ko 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_hop.ko 2024-03-07T06:38:59.008 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_mobility.ko 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_routing.ko 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/6lowpan/nhc_udp.ko 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/garp.ko 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/mrp.ko 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/p8022.ko 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/psnap.ko 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/802/stp.ko 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/8021q/8021q.ko 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet.ko 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_fd.ko 2024-03-07T06:38:59.009 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_rdma.ko 2024-03-07T06:38:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/9p/9pnet_virtio.ko 2024-03-07T06:38:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv 2024-03-07T06:38:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/batman-adv/batman-adv.ko 2024-03-07T06:38:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge 2024-03-07T06:38:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/br_netfilter.ko 2024-03-07T06:38:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/bridge.ko 2024-03-07T06:38:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter 2024-03-07T06:38:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-07T06:38:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-07T06:38:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-07T06:38:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-07T06:38:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-07T06:38:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-07T06:38:59.010 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/ebtables.ko 2024-03-07T06:38:59.011 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-07T06:38:59.012 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-07T06:38:59.012 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-07T06:38:59.012 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph 2024-03-07T06:38:59.012 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ceph/libceph.ko 2024-03-07T06:38:59.012 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core 2024-03-07T06:38:59.012 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/failover.ko 2024-03-07T06:38:59.012 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/core/pktgen.ko 2024-03-07T06:38:59.012 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp 2024-03-07T06:38:59.012 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp.ko 2024-03-07T06:38:59.012 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_diag.ko 2024-03-07T06:38:59.012 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv4.ko 2024-03-07T06:38:59.012 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dccp/dccp_ipv6.ko 2024-03-07T06:38:59.012 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver 2024-03-07T06:38:59.012 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dns_resolver/dns_resolver.ko 2024-03-07T06:38:59.013 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa 2024-03-07T06:38:59.013 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/dsa_core.ko 2024-03-07T06:38:59.013 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_brcm.ko 2024-03-07T06:38:59.013 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_dsa.ko 2024-03-07T06:38:59.013 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_none.ko 2024-03-07T06:38:59.013 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/dsa/tag_trailer.ko 2024-03-07T06:38:59.013 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154 2024-03-07T06:38:59.013 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan 2024-03-07T06:38:59.013 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-07T06:38:59.013 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154.ko 2024-03-07T06:38:59.013 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-07T06:38:59.013 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4 2024-03-07T06:38:59.013 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ah4.ko 2024-03-07T06:38:59.013 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/esp4.ko 2024-03-07T06:38:59.014 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/fou.ko 2024-03-07T06:38:59.014 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/gre.ko 2024-03-07T06:38:59.014 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/inet_diag.ko 2024-03-07T06:38:59.014 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_gre.ko 2024-03-07T06:38:59.014 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_tunnel.ko 2024-03-07T06:38:59.014 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ip_vti.ko 2024-03-07T06:38:59.014 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipcomp.ko 2024-03-07T06:38:59.014 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/ipip.ko 2024-03-07T06:38:59.014 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter 2024-03-07T06:38:59.014 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-07T06:38:59.014 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-07T06:38:59.014 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-07T06:38:59.014 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-07T06:38:59.014 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-07T06:38:59.015 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-07T06:38:59.016 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-07T06:38:59.016 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_bic.ko 2024-03-07T06:38:59.016 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_dctcp.ko 2024-03-07T06:38:59.016 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_diag.ko 2024-03-07T06:38:59.016 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_highspeed.ko 2024-03-07T06:38:59.016 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_htcp.ko 2024-03-07T06:38:59.016 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_hybla.ko 2024-03-07T06:38:59.016 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_illinois.ko 2024-03-07T06:38:59.016 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_lp.ko 2024-03-07T06:38:59.016 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_scalable.ko 2024-03-07T06:38:59.016 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_vegas.ko 2024-03-07T06:38:59.016 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_veno.ko 2024-03-07T06:38:59.016 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_westwood.ko 2024-03-07T06:38:59.016 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tcp_yeah.ko 2024-03-07T06:38:59.017 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/tunnel4.ko 2024-03-07T06:38:59.017 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_diag.ko 2024-03-07T06:38:59.017 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/udp_tunnel.ko 2024-03-07T06:38:59.017 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-07T06:38:59.017 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6 2024-03-07T06:38:59.017 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ah6.ko 2024-03-07T06:38:59.017 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/esp6.ko 2024-03-07T06:38:59.017 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/fou6.ko 2024-03-07T06:38:59.017 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_tunnel.ko 2024-03-07T06:38:59.017 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-07T06:38:59.017 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ip6_vti.ko 2024-03-07T06:38:59.017 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/ipcomp6.ko 2024-03-07T06:38:59.017 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter 2024-03-07T06:38:59.017 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-07T06:38:59.018 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/sit.ko 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/tunnel6.ko 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/key/af_key.ko 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp 2024-03-07T06:38:59.019 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_core.ko 2024-03-07T06:38:59.020 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-07T06:38:59.020 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_eth.ko 2024-03-07T06:38:59.020 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip.ko 2024-03-07T06:38:59.020 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ip6.ko 2024-03-07T06:38:59.020 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_netlink.ko 2024-03-07T06:38:59.020 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/l2tp/l2tp_ppp.ko 2024-03-07T06:38:59.020 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc 2024-03-07T06:38:59.020 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/llc/llc.ko 2024-03-07T06:38:59.020 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154 2024-03-07T06:38:59.020 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mac802154/mac802154.ko 2024-03-07T06:38:59.020 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls 2024-03-07T06:38:59.020 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/mpls/mpls_gso.ko 2024-03-07T06:38:59.020 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter 2024-03-07T06:38:59.020 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset 2024-03-07T06:38:59.021 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set.ko 2024-03-07T06:38:59.021 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-07T06:38:59.021 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-07T06:38:59.021 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-07T06:38:59.021 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-07T06:38:59.021 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-07T06:38:59.021 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-07T06:38:59.021 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-07T06:38:59.021 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-07T06:38:59.021 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-07T06:38:59.021 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-07T06:38:59.021 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-07T06:38:59.021 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-07T06:38:59.021 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-07T06:38:59.022 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-07T06:38:59.023 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-07T06:38:59.023 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-07T06:38:59.023 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conncount.ko 2024-03-07T06:38:59.023 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack.ko 2024-03-07T06:38:59.023 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-07T06:38:59.023 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-07T06:38:59.023 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-07T06:38:59.023 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-07T06:38:59.023 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-07T06:38:59.023 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-07T06:38:59.023 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-07T06:38:59.023 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-07T06:38:59.023 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-07T06:38:59.023 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_log_syslog.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_irc.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_sip.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nf_tables.ko 2024-03-07T06:38:59.024 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink.ko 2024-03-07T06:38:59.025 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-07T06:38:59.025 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_log.ko 2024-03-07T06:38:59.025 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-07T06:38:59.025 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-07T06:38:59.025 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_chain_nat.ko 2024-03-07T06:38:59.025 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_compat.ko 2024-03-07T06:38:59.025 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_ct.ko 2024-03-07T06:38:59.025 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-07T06:38:59.025 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fib.ko 2024-03-07T06:38:59.025 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-07T06:38:59.025 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_hash.ko 2024-03-07T06:38:59.025 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_limit.ko 2024-03-07T06:38:59.025 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_log.ko 2024-03-07T06:38:59.025 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_masq.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_nat.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_numgen.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_queue.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_quota.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_redir.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_reject_inet.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_socket.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/nft_tunnel.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_AUDIT.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_CT.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_DSCP.ko 2024-03-07T06:38:59.026 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HL.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_HMARK.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LED.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_LOG.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NETMAP.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFLOG.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_RATEEST.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_SECMARK.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TEE.ko 2024-03-07T06:38:59.027 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TPROXY.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_TRACE.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_addrtype.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_bpf.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cgroup.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cluster.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_comment.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connbytes.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlabel.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connlimit.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_connmark.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_conntrack.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_cpu.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dccp.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_devgroup.ko 2024-03-07T06:38:59.028 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_dscp.ko 2024-03-07T06:38:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ecn.ko 2024-03-07T06:38:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_esp.ko 2024-03-07T06:38:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hashlimit.ko 2024-03-07T06:38:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_helper.ko 2024-03-07T06:38:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_hl.ko 2024-03-07T06:38:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipcomp.ko 2024-03-07T06:38:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_iprange.ko 2024-03-07T06:38:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_ipvs.ko 2024-03-07T06:38:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_l2tp.ko 2024-03-07T06:38:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_length.ko 2024-03-07T06:38:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_limit.ko 2024-03-07T06:38:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mac.ko 2024-03-07T06:38:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_mark.ko 2024-03-07T06:38:59.029 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_multiport.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nat.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_nfacct.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_osf.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_owner.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_physdev.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_pkttype.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_policy.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_quota.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_rateest.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_realm.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_recent.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_sctp.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_set.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_socket.ko 2024-03-07T06:38:59.030 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_state.ko 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_statistic.ko 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_string.ko 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_tcpmss.ko 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_time.ko 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netfilter/xt_u32.ko 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/netlink/netlink_diag.ko 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/nsh/nsh.ko 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/openvswitch.ko 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-geneve.ko 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-gre.ko 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/openvswitch/vport-vxlan.ko 2024-03-07T06:38:59.031 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet 2024-03-07T06:38:59.032 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/packet/af_packet_diag.ko 2024-03-07T06:38:59.032 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds 2024-03-07T06:38:59.032 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds.ko 2024-03-07T06:38:59.032 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_rdma.ko 2024-03-07T06:38:59.032 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/rds/rds_tcp.ko 2024-03-07T06:38:59.032 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched 2024-03-07T06:38:59.032 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_csum.ko 2024-03-07T06:38:59.032 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_gact.ko 2024-03-07T06:38:59.032 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_mirred.ko 2024-03-07T06:38:59.032 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_nat.ko 2024-03-07T06:38:59.032 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_pedit.ko 2024-03-07T06:38:59.032 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_police.ko 2024-03-07T06:38:59.032 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_simple.ko 2024-03-07T06:38:59.032 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_skbedit.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/act_vlan.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_basic.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_bpf.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_flow.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_fw.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_route.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/cls_u32.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_cmp.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_ipset.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_meta.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_nbyte.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_text.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/em_u32.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_choke.ko 2024-03-07T06:38:59.033 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_codel.ko 2024-03-07T06:38:59.034 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_drr.ko 2024-03-07T06:38:59.034 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_fq.ko 2024-03-07T06:38:59.034 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_gred.ko 2024-03-07T06:38:59.034 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hfsc.ko 2024-03-07T06:38:59.034 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_hhf.ko 2024-03-07T06:38:59.034 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_htb.ko 2024-03-07T06:38:59.034 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_ingress.ko 2024-03-07T06:38:59.034 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio.ko 2024-03-07T06:38:59.034 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_mqprio_lib.ko 2024-03-07T06:38:59.034 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_multiq.ko 2024-03-07T06:38:59.034 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_netem.ko 2024-03-07T06:38:59.034 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_pie.ko 2024-03-07T06:38:59.034 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_plug.ko 2024-03-07T06:38:59.034 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_prio.ko 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_qfq.ko 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_red.ko 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfb.ko 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_sfq.ko 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_tbf.ko 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sched/sch_teql.ko 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp.ko 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sctp/sctp_diag.ko 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/sunrpc.ko 2024-03-07T06:38:59.035 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma 2024-03-07T06:38:59.036 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-07T06:38:59.036 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc 2024-03-07T06:38:59.036 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/diag.ko 2024-03-07T06:38:59.036 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/tipc/tipc.ko 2024-03-07T06:38:59.036 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix 2024-03-07T06:38:59.036 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/unix/unix_diag.ko 2024-03-07T06:38:59.036 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock 2024-03-07T06:38:59.036 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-07T06:38:59.036 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-07T06:38:59.036 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock.ko 2024-03-07T06:38:59.036 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-07T06:38:59.036 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-07T06:38:59.036 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm 2024-03-07T06:38:59.036 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/security/keys/trusted-keys/trusted.ko 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/kernel/virt/lib/irqbypass.ko 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.alias.bin 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.alias.bin 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.bin 2024-03-07T06:38:59.037 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.builtin.modinfo 2024-03-07T06:38:59.038 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep 2024-03-07T06:38:59.038 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.dep.bin 2024-03-07T06:38:59.038 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.devname 2024-03-07T06:38:59.038 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.order 2024-03-07T06:38:59.038 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.softdep 2024-03-07T06:38:59.038 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols 2024-03-07T06:38:59.038 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/modules.symbols.bin 2024-03-07T06:38:59.038 INFO:teuthology.orchestra.run.smithi161.stdout:/lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-07T06:38:59.041 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:38:59.042 DEBUG:teuthology.orchestra.run.smithi161:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-07T06:38:59.064 INFO:teuthology.orchestra.run.smithi161.stdout:postinstall scriptlet (using /bin/sh): 2024-03-07T06:38:59.064 INFO:teuthology.orchestra.run.smithi161.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-07T06:38:59.064 INFO:teuthology.orchestra.run.smithi161.stdout: /usr/bin/kernel-install add 6.8.0-rc7-gdd5c1cd17167 /lib/modules/6.8.0-rc7-gdd5c1cd17167/vmlinuz 2024-03-07T06:38:59.064 INFO:teuthology.orchestra.run.smithi161.stdout:fi 2024-03-07T06:38:59.064 INFO:teuthology.orchestra.run.smithi161.stdout:for file in vmlinuz System.map config; do 2024-03-07T06:38:59.064 INFO:teuthology.orchestra.run.smithi161.stdout: if ! cmp --silent "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167"; then 2024-03-07T06:38:59.064 INFO:teuthology.orchestra.run.smithi161.stdout: cp "/lib/modules/6.8.0-rc7-gdd5c1cd17167/${file}" "/boot/${file}-6.8.0-rc7-gdd5c1cd17167" 2024-03-07T06:38:59.064 INFO:teuthology.orchestra.run.smithi161.stdout: fi 2024-03-07T06:38:59.064 INFO:teuthology.orchestra.run.smithi161.stdout:done 2024-03-07T06:38:59.064 INFO:teuthology.orchestra.run.smithi161.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-07T06:38:59.064 INFO:teuthology.orchestra.run.smithi161.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-07T06:38:59.065 INFO:teuthology.orchestra.run.smithi161.stdout:new-kernel-pkg --remove 6.8.0-rc7-gdd5c1cd17167 --rminitrd --initrdfile=/boot/initramfs-6.8.0-rc7-gdd5c1cd17167.img 2024-03-07T06:38:59.065 INFO:teuthology.orchestra.run.smithi161.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-07T06:38:59.065 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-install remove 6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:38:59.065 INFO:teuthology.orchestra.run.smithi161.stdout:fi 2024-03-07T06:38:59.065 INFO:teuthology.orchestra.run.smithi161.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-07T06:38:59.065 INFO:teuthology.orchestra.run.smithi161.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-07T06:38:59.065 INFO:teuthology.orchestra.run.smithi161.stdout:/sbin/update-bootloader --remove 6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:38:59.065 INFO:teuthology.orchestra.run.smithi161.stdout:fi 2024-03-07T06:38:59.066 DEBUG:teuthology.orchestra.run.smithi161:> sudo rpm -qi grub2-tools 2024-03-07T06:38:59.140 INFO:teuthology.orchestra.run.smithi161.stdout:Name : grub2-tools 2024-03-07T06:38:59.140 INFO:teuthology.orchestra.run.smithi161.stdout:Epoch : 1 2024-03-07T06:38:59.140 INFO:teuthology.orchestra.run.smithi161.stdout:Version : 2.06 2024-03-07T06:38:59.140 INFO:teuthology.orchestra.run.smithi161.stdout:Release : 68.el9 2024-03-07T06:38:59.140 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: x86_64 2024-03-07T06:38:59.140 INFO:teuthology.orchestra.run.smithi161.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-07T06:38:59.140 INFO:teuthology.orchestra.run.smithi161.stdout:Group : Unspecified 2024-03-07T06:38:59.140 INFO:teuthology.orchestra.run.smithi161.stdout:Size : 8264254 2024-03-07T06:38:59.140 INFO:teuthology.orchestra.run.smithi161.stdout:License : GPLv3+ 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout:Packager : builder@centos.org 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout:Vendor : CentOS 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout:Summary : Support tools for GRUB. 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout:Description : 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout:hardware devices. 2024-03-07T06:38:59.141 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:38:59.142 INFO:teuthology.orchestra.run.smithi161.stdout:This subpackage provides tools for support of all platforms. 2024-03-07T06:38:59.142 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-07T06:38:59.142 INFO:teuthology.task.kernel:Updating grub on smithi161 to boot 6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:38:59.143 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-07T06:38:59.714 INFO:teuthology.orchestra.run.smithi161.stderr:Generating grub configuration file ... 2024-03-07T06:39:01.210 INFO:teuthology.orchestra.run.smithi161.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-07T06:39:01.242 INFO:teuthology.orchestra.run.smithi161.stderr:done 2024-03-07T06:39:01.246 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2024-03-07T06:39:01.261 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.OozXeebnMP 2024-03-07T06:39:01.261 DEBUG:teuthology.orchestra.run.smithi161:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.OozXeebnMP 2024-03-07T06:39:01.343 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.OozXeebnMP 2024-03-07T06:39:01.481 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.OozXeebnMP is 6KB 2024-03-07T06:39:01.531 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.OozXeebnMP 2024-03-07T06:39:01.546 DEBUG:teuthology.orchestra.run.smithi161:> sudo /bin/ls /boot/loader/entries || true 2024-03-07T06:39:01.611 INFO:teuthology.orchestra.run.smithi161.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-07T06:39:01.611 INFO:teuthology.orchestra.run.smithi161.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-07T06:39:01.611 INFO:teuthology.orchestra.run.smithi161.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-07T06:39:01.611 INFO:teuthology.orchestra.run.smithi161.stdout:15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167.conf 2024-03-07T06:39:01.612 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:39:01.769 DEBUG:teuthology.orchestra.run.smithi161:> sudo shutdown -r now 2024-03-07T06:39:06.916 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-07T06:39:06.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-07T06:39:06.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T06:39:25.244 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-03-07T06:39:31.776 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-07T06:39:31.776 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-07T06:39:31.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T06:39:34.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-07T06:39:34.248 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T06:39:37.308 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-03-07T06:39:49.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-07T06:39:49.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T06:39:50.332 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-03-07T06:39:52.380 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-03-07T06:39:59.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-07T06:39:59.337 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T06:40:02.396 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-03-07T06:40:07.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-07T06:40:07.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T06:40:10.464 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-03-07T06:40:14.400 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-07T06:40:14.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T06:40:28.476 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-07T06:40:28.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T06:40:28.982 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-03-07T06:40:29.616 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-07T06:40:29.616 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-rc7-gdd5c1cd17167"... 2024-03-07T06:40:29.616 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-03-07T06:40:29.655 INFO:teuthology.orchestra.run.smithi133.stdout:6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:40:29.655 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-gdd5c1cd17167 vs 6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:40:29.655 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-07T06:40:29.655 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-07T06:40:30.656 DEBUG:teuthology.parallel:result is None 2024-03-07T06:40:32.828 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-03-07T06:40:47.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-07T06:40:47.830 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T06:40:50.908 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2024-03-07T06:41:08.909 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-07T06:41:08.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T06:41:09.654 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-03-07T06:41:10.307 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-07T06:41:10.308 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.8.0-rc7-gdd5c1cd17167"... 2024-03-07T06:41:10.308 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2024-03-07T06:41:10.386 INFO:teuthology.orchestra.run.smithi161.stdout:6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:41:10.386 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-rc7-gdd5c1cd17167 vs 6.8.0-rc7-gdd5c1cd17167 2024-03-07T06:41:10.386 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-07T06:41:10.387 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-07T06:41:11.390 DEBUG:teuthology.parallel:result is None 2024-03-07T06:41:11.390 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-07T06:41:11.396 INFO:teuthology.task.internal:Creating test directory... 2024-03-07T06:41:11.397 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-07T06:41:11.406 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-07T06:41:11.480 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-07T06:41:11.486 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-07T06:41:11.495 INFO:teuthology.task.internal:Creating archive directory... 2024-03-07T06:41:11.495 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-07T06:41:11.502 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-07T06:41:11.599 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-07T06:41:11.606 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-07T06:41:11.607 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-07T06:41:11.614 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-07T06:41:11.778 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-07T06:41:11.795 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-07T06:41:11.848 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-07T06:41:11.849 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-07T06:41:11.853 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-07T06:41:11.861 INFO:teuthology.task.internal:Configuring sudo... 2024-03-07T06:41:11.861 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-07T06:41:11.868 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-07T06:41:11.957 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-07T06:41:11.966 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-07T06:41:11.966 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-07T06:41:11.974 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-07T06:41:12.018 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-07T06:41:12.098 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-07T06:41:12.205 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-07T06:41:12.290 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-07T06:41:12.384 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:41:12.384 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-07T06:41:12.476 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-07T06:41:12.551 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-07T06:41:12.653 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-07T06:41:12.735 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-07T06:41:12.820 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:41:12.820 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-07T06:41:12.900 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-03-07T06:41:12.907 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2024-03-07T06:41:13.030 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-07T06:41:13.045 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-07T06:41:13.388 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-07T06:41:13.396 INFO:teuthology.task.internal:Starting timer... 2024-03-07T06:41:13.396 INFO:teuthology.run_tasks:Running task pcp... 2024-03-07T06:41:13.420 INFO:teuthology.run_tasks:Running task selinux... 2024-03-07T06:41:13.429 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2024-03-07T06:41:13.543 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2024-03-07T06:41:13.547 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2024-03-07T06:41:13.715 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2024-03-07T06:41:13.720 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-07T06:41:13.720 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2024-03-07T06:41:13.782 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2024-03-07T06:41:13.782 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2024-03-07T06:41:13.883 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2024-03-07T06:41:13.883 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi133.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi161.front.sepia.ceph.com': 'permissive'} 2024-03-07T06:41:13.884 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-07T06:41:13.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:41:13.978 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-07T06:41:14.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:41:14.048 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-07T06:41:14.048 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2024-03-07T06:41:14.137 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2024-03-07T06:41:14.226 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-07T06:41:14.235 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-07T06:41:14.241 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-07T06:41:14.384 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-07T06:41:14.403 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-07T06:41:14.404 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi133.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-07T06:47:36.243 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi133.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2024-03-07T06:47:36.244 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-07T06:47:36.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T06:47:36.397 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-03-07T06:47:36.558 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-03-07T06:47:36.559 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-07T06:47:36.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-07T06:47:36.698 DEBUG:teuthology.orchestra.run.smithi161:> true 2024-03-07T06:47:36.852 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2024-03-07T06:47:36.852 INFO:teuthology.run_tasks:Running task clock... 2024-03-07T06:47:36.863 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-07T06:47:36.863 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-07T06:47:36.863 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-07T06:47:36.871 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-07T06:47:36.871 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-07T06:47:36.970 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-07T06:47:36.972 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-07T06:47:37.021 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-07T06:47:37.025 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-07T06:47:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2024-03-07T06:47:37.116 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2024-03-07T06:47:37.147 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2024-03-07T06:47:37.154 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2024-03-07T06:47:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-07T06:47:37.205 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-07T06:47:37.250 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-07T06:47:37.258 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-07T06:47:37.376 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-03-07T06:47:37.384 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-07T06:47:37.384 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-03-07T06:47:37.384 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T06:47:37.384 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T06:47:37.384 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T06:47:37.384 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T06:47:37.390 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-03-07T06:47:37.398 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-07T06:47:37.399 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-03-07T06:47:37.399 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T06:47:37.399 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T06:47:37.399 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T06:47:37.399 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T06:47:37.400 INFO:teuthology.run_tasks:Running task install... 2024-03-07T06:47:37.408 DEBUG:teuthology.task.install:project ceph 2024-03-07T06:47:37.408 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}} 2024-03-07T06:47:37.408 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'} 2024-03-07T06:47:37.408 INFO:teuthology.task.install:Using flavor: default 2024-03-07T06:47:37.414 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-07T06:47:37.414 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-07T06:47:37.415 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'tag': None, 'wait_for_package': False} 2024-03-07T06:47:37.415 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T06:47:37.417 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'tag': None, 'wait_for_package': False} 2024-03-07T06:47:37.418 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T06:47:37.563 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/ 2024-03-07T06:47:37.563 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1848.g8b896a9e 2024-03-07T06:47:37.638 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/ 2024-03-07T06:47:37.638 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1848.g8b896a9e 2024-03-07T06:47:37.676 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-07T06:47:37.676 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:47:37.676 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-07T06:47:37.782 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-07T06:47:37.782 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8b896a9e145796119f0451201dc4d53ddac97db0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-07T06:47:37.807 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-lusov-qdb/8b896a9e145796119f0451201dc4d53ddac97db0/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-07T06:47:37.807 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:47:37.807 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-07T06:47:37.908 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-07T06:47:37.909 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8b896a9e145796119f0451201dc4d53ddac97db0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-07T06:47:37.917 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-07T06:47:38.047 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-07T06:47:38.052 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-07T06:47:38.144 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2024-03-07T06:47:38.148 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2024-03-07T06:47:38.176 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-07T06:47:38.258 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2024-03-07T06:47:38.262 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2024-03-07T06:47:38.510 INFO:teuthology.orchestra.run.smithi133.stdout:74 files removed 2024-03-07T06:47:38.549 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-07T06:47:38.610 INFO:teuthology.orchestra.run.smithi161.stdout:74 files removed 2024-03-07T06:47:38.649 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-07T06:47:39.571 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 204 kB/s | 83 kB 00:00 2024-03-07T06:47:39.596 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 153 kB/s | 83 kB 00:00 2024-03-07T06:47:40.039 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 29 kB/s | 10 kB 00:00 2024-03-07T06:47:40.064 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 25 kB/s | 10 kB 00:00 2024-03-07T06:47:40.598 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-03-07T06:47:40.631 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2024-03-07T06:47:41.243 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-9 - AppStream 14 MB/s | 8.1 MB 00:00 2024-03-07T06:47:41.285 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS-9 - AppStream 14 MB/s | 8.1 MB 00:00 2024-03-07T06:47:43.110 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS-9 - BaseOS 6.8 MB/s | 2.5 MB 00:00 2024-03-07T06:47:43.126 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.5 MB 00:00 2024-03-07T06:47:44.140 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-03-07T06:47:44.314 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-03-07T06:47:46.382 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-03-07T06:47:48.248 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 19 MB 00:02 2024-03-07T06:47:50.858 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.1 MB 00:00 2024-03-07T06:47:52.500 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 15 kB 00:00 2024-03-07T06:47:53.866 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.1 MB 00:01 2024-03-07T06:47:55.650 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 15 kB 00:00 2024-03-07T06:47:57.624 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-03-07T06:47:58.305 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 3.7 MB/s | 21 MB 00:05 2024-03-07T06:48:03.460 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-07T06:48:04.093 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-03-07T06:48:06.068 INFO:teuthology.orchestra.run.smithi133.stderr:Modular dependency problems: 2024-03-07T06:48:06.068 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-03-07T06:48:06.068 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 1: conflicting requests 2024-03-07T06:48:06.069 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240227110532:20125149.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.069 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 2: conflicting requests 2024-03-07T06:48:06.069 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.069 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 3: conflicting requests 2024-03-07T06:48:06.069 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.069 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 4: conflicting requests 2024-03-07T06:48:06.069 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.069 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 5: conflicting requests 2024-03-07T06:48:06.069 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.069 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 6: conflicting requests 2024-03-07T06:48:06.069 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.069 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 7: conflicting requests 2024-03-07T06:48:06.069 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240222150954:49cc9d1b.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.069 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 8: conflicting requests 2024-03-07T06:48:06.070 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.070 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 9: conflicting requests 2024-03-07T06:48:06.070 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.070 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 10: conflicting requests 2024-03-07T06:48:06.070 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.070 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 11: conflicting requests 2024-03-07T06:48:06.070 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.070 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 12: conflicting requests 2024-03-07T06:48:06.070 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.070 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 13: conflicting requests 2024-03-07T06:48:06.070 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.070 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 14: conflicting requests 2024-03-07T06:48:06.070 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 15: conflicting requests 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 16: conflicting requests 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 17: conflicting requests 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 18: conflicting requests 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 19: conflicting requests 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 20: conflicting requests 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 21: conflicting requests 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.071 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 22: conflicting requests 2024-03-07T06:48:06.072 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.072 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 23: conflicting requests 2024-03-07T06:48:06.072 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.072 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 24: conflicting requests 2024-03-07T06:48:06.072 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.072 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 25: conflicting requests 2024-03-07T06:48:06.072 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.072 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 26: conflicting requests 2024-03-07T06:48:06.072 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.072 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 27: conflicting requests 2024-03-07T06:48:06.072 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.072 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 28: conflicting requests 2024-03-07T06:48:06.072 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.073 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 29: conflicting requests 2024-03-07T06:48:06.073 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.073 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 30: conflicting requests 2024-03-07T06:48:06.073 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.073 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 31: conflicting requests 2024-03-07T06:48:06.073 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.073 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 32: conflicting requests 2024-03-07T06:48:06.073 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.073 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 33: conflicting requests 2024-03-07T06:48:06.073 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.073 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 34: conflicting requests 2024-03-07T06:48:06.073 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.073 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 35: conflicting requests 2024-03-07T06:48:06.073 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 36: conflicting requests 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 37: conflicting requests 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 38: cannot install the best candidate for the job 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 39: conflicting requests 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.074 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 40: cannot install the best candidate for the job 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 41: cannot install the best candidate for the job 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 42: cannot install the best candidate for the job 2024-03-07T06:48:06.075 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.076 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.076 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.077 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.077 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 43: cannot install the best candidate for the job 2024-03-07T06:48:06.077 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.077 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.077 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.077 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.077 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 44: cannot install the best candidate for the job 2024-03-07T06:48:06.078 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.078 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.078 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.078 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.078 INFO:teuthology.orchestra.run.smithi133.stderr: Problem 45: cannot install the best candidate for the job 2024-03-07T06:48:06.078 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.078 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.079 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.079 INFO:teuthology.orchestra.run.smithi133.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.670 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-07T06:48:06.672 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-07T06:48:06.682 INFO:teuthology.orchestra.run.smithi161.stderr:Modular dependency problems: 2024-03-07T06:48:06.682 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-03-07T06:48:06.682 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 1: conflicting requests 2024-03-07T06:48:06.682 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240227110532:20125149.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.682 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 2: conflicting requests 2024-03-07T06:48:06.682 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.682 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 3: conflicting requests 2024-03-07T06:48:06.682 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.682 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 4: conflicting requests 2024-03-07T06:48:06.682 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.682 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 5: conflicting requests 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820240213202549:b754926a.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 6: conflicting requests 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820240216084734:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 7: conflicting requests 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module idm:client:820240222150954:49cc9d1b.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 8: conflicting requests 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 9: conflicting requests 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 10: conflicting requests 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 11: conflicting requests 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 12: conflicting requests 2024-03-07T06:48:06.683 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.684 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 13: conflicting requests 2024-03-07T06:48:06.684 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.684 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 14: conflicting requests 2024-03-07T06:48:06.684 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.684 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 15: conflicting requests 2024-03-07T06:48:06.684 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.684 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 16: conflicting requests 2024-03-07T06:48:06.684 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.684 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 17: conflicting requests 2024-03-07T06:48:06.684 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.684 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 18: conflicting requests 2024-03-07T06:48:06.684 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.684 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 19: conflicting requests 2024-03-07T06:48:06.684 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.685 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 20: conflicting requests 2024-03-07T06:48:06.685 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.685 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 21: conflicting requests 2024-03-07T06:48:06.685 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.685 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 22: conflicting requests 2024-03-07T06:48:06.685 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.685 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 23: conflicting requests 2024-03-07T06:48:06.685 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240208011952:182f7c73.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.685 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 24: conflicting requests 2024-03-07T06:48:06.686 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.686 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 25: conflicting requests 2024-03-07T06:48:06.686 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.686 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 26: conflicting requests 2024-03-07T06:48:06.686 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240214182535:17377f89.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.686 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 27: conflicting requests 2024-03-07T06:48:06.686 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.686 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 28: conflicting requests 2024-03-07T06:48:06.686 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.686 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 29: conflicting requests 2024-03-07T06:48:06.686 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.686 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 30: conflicting requests 2024-03-07T06:48:06.686 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.686 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 31: conflicting requests 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 32: conflicting requests 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module squid:4:820240214183150:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 33: conflicting requests 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 34: conflicting requests 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 35: conflicting requests 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 36: conflicting requests 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240226094445:9edba152.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 37: conflicting requests 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.687 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 38: cannot install the best candidate for the job 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 39: conflicting requests 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 40: cannot install the best candidate for the job 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.688 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.689 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.689 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.689 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 41: cannot install the best candidate for the job 2024-03-07T06:48:06.689 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.689 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.689 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.689 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.689 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 42: cannot install the best candidate for the job 2024-03-07T06:48:06.689 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.689 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.689 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.689 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.689 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 43: cannot install the best candidate for the job 2024-03-07T06:48:06.689 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.690 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.690 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.690 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.690 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 44: cannot install the best candidate for the job 2024-03-07T06:48:06.690 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.690 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.690 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.690 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.691 INFO:teuthology.orchestra.run.smithi161.stderr: Problem 45: cannot install the best candidate for the job 2024-03-07T06:48:06.691 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.691 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.691 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.691 INFO:teuthology.orchestra.run.smithi161.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-03-07T06:48:06.694 INFO:teuthology.orchestra.run.smithi133.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-07T06:48:06.696 INFO:teuthology.orchestra.run.smithi133.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-07T06:48:06.697 INFO:teuthology.orchestra.run.smithi133.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-07T06:48:06.698 INFO:teuthology.orchestra.run.smithi133.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-07T06:48:06.701 INFO:teuthology.orchestra.run.smithi133.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-07T06:48:06.705 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-07T06:48:06.706 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-07T06:48:06.807 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-03-07T06:48:06.829 INFO:teuthology.orchestra.run.smithi133.stdout:========================================================================================= 2024-03-07T06:48:06.829 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-03-07T06:48:06.829 INFO:teuthology.orchestra.run.smithi133.stdout:========================================================================================= 2024-03-07T06:48:06.829 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-03-07T06:48:06.829 INFO:teuthology.orchestra.run.smithi133.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-03-07T06:48:06.829 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 6.6 k 2024-03-07T06:48:06.829 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.3 M 2024-03-07T06:48:06.829 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 889 k 2024-03-07T06:48:06.829 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 145 k 2024-03-07T06:48:06.829 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 1.5 M 2024-03-07T06:48:06.829 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 138 k 2024-03-07T06:48:06.829 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 1.8 M 2024-03-07T06:48:06.830 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 7.4 M 2024-03-07T06:48:06.830 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 49 k 2024-03-07T06:48:06.830 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 9.8 M 2024-03-07T06:48:06.830 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 47 M 2024-03-07T06:48:06.830 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 264 k 2024-03-07T06:48:06.830 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 748 k 2024-03-07T06:48:06.830 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 220 k 2024-03-07T06:48:06.830 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 27 k 2024-03-07T06:48:06.830 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 18 k 2024-03-07T06:48:06.830 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-07T06:48:06.830 INFO:teuthology.orchestra.run.smithi133.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-03-07T06:48:06.830 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-03-07T06:48:06.830 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 31 k 2024-03-07T06:48:06.830 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 746 k 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 127 k 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 161 k 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 320 k 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 298 k 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 99 k 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 85 k 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 170 k 2024-03-07T06:48:06.831 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-07T06:48:06.832 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2024-03-07T06:48:06.832 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-07T06:48:06.832 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-07T06:48:06.832 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-07T06:48:06.832 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-07T06:48:06.832 INFO:teuthology.orchestra.run.smithi133.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-07T06:48:06.832 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-07T06:48:06.832 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-07T06:48:06.832 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-07T06:48:06.832 INFO:teuthology.orchestra.run.smithi133.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-07T06:48:06.832 INFO:teuthology.orchestra.run.smithi133.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-07T06:48:06.832 INFO:teuthology.orchestra.run.smithi133.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-07T06:48:06.833 INFO:teuthology.orchestra.run.smithi133.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-07T06:48:06.833 INFO:teuthology.orchestra.run.smithi133.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-07T06:48:06.833 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.4 M 2024-03-07T06:48:06.833 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-07T06:48:06.833 INFO:teuthology.orchestra.run.smithi133.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-07T06:48:06.833 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-07T06:48:06.833 INFO:teuthology.orchestra.run.smithi133.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-07T06:48:06.833 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-07T06:48:06.833 INFO:teuthology.orchestra.run.smithi133.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-07T06:48:06.833 INFO:teuthology.orchestra.run.smithi133.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-07T06:48:06.833 INFO:teuthology.orchestra.run.smithi133.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-07T06:48:06.833 INFO:teuthology.orchestra.run.smithi133.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-03-07T06:48:06.833 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-07T06:48:06.833 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 20 M 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 31 k 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 2.3 M 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 248 k 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 4.7 M 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 17 M 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 16 k 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 25 k 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-03-07T06:48:06.834 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-03-07T06:48:06.835 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-07T06:48:06.835 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-07T06:48:06.835 INFO:teuthology.orchestra.run.smithi133.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-07T06:48:06.835 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-07T06:48:06.835 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-07T06:48:06.835 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 163 k 2024-03-07T06:48:06.835 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-07T06:48:06.835 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-07T06:48:06.835 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-03-07T06:48:06.835 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-07T06:48:06.835 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-03-07T06:48:06.835 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 501 k 2024-03-07T06:48:06.835 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-03-07T06:48:06.835 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.1 M 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-07T06:48:06.836 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-03-07T06:48:06.837 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-03-07T06:48:06.837 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-07T06:48:06.837 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-07T06:48:06.837 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 45 k 2024-03-07T06:48:06.837 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 131 k 2024-03-07T06:48:06.837 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-07T06:48:06.837 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-03-07T06:48:06.837 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-07T06:48:06.837 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-07T06:48:06.837 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-07T06:48:06.837 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-07T06:48:06.837 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-03-07T06:48:06.837 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-07T06:48:06.837 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-03-07T06:48:06.838 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-03-07T06:48:06.839 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-07T06:48:06.840 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-03-07T06:48:06.840 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-07T06:48:06.840 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-07T06:48:06.840 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-03-07T06:48:06.840 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-07T06:48:06.840 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-07T06:48:06.840 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-03-07T06:48:06.840 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-07T06:48:06.840 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-03-07T06:48:06.840 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-07T06:48:06.840 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-03-07T06:48:06.840 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-03-07T06:48:06.840 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-07T06:48:06.840 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout:Installing weak dependencies: 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-03-07T06:48:06.841 INFO:teuthology.orchestra.run.smithi133.stdout:========================================================================================= 2024-03-07T06:48:06.842 INFO:teuthology.orchestra.run.smithi133.stdout:Install 150 Packages 2024-03-07T06:48:06.842 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 24 Packages 2024-03-07T06:48:06.842 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:48:06.842 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 226 M 2024-03-07T06:48:06.842 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-03-07T06:48:07.259 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-07T06:48:07.260 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-07T06:48:07.276 INFO:teuthology.orchestra.run.smithi161.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-07T06:48:07.276 INFO:teuthology.orchestra.run.smithi161.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-07T06:48:07.276 INFO:teuthology.orchestra.run.smithi161.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-07T06:48:07.276 INFO:teuthology.orchestra.run.smithi161.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-07T06:48:07.278 INFO:teuthology.orchestra.run.smithi161.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-07T06:48:07.281 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-07T06:48:07.282 INFO:teuthology.orchestra.run.smithi161.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-07T06:48:07.379 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2024-03-07T06:48:07.408 INFO:teuthology.orchestra.run.smithi161.stdout:========================================================================================= 2024-03-07T06:48:07.408 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2024-03-07T06:48:07.408 INFO:teuthology.orchestra.run.smithi161.stdout:========================================================================================= 2024-03-07T06:48:07.408 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2024-03-07T06:48:07.408 INFO:teuthology.orchestra.run.smithi161.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-03-07T06:48:07.408 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 6.6 k 2024-03-07T06:48:07.408 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.3 M 2024-03-07T06:48:07.409 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 889 k 2024-03-07T06:48:07.409 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 145 k 2024-03-07T06:48:07.409 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 1.5 M 2024-03-07T06:48:07.409 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 138 k 2024-03-07T06:48:07.409 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 1.8 M 2024-03-07T06:48:07.409 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 7.4 M 2024-03-07T06:48:07.409 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 49 k 2024-03-07T06:48:07.409 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 9.8 M 2024-03-07T06:48:07.409 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 47 M 2024-03-07T06:48:07.409 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 264 k 2024-03-07T06:48:07.409 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 748 k 2024-03-07T06:48:07.409 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 220 k 2024-03-07T06:48:07.409 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 27 k 2024-03-07T06:48:07.409 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 18 k 2024-03-07T06:48:07.410 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-07T06:48:07.410 INFO:teuthology.orchestra.run.smithi161.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-03-07T06:48:07.410 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 18 k 2024-03-07T06:48:07.410 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 31 k 2024-03-07T06:48:07.410 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 746 k 2024-03-07T06:48:07.410 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 127 k 2024-03-07T06:48:07.410 INFO:teuthology.orchestra.run.smithi161.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-03-07T06:48:07.410 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-07T06:48:07.410 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-07T06:48:07.410 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql x86_64 13.11-1.el9 appstream 1.6 M 2024-03-07T06:48:07.410 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib x86_64 13.11-1.el9 appstream 905 k 2024-03-07T06:48:07.410 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server x86_64 13.11-1.el9 appstream 5.7 M 2024-03-07T06:48:07.410 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 161 k 2024-03-07T06:48:07.410 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 320 k 2024-03-07T06:48:07.411 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 298 k 2024-03-07T06:48:07.411 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 99 k 2024-03-07T06:48:07.411 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 85 k 2024-03-07T06:48:07.411 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-07T06:48:07.411 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 170 k 2024-03-07T06:48:07.411 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-07T06:48:07.411 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2024-03-07T06:48:07.411 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager x86_64 1:1.46.0-1.el9 baseos 2.3 M 2024-03-07T06:48:07.411 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-libnm x86_64 1:1.46.0-1.el9 baseos 1.8 M 2024-03-07T06:48:07.411 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-team x86_64 1:1.46.0-1.el9 baseos 42 k 2024-03-07T06:48:07.411 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-tui x86_64 1:1.46.0-1.el9 baseos 248 k 2024-03-07T06:48:07.411 INFO:teuthology.orchestra.run.smithi161.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-07T06:48:07.411 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-07T06:48:07.411 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-07T06:48:07.412 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-07T06:48:07.412 INFO:teuthology.orchestra.run.smithi161.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-07T06:48:07.412 INFO:teuthology.orchestra.run.smithi161.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-07T06:48:07.412 INFO:teuthology.orchestra.run.smithi161.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-07T06:48:07.412 INFO:teuthology.orchestra.run.smithi161.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-07T06:48:07.412 INFO:teuthology.orchestra.run.smithi161.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-07T06:48:07.412 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.4 M 2024-03-07T06:48:07.412 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 3.1 M 2024-03-07T06:48:07.412 INFO:teuthology.orchestra.run.smithi161.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-07T06:48:07.412 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-07T06:48:07.412 INFO:teuthology.orchestra.run.smithi161.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-07T06:48:07.412 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-07T06:48:07.412 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-07T06:48:07.412 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout: xfsdump x86_64 3.1.12-4.el9 baseos 323 k 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 20 M 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 31 k 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 2.3 M 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 248 k 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 4.7 M 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 17 M 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1848.g8b896a9e.el9 ceph-noarch 16 k 2024-03-07T06:48:07.413 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 25 k 2024-03-07T06:48:07.414 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-03-07T06:48:07.414 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-03-07T06:48:07.414 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-03-07T06:48:07.414 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-03-07T06:48:07.414 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-07T06:48:07.414 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-07T06:48:07.414 INFO:teuthology.orchestra.run.smithi161.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-07T06:48:07.414 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-07T06:48:07.414 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-07T06:48:07.414 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 163 k 2024-03-07T06:48:07.414 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-07T06:48:07.414 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-07T06:48:07.414 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-03-07T06:48:07.414 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-07T06:48:07.415 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-03-07T06:48:07.415 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 501 k 2024-03-07T06:48:07.415 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-03-07T06:48:07.415 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 5.1 M 2024-03-07T06:48:07.415 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-07T06:48:07.415 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-03-07T06:48:07.415 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-03-07T06:48:07.415 INFO:teuthology.orchestra.run.smithi161.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-03-07T06:48:07.415 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-07T06:48:07.415 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-07T06:48:07.415 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma x86_64 6.9.6-1.el9.5 baseos 218 k 2024-03-07T06:48:07.415 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-03-07T06:48:07.415 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-03-07T06:48:07.415 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-07T06:48:07.416 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs x86_64 13.11-1.el9 appstream 135 k 2024-03-07T06:48:07.416 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-03-07T06:48:07.416 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-07T06:48:07.416 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-03-07T06:48:07.416 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-03-07T06:48:07.416 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-03-07T06:48:07.416 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-07T06:48:07.416 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-07T06:48:07.416 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 45 k 2024-03-07T06:48:07.416 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:19.0.0-1848.g8b896a9e.el9 ceph 131 k 2024-03-07T06:48:07.416 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-07T06:48:07.416 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-03-07T06:48:07.416 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-07T06:48:07.416 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-07T06:48:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-07T06:48:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-07T06:48:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-03-07T06:48:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-07T06:48:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:2.28.0-1.el9 epel 203 k 2024-03-07T06:48:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-07T06:48:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-07T06:48:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-07T06:48:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-07T06:48:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-07T06:48:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-07T06:48:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-07T06:48:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-07T06:48:07.417 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-07T06:48:07.418 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-07T06:48:07.419 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-03-07T06:48:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-07T06:48:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-03-07T06:48:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-03-07T06:48:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-07T06:48:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-07T06:48:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-07T06:48:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-07T06:48:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-07T06:48:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-07T06:48:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-07T06:48:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-03-07T06:48:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-07T06:48:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-03-07T06:48:07.420 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-03-07T06:48:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2024-03-07T06:48:07.421 INFO:teuthology.orchestra.run.smithi161.stdout:Installing weak dependencies: 2024-03-07T06:48:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-07T06:48:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:48:07.421 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2024-03-07T06:48:07.421 INFO:teuthology.orchestra.run.smithi161.stdout:========================================================================================= 2024-03-07T06:48:07.421 INFO:teuthology.orchestra.run.smithi161.stdout:Install 150 Packages 2024-03-07T06:48:07.421 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 24 Packages 2024-03-07T06:48:07.421 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:48:07.422 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 226 M 2024-03-07T06:48:07.422 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2024-03-07T06:48:07.753 INFO:teuthology.orchestra.run.smithi133.stdout:(1/174): ceph-19.0.0-1848.g8b896a9e.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-03-07T06:48:08.178 INFO:teuthology.orchestra.run.smithi133.stdout:(2/174): ceph-fuse-19.0.0-1848.g8b896a9e.el9.x8 2.0 MB/s | 889 kB 00:00 2024-03-07T06:48:08.288 INFO:teuthology.orchestra.run.smithi133.stdout:(3/174): ceph-immutable-object-cache-19.0.0-184 1.3 MB/s | 145 kB 00:00 2024-03-07T06:48:08.294 INFO:teuthology.orchestra.run.smithi161.stdout:(1/174): ceph-19.0.0-1848.g8b896a9e.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-03-07T06:48:08.953 INFO:teuthology.orchestra.run.smithi161.stdout:(2/174): ceph-fuse-19.0.0-1848.g8b896a9e.el9.x8 1.3 MB/s | 889 kB 00:00 2024-03-07T06:48:09.012 INFO:teuthology.orchestra.run.smithi133.stdout:(4/174): ceph-common-19.0.0-1848.g8b896a9e.el9. 13 MB/s | 20 MB 00:01 2024-03-07T06:48:09.019 INFO:teuthology.orchestra.run.smithi161.stdout:(3/174): ceph-immutable-object-cache-19.0.0-184 2.2 MB/s | 145 kB 00:00 2024-03-07T06:48:09.103 INFO:teuthology.orchestra.run.smithi133.stdout:(5/174): ceph-mds-19.0.0-1848.g8b896a9e.el9.x86 2.8 MB/s | 2.3 MB 00:00 2024-03-07T06:48:09.179 INFO:teuthology.orchestra.run.smithi133.stdout:(6/174): ceph-mgr-19.0.0-1848.g8b896a9e.el9.x86 9.0 MB/s | 1.5 MB 00:00 2024-03-07T06:48:09.279 INFO:teuthology.orchestra.run.smithi133.stdout:(7/174): ceph-base-19.0.0-1848.g8b896a9e.el9.x8 3.0 MB/s | 5.3 MB 00:01 2024-03-07T06:48:10.137 INFO:teuthology.orchestra.run.smithi161.stdout:(4/174): ceph-mds-19.0.0-1848.g8b896a9e.el9.x86 2.1 MB/s | 2.3 MB 00:01 2024-03-07T06:48:10.570 INFO:teuthology.orchestra.run.smithi161.stdout:(5/174): ceph-mgr-19.0.0-1848.g8b896a9e.el9.x86 3.5 MB/s | 1.5 MB 00:00 2024-03-07T06:48:10.804 INFO:teuthology.orchestra.run.smithi161.stdout:(6/174): ceph-base-19.0.0-1848.g8b896a9e.el9.x8 1.9 MB/s | 5.3 MB 00:02 2024-03-07T06:48:10.829 INFO:teuthology.orchestra.run.smithi133.stdout:(8/174): ceph-osd-19.0.0-1848.g8b896a9e.el9.x86 10 MB/s | 17 MB 00:01 2024-03-07T06:48:10.954 INFO:teuthology.orchestra.run.smithi133.stdout:(9/174): ceph-mon-19.0.0-1848.g8b896a9e.el9.x86 2.5 MB/s | 4.7 MB 00:01 2024-03-07T06:48:10.971 INFO:teuthology.orchestra.run.smithi133.stdout:(10/174): ceph-selinux-19.0.0-1848.g8b896a9e.el 178 kB/s | 25 kB 00:00 2024-03-07T06:48:11.038 INFO:teuthology.orchestra.run.smithi133.stdout:(11/174): cephfs-mirror-19.0.0-1848.g8b896a9e.e 3.3 MB/s | 220 kB 00:00 2024-03-07T06:48:11.105 INFO:teuthology.orchestra.run.smithi133.stdout:(12/174): cephfs-shell-19.0.0-1848.g8b896a9e.el 411 kB/s | 27 kB 00:00 2024-03-07T06:48:11.172 INFO:teuthology.orchestra.run.smithi133.stdout:(13/174): libcephfs-devel-19.0.0-1848.g8b896a9e 474 kB/s | 31 kB 00:00 2024-03-07T06:48:11.314 INFO:teuthology.orchestra.run.smithi133.stdout:(14/174): libcephfs2-19.0.0-1848.g8b896a9e.el9. 5.2 MB/s | 746 kB 00:00 2024-03-07T06:48:11.389 INFO:teuthology.orchestra.run.smithi133.stdout:(15/174): libcephsqlite-19.0.0-1848.g8b896a9e.e 2.2 MB/s | 163 kB 00:00 2024-03-07T06:48:11.615 INFO:teuthology.orchestra.run.smithi133.stdout:(16/174): ceph-radosgw-19.0.0-1848.g8b896a9e.el 4.2 MB/s | 9.8 MB 00:02 2024-03-07T06:48:11.648 INFO:teuthology.orchestra.run.smithi133.stdout:(17/174): librados-devel-19.0.0-1848.g8b896a9e. 491 kB/s | 127 kB 00:00 2024-03-07T06:48:11.779 INFO:teuthology.orchestra.run.smithi161.stdout:(7/174): ceph-mon-19.0.0-1848.g8b896a9e.el9.x86 3.9 MB/s | 4.7 MB 00:01 2024-03-07T06:48:11.807 INFO:teuthology.orchestra.run.smithi133.stdout:(18/174): libradosstriper1-19.0.0-1848.g8b896a9 2.6 MB/s | 501 kB 00:00 2024-03-07T06:48:11.932 INFO:teuthology.orchestra.run.smithi133.stdout:(19/174): python3-ceph-argparse-19.0.0-1848.g8b 364 kB/s | 45 kB 00:00 2024-03-07T06:48:12.049 INFO:teuthology.orchestra.run.smithi133.stdout:(20/174): python3-ceph-common-19.0.0-1848.g8b89 1.1 MB/s | 131 kB 00:00 2024-03-07T06:48:12.166 INFO:teuthology.orchestra.run.smithi133.stdout:(21/174): python3-cephfs-19.0.0-1848.g8b896a9e. 1.4 MB/s | 161 kB 00:00 2024-03-07T06:48:12.267 INFO:teuthology.orchestra.run.smithi133.stdout:(22/174): librgw2-19.0.0-1848.g8b896a9e.el9.x86 8.3 MB/s | 5.1 MB 00:00 2024-03-07T06:48:12.300 INFO:teuthology.orchestra.run.smithi133.stdout:(23/174): python3-rados-19.0.0-1848.g8b896a9e.e 2.4 MB/s | 320 kB 00:00 2024-03-07T06:48:12.342 INFO:teuthology.orchestra.run.smithi133.stdout:(24/174): python3-rbd-19.0.0-1848.g8b896a9e.el9 3.9 MB/s | 298 kB 00:00 2024-03-07T06:48:12.392 INFO:teuthology.orchestra.run.smithi133.stdout:(25/174): rbd-fuse-19.0.0-1848.g8b896a9e.el9.x8 1.7 MB/s | 85 kB 00:00 2024-03-07T06:48:12.409 INFO:teuthology.orchestra.run.smithi133.stdout:(26/174): python3-rgw-19.0.0-1848.g8b896a9e.el9 917 kB/s | 99 kB 00:00 2024-03-07T06:48:12.509 INFO:teuthology.orchestra.run.smithi133.stdout:(27/174): rbd-nbd-19.0.0-1848.g8b896a9e.el9.x86 1.7 MB/s | 170 kB 00:00 2024-03-07T06:48:12.579 INFO:teuthology.orchestra.run.smithi161.stdout:(8/174): ceph-common-19.0.0-1848.g8b896a9e.el9. 4.5 MB/s | 20 MB 00:04 2024-03-07T06:48:12.593 INFO:teuthology.orchestra.run.smithi133.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1848.g 381 kB/s | 31 kB 00:00 2024-03-07T06:48:12.638 INFO:teuthology.orchestra.run.smithi161.stdout:(9/174): ceph-selinux-19.0.0-1848.g8b896a9e.el9 440 kB/s | 25 kB 00:00 2024-03-07T06:48:12.677 INFO:teuthology.orchestra.run.smithi133.stdout:(29/174): rbd-mirror-19.0.0-1848.g8b896a9e.el9. 11 MB/s | 3.1 MB 00:00 2024-03-07T06:48:12.735 INFO:teuthology.orchestra.run.smithi133.stdout:(30/174): ceph-mgr-cephadm-19.0.0-1848.g8b896a9 980 kB/s | 138 kB 00:00 2024-03-07T06:48:12.894 INFO:teuthology.orchestra.run.smithi133.stdout:(31/174): ceph-mgr-dashboard-19.0.0-1848.g8b896 8.2 MB/s | 1.8 MB 00:00 2024-03-07T06:48:12.953 INFO:teuthology.orchestra.run.smithi133.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1848.g8b 4.2 MB/s | 248 kB 00:00 2024-03-07T06:48:13.003 INFO:teuthology.orchestra.run.smithi133.stdout:(33/174): ceph-mgr-rook-19.0.0-1848.g8b896a9e.e 1.0 MB/s | 49 kB 00:00 2024-03-07T06:48:13.046 INFO:teuthology.orchestra.run.smithi161.stdout:(10/174): ceph-osd-19.0.0-1848.g8b896a9e.el9.x8 7.5 MB/s | 17 MB 00:02 2024-03-07T06:48:13.061 INFO:teuthology.orchestra.run.smithi133.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1848.g8 286 kB/s | 16 kB 00:00 2024-03-07T06:48:13.122 INFO:teuthology.orchestra.run.smithi161.stdout:(11/174): cephfs-mirror-19.0.0-1848.g8b896a9e.e 2.9 MB/s | 220 kB 00:00 2024-03-07T06:48:13.137 INFO:teuthology.orchestra.run.smithi133.stdout:(35/174): ceph-volume-19.0.0-1848.g8b896a9e.el9 3.5 MB/s | 264 kB 00:00 2024-03-07T06:48:13.197 INFO:teuthology.orchestra.run.smithi161.stdout:(12/174): cephfs-shell-19.0.0-1848.g8b896a9e.el 365 kB/s | 27 kB 00:00 2024-03-07T06:48:13.220 INFO:teuthology.orchestra.run.smithi133.stdout:(36/174): cephadm-19.0.0-1848.g8b896a9e.el9.noa 8.8 MB/s | 748 kB 00:00 2024-03-07T06:48:13.272 INFO:teuthology.orchestra.run.smithi161.stdout:(13/174): libcephfs-devel-19.0.0-1848.g8b896a9e 422 kB/s | 31 kB 00:00 2024-03-07T06:48:13.279 INFO:teuthology.orchestra.run.smithi133.stdout:(37/174): cephfs-top-19.0.0-1848.g8b896a9e.el9. 318 kB/s | 18 kB 00:00 2024-03-07T06:48:13.429 INFO:teuthology.orchestra.run.smithi133.stdout:(38/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 118 kB/s | 18 kB 00:00 2024-03-07T06:48:13.439 INFO:teuthology.orchestra.run.smithi161.stdout:(14/174): libcephfs2-19.0.0-1848.g8b896a9e.el9. 4.4 MB/s | 746 kB 00:00 2024-03-07T06:48:13.531 INFO:teuthology.orchestra.run.smithi161.stdout:(15/174): libcephsqlite-19.0.0-1848.g8b896a9e.e 1.8 MB/s | 163 kB 00:00 2024-03-07T06:48:13.606 INFO:teuthology.orchestra.run.smithi161.stdout:(16/174): librados-devel-19.0.0-1848.g8b896a9e. 1.7 MB/s | 127 kB 00:00 2024-03-07T06:48:13.707 INFO:teuthology.orchestra.run.smithi161.stdout:(17/174): libradosstriper1-19.0.0-1848.g8b896a9 4.9 MB/s | 501 kB 00:00 2024-03-07T06:48:14.038 INFO:teuthology.orchestra.run.smithi133.stdout:(39/174): ceph-mgr-diskprediction-local-19.0.0- 5.7 MB/s | 7.4 MB 00:01 2024-03-07T06:48:14.197 INFO:teuthology.orchestra.run.smithi133.stdout:(40/174): fuse-2.9.9-15.el9.x86_64.rpm 105 kB/s | 80 kB 00:00 2024-03-07T06:48:14.282 INFO:teuthology.orchestra.run.smithi161.stdout:(18/174): librgw2-19.0.0-1848.g8b896a9e.el9.x86 8.9 MB/s | 5.1 MB 00:00 2024-03-07T06:48:14.549 INFO:teuthology.orchestra.run.smithi161.stdout:(19/174): ceph-radosgw-19.0.0-1848.g8b896a9e.el 3.5 MB/s | 9.8 MB 00:02 2024-03-07T06:48:14.607 INFO:teuthology.orchestra.run.smithi161.stdout:(20/174): python3-ceph-argparse-19.0.0-1848.g8b 139 kB/s | 45 kB 00:00 2024-03-07T06:48:14.641 INFO:teuthology.orchestra.run.smithi161.stdout:(21/174): python3-ceph-common-19.0.0-1848.g8b89 1.4 MB/s | 131 kB 00:00 2024-03-07T06:48:14.674 INFO:teuthology.orchestra.run.smithi161.stdout:(22/174): python3-cephfs-19.0.0-1848.g8b896a9e. 2.4 MB/s | 161 kB 00:00 2024-03-07T06:48:14.733 INFO:teuthology.orchestra.run.smithi161.stdout:(23/174): python3-rados-19.0.0-1848.g8b896a9e.e 3.4 MB/s | 320 kB 00:00 2024-03-07T06:48:14.775 INFO:teuthology.orchestra.run.smithi161.stdout:(24/174): python3-rbd-19.0.0-1848.g8b896a9e.el9 2.9 MB/s | 298 kB 00:00 2024-03-07T06:48:14.801 INFO:teuthology.orchestra.run.smithi161.stdout:(25/174): python3-rgw-19.0.0-1848.g8b896a9e.el9 1.5 MB/s | 99 kB 00:00 2024-03-07T06:48:14.834 INFO:teuthology.orchestra.run.smithi161.stdout:(26/174): rbd-fuse-19.0.0-1848.g8b896a9e.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-07T06:48:14.901 INFO:teuthology.orchestra.run.smithi161.stdout:(27/174): rbd-nbd-19.0.0-1848.g8b896a9e.el9.x86 2.6 MB/s | 170 kB 00:00 2024-03-07T06:48:14.993 INFO:teuthology.orchestra.run.smithi161.stdout:(28/174): ceph-grafana-dashboards-19.0.0-1848.g 346 kB/s | 31 kB 00:00 2024-03-07T06:48:15.031 INFO:teuthology.orchestra.run.smithi133.stdout:(41/174): jq-1.6-16.el9.x86_64.rpm 189 kB/s | 187 kB 00:00 2024-03-07T06:48:15.060 INFO:teuthology.orchestra.run.smithi161.stdout:(29/174): ceph-mgr-cephadm-19.0.0-1848.g8b896a9 2.1 MB/s | 138 kB 00:00 2024-03-07T06:48:15.294 INFO:teuthology.orchestra.run.smithi161.stdout:(30/174): ceph-mgr-dashboard-19.0.0-1848.g8b896 7.7 MB/s | 1.8 MB 00:00 2024-03-07T06:48:15.452 INFO:teuthology.orchestra.run.smithi161.stdout:(31/174): rbd-mirror-19.0.0-1848.g8b896a9e.el9. 4.7 MB/s | 3.1 MB 00:00 2024-03-07T06:48:15.553 INFO:teuthology.orchestra.run.smithi161.stdout:(32/174): ceph-mgr-modules-core-19.0.0-1848.g8b 2.5 MB/s | 248 kB 00:00 2024-03-07T06:48:15.611 INFO:teuthology.orchestra.run.smithi161.stdout:(33/174): ceph-mgr-rook-19.0.0-1848.g8b896a9e.e 860 kB/s | 49 kB 00:00 2024-03-07T06:48:15.670 INFO:teuthology.orchestra.run.smithi161.stdout:(34/174): ceph-prometheus-alerts-19.0.0-1848.g8 287 kB/s | 16 kB 00:00 2024-03-07T06:48:15.787 INFO:teuthology.orchestra.run.smithi161.stdout:(35/174): ceph-volume-19.0.0-1848.g8b896a9e.el9 2.2 MB/s | 264 kB 00:00 2024-03-07T06:48:15.872 INFO:teuthology.orchestra.run.smithi133.stdout:(42/174): libgfortran-11.4.1-3.el9.x86_64.rpm 480 kB/s | 803 kB 00:01 2024-03-07T06:48:15.939 INFO:teuthology.orchestra.run.smithi133.stdout:(43/174): libquadmath-11.4.1-3.el9.x86_64.rpm 215 kB/s | 195 kB 00:00 2024-03-07T06:48:16.037 INFO:teuthology.orchestra.run.smithi161.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-03-07T06:48:16.079 INFO:teuthology.orchestra.run.smithi161.stdout:(37/174): cephadm-19.0.0-1848.g8b896a9e.el9.noa 2.5 MB/s | 748 kB 00:00 2024-03-07T06:48:16.104 INFO:teuthology.orchestra.run.smithi161.stdout:(38/174): cephfs-top-19.0.0-1848.g8b896a9e.el9. 274 kB/s | 18 kB 00:00 2024-03-07T06:48:16.230 INFO:teuthology.orchestra.run.smithi161.stdout:(39/174): libaio-devel-0.3.112-1.el8.x86_64.rpm 117 kB/s | 18 kB 00:00 2024-03-07T06:48:16.280 INFO:teuthology.orchestra.run.smithi161.stdout:(40/174): fuse-2.9.9-15.el9.x86_64.rpm 461 kB/s | 80 kB 00:00 2024-03-07T06:48:16.422 INFO:teuthology.orchestra.run.smithi161.stdout:(41/174): jq-1.6-16.el9.x86_64.rpm 980 kB/s | 187 kB 00:00 2024-03-07T06:48:16.464 INFO:teuthology.orchestra.run.smithi161.stdout:(42/174): libgfortran-11.4.1-3.el9.x86_64.rpm 4.3 MB/s | 803 kB 00:00 2024-03-07T06:48:16.497 INFO:teuthology.orchestra.run.smithi161.stdout:(43/174): libquadmath-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2024-03-07T06:48:16.557 INFO:teuthology.orchestra.run.smithi161.stdout:(44/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.3 MB/s | 218 kB 00:00 2024-03-07T06:48:16.590 INFO:teuthology.orchestra.run.smithi161.stdout:(45/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-03-07T06:48:16.640 INFO:teuthology.orchestra.run.smithi161.stdout:(46/174): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-03-07T06:48:16.707 INFO:teuthology.orchestra.run.smithi161.stdout:(47/174): python3-cryptography-36.0.1-4.el9.x86 8.2 MB/s | 1.2 MB 00:00 2024-03-07T06:48:16.740 INFO:teuthology.orchestra.run.smithi161.stdout:(48/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-03-07T06:48:16.774 INFO:teuthology.orchestra.run.smithi161.stdout:(49/174): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-03-07T06:48:16.808 INFO:teuthology.orchestra.run.smithi161.stdout:(50/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 215 kB 00:00 2024-03-07T06:48:16.814 INFO:teuthology.orchestra.run.smithi133.stdout:(44/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 232 kB/s | 218 kB 00:00 2024-03-07T06:48:16.914 INFO:teuthology.orchestra.run.smithi133.stdout:(45/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 260 kB/s | 253 kB 00:00 2024-03-07T06:48:16.966 INFO:teuthology.orchestra.run.smithi161.stdout:(51/174): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-03-07T06:48:17.025 INFO:teuthology.orchestra.run.smithi161.stdout:(52/174): bison-3.7.4-5.el9.x86_64.rpm 4.3 MB/s | 943 kB 00:00 2024-03-07T06:48:17.059 INFO:teuthology.orchestra.run.smithi161.stdout:(53/174): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-03-07T06:48:17.084 INFO:teuthology.orchestra.run.smithi161.stdout:(54/174): elfutils-libelf-devel-0.190-2.el9.x86 373 kB/s | 21 kB 00:00 2024-03-07T06:48:17.118 INFO:teuthology.orchestra.run.smithi161.stdout:(55/174): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-03-07T06:48:17.134 INFO:teuthology.orchestra.run.smithi161.stdout:(56/174): flexiblas-3.0.4-8.el9.x86_64.rpm 659 kB/s | 33 kB 00:00 2024-03-07T06:48:17.168 INFO:teuthology.orchestra.run.smithi161.stdout:(57/174): flexiblas-openblas-openmp-3.0.4-8.el9 560 kB/s | 18 kB 00:00 2024-03-07T06:48:17.243 INFO:teuthology.orchestra.run.smithi161.stdout:(58/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-03-07T06:48:17.260 INFO:teuthology.orchestra.run.smithi161.stdout:(59/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-03-07T06:48:17.277 INFO:teuthology.orchestra.run.smithi161.stdout:(60/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-03-07T06:48:17.344 INFO:teuthology.orchestra.run.smithi161.stdout:(61/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-03-07T06:48:17.395 INFO:teuthology.orchestra.run.smithi161.stdout:(62/174): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-03-07T06:48:17.411 INFO:teuthology.orchestra.run.smithi161.stdout:(63/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 387 kB/s | 25 kB 00:00 2024-03-07T06:48:17.428 INFO:teuthology.orchestra.run.smithi161.stdout:(64/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-03-07T06:48:17.462 INFO:teuthology.orchestra.run.smithi161.stdout:(65/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-03-07T06:48:17.487 INFO:teuthology.orchestra.run.smithi161.stdout:(66/174): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-03-07T06:48:17.513 INFO:teuthology.orchestra.run.smithi161.stdout:(67/174): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-03-07T06:48:17.673 INFO:teuthology.orchestra.run.smithi133.stdout:(46/174): python3-ply-3.11-14.el9.noarch.rpm 141 kB/s | 106 kB 00:00 2024-03-07T06:48:17.680 INFO:teuthology.orchestra.run.smithi161.stdout:(68/174): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-03-07T06:48:17.763 INFO:teuthology.orchestra.run.smithi161.stdout:(69/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 17 MB/s | 4.1 MB 00:00 2024-03-07T06:48:17.855 INFO:teuthology.orchestra.run.smithi161.stdout:(70/174): postgresql-13.11-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-03-07T06:48:17.889 INFO:teuthology.orchestra.run.smithi161.stdout:(71/174): postgresql-contrib-13.11-1.el9.x86_64 7.1 MB/s | 905 kB 00:00 2024-03-07T06:48:17.914 INFO:teuthology.orchestra.run.smithi161.stdout:(72/174): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 135 kB 00:00 2024-03-07T06:48:18.006 INFO:teuthology.orchestra.run.smithi161.stdout:(73/174): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-03-07T06:48:18.048 INFO:teuthology.orchestra.run.smithi161.stdout:(74/174): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-03-07T06:48:18.190 INFO:teuthology.orchestra.run.smithi161.stdout:(75/174): postgresql-server-13.11-1.el9.x86_64. 19 MB/s | 5.7 MB 00:00 2024-03-07T06:48:18.248 INFO:teuthology.orchestra.run.smithi161.stdout:(76/174): python3-devel-3.9.17-2.el9.x86_64.rpm 4.2 MB/s | 245 kB 00:00 2024-03-07T06:48:18.307 INFO:teuthology.orchestra.run.smithi161.stdout:(77/174): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-03-07T06:48:18.349 INFO:teuthology.orchestra.run.smithi161.stdout:(78/174): python3-jsonpatch-1.21-16.el9.noarch. 629 kB/s | 26 kB 00:00 2024-03-07T06:48:18.458 INFO:teuthology.orchestra.run.smithi161.stdout:(79/174): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-03-07T06:48:18.490 INFO:teuthology.orchestra.run.smithi133.stdout:(47/174): python3-pycparser-2.20-6.el9.noarch.r 166 kB/s | 135 kB 00:00 2024-03-07T06:48:18.503 INFO:teuthology.orchestra.run.smithi161.stdout:(80/174): python3-jsonpointer-2.0-4.el9.noarch. 126 kB/s | 19 kB 00:00 2024-03-07T06:48:18.542 INFO:teuthology.orchestra.run.smithi161.stdout:(81/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-03-07T06:48:18.567 INFO:teuthology.orchestra.run.smithi161.stdout:(82/174): python3-markupsafe-1.1.1-12.el9.x86_6 549 kB/s | 35 kB 00:00 2024-03-07T06:48:18.626 INFO:teuthology.orchestra.run.smithi161.stdout:(83/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-03-07T06:48:19.135 INFO:teuthology.orchestra.run.smithi161.stdout:(84/174): ceph-test-19.0.0-1848.g8b896a9e.el9.x 7.3 MB/s | 47 MB 00:06 2024-03-07T06:48:19.168 INFO:teuthology.orchestra.run.smithi161.stdout:(85/174): python3-oauthlib-3.1.1-5.el9.noarch.r 411 kB/s | 222 kB 00:00 2024-03-07T06:48:19.218 INFO:teuthology.orchestra.run.smithi161.stdout:(86/174): python3-prettytable-0.7.2-27.el9.noar 861 kB/s | 42 kB 00:00 2024-03-07T06:48:19.260 INFO:teuthology.orchestra.run.smithi161.stdout:(87/174): python3-packaging-20.9-5.el9.noarch.r 621 kB/s | 77 kB 00:00 2024-03-07T06:48:19.285 INFO:teuthology.orchestra.run.smithi161.stdout:(88/174): python3-protobuf-3.14.0-13.el9.noarch 4.0 MB/s | 269 kB 00:00 2024-03-07T06:48:19.307 INFO:teuthology.orchestra.run.smithi133.stdout:(48/174): ceph-test-19.0.0-1848.g8b896a9e.el9.x 5.7 MB/s | 47 MB 00:08 2024-03-07T06:48:19.357 INFO:teuthology.orchestra.run.smithi133.stdout:(49/174): python3-cryptography-36.0.1-4.el9.x86 494 kB/s | 1.2 MB 00:02 2024-03-07T06:48:19.386 INFO:teuthology.orchestra.run.smithi161.stdout:(89/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.0 MB/s | 5.1 MB 00:00 2024-03-07T06:48:19.420 INFO:teuthology.orchestra.run.smithi161.stdout:(90/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-03-07T06:48:19.467 INFO:teuthology.orchestra.run.smithi161.stdout:(91/174): python3-pytz-2021.1-5.el9.noarch.rpm 641 kB/s | 51 kB 00:00 2024-03-07T06:48:19.495 INFO:teuthology.orchestra.run.smithi161.stdout:(92/174): python3-requests-oauthlib-1.3.0-12.el 721 kB/s | 54 kB 00:00 2024-03-07T06:48:19.529 INFO:teuthology.orchestra.run.smithi161.stdout:(93/174): python3-pyasn1-modules-0.4.8-6.el9.no 1.1 MB/s | 279 kB 00:00 2024-03-07T06:48:19.532 INFO:teuthology.orchestra.run.smithi133.stdout:(50/174): automake-1.16.2-8.el9.noarch.rpm 3.9 MB/s | 689 kB 00:00 2024-03-07T06:48:19.753 INFO:teuthology.orchestra.run.smithi161.stdout:(94/174): python3-toml-0.10.2-6.el9.noarch.rpm 162 kB/s | 42 kB 00:00 2024-03-07T06:48:19.757 INFO:teuthology.orchestra.run.smithi133.stdout:(51/174): bison-3.7.4-5.el9.x86_64.rpm 5.9 MB/s | 943 kB 00:00 2024-03-07T06:48:19.770 INFO:teuthology.orchestra.run.smithi161.stdout:(95/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 183 kB/s | 44 kB 00:00 2024-03-07T06:48:19.791 INFO:teuthology.orchestra.run.smithi133.stdout:(52/174): python3-requests-2.25.1-8.el9.noarch. 96 kB/s | 125 kB 00:01 2024-03-07T06:48:19.804 INFO:teuthology.orchestra.run.smithi161.stdout:(96/174): qatlib-service-23.11.0-1.el9.x86_64.r 1.0 MB/s | 35 kB 00:00 2024-03-07T06:48:19.816 INFO:teuthology.orchestra.run.smithi133.stdout:(53/174): boost-program-options-1.75.0-8.el9.x8 1.9 MB/s | 107 kB 00:00 2024-03-07T06:48:19.861 INFO:teuthology.orchestra.run.smithi161.stdout:(97/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.0 MB/s | 216 kB 00:00 2024-03-07T06:48:19.875 INFO:teuthology.orchestra.run.smithi133.stdout:(54/174): elfutils-libelf-devel-0.190-2.el9.x86 260 kB/s | 21 kB 00:00 2024-03-07T06:48:19.888 INFO:teuthology.orchestra.run.smithi161.stdout:(98/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 535 kB/s | 44 kB 00:00 2024-03-07T06:48:19.909 INFO:teuthology.orchestra.run.smithi133.stdout:(55/174): flex-2.6.4-9.el9.x86_64.rpm 3.4 MB/s | 313 kB 00:00 2024-03-07T06:48:19.922 INFO:teuthology.orchestra.run.smithi161.stdout:(99/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 305 kB 00:00 2024-03-07T06:48:19.934 INFO:teuthology.orchestra.run.smithi133.stdout:(56/174): flexiblas-3.0.4-8.el9.x86_64.rpm 566 kB/s | 33 kB 00:00 2024-03-07T06:48:19.939 INFO:teuthology.orchestra.run.smithi161.stdout:(100/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-03-07T06:48:19.973 INFO:teuthology.orchestra.run.smithi161.stdout:(101/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.9 MB/s | 94 kB 00:00 2024-03-07T06:48:19.976 INFO:teuthology.orchestra.run.smithi133.stdout:(57/174): flexiblas-openblas-openmp-3.0.4-8.el9 444 kB/s | 18 kB 00:00 2024-03-07T06:48:19.998 INFO:teuthology.orchestra.run.smithi161.stdout:(102/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-03-07T06:48:20.023 INFO:teuthology.orchestra.run.smithi161.stdout:(103/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 916 kB/s | 45 kB 00:00 2024-03-07T06:48:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:(58/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-03-07T06:48:20.085 INFO:teuthology.orchestra.run.smithi133.stdout:(59/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-03-07T06:48:20.148 INFO:teuthology.orchestra.run.smithi161.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 149 kB/s | 22 kB 00:00 2024-03-07T06:48:20.210 INFO:teuthology.orchestra.run.smithi133.stdout:(60/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2024-03-07T06:48:20.244 INFO:teuthology.orchestra.run.smithi133.stdout:(61/174): python3-urllib3-1.26.5-5.el9.noarch.r 230 kB/s | 215 kB 00:00 2024-03-07T06:48:20.285 INFO:teuthology.orchestra.run.smithi133.stdout:(62/174): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-03-07T06:48:20.290 INFO:teuthology.orchestra.run.smithi161.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 3.2 MB/s | 863 kB 00:00 2024-03-07T06:48:20.319 INFO:teuthology.orchestra.run.smithi133.stdout:(63/174): libtool-2.4.6-45.el9.x86_64.rpm 5.3 MB/s | 582 kB 00:00 2024-03-07T06:48:20.344 INFO:teuthology.orchestra.run.smithi133.stdout:(64/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 256 kB/s | 25 kB 00:00 2024-03-07T06:48:20.393 INFO:teuthology.orchestra.run.smithi133.stdout:(65/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 443 kB/s | 47 kB 00:00 2024-03-07T06:48:20.428 INFO:teuthology.orchestra.run.smithi133.stdout:(66/174): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-03-07T06:48:20.454 INFO:teuthology.orchestra.run.smithi133.stdout:(67/174): openblas-0.3.21-2.el9.x86_64.rpm 594 kB/s | 35 kB 00:00 2024-03-07T06:48:20.487 INFO:teuthology.orchestra.run.smithi133.stdout:(68/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-03-07T06:48:20.549 INFO:teuthology.orchestra.run.smithi161.stdout:(106/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-03-07T06:48:20.674 INFO:teuthology.orchestra.run.smithi161.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-03-07T06:48:20.758 INFO:teuthology.orchestra.run.smithi161.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 96 kB/s | 19 kB 00:00 2024-03-07T06:48:20.788 INFO:teuthology.orchestra.run.smithi133.stdout:(69/174): postgresql-13.11-1.el9.x86_64.rpm 5.3 MB/s | 1.6 MB 00:00 2024-03-07T06:48:20.791 INFO:teuthology.orchestra.run.smithi161.stdout:(109/174): gperftools-libs-2.9.1-2.el9.x86_64.r 617 kB/s | 309 kB 00:00 2024-03-07T06:48:20.817 INFO:teuthology.orchestra.run.smithi161.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 449 kB/s | 26 kB 00:00 2024-03-07T06:48:20.850 INFO:teuthology.orchestra.run.smithi161.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 841 kB/s | 49 kB 00:00 2024-03-07T06:48:20.867 INFO:teuthology.orchestra.run.smithi161.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-03-07T06:48:20.917 INFO:teuthology.orchestra.run.smithi161.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-03-07T06:48:20.946 INFO:teuthology.orchestra.run.smithi133.stdout:(70/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 8.4 MB/s | 4.1 MB 00:00 2024-03-07T06:48:20.996 INFO:teuthology.orchestra.run.smithi133.stdout:(71/174): postgresql-private-libs-13.11-1.el9.x 2.7 MB/s | 135 kB 00:00 2024-03-07T06:48:21.071 INFO:teuthology.orchestra.run.smithi133.stdout:(72/174): postgresql-contrib-13.11-1.el9.x86_64 3.1 MB/s | 905 kB 00:00 2024-03-07T06:48:21.076 INFO:teuthology.orchestra.run.smithi161.stdout:(114/174): python3-asyncssh-2.13.2-4.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-03-07T06:48:21.135 INFO:teuthology.orchestra.run.smithi161.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 838 kB 00:00 2024-03-07T06:48:21.160 INFO:teuthology.orchestra.run.smithi161.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 526 kB/s | 43 kB 00:00 2024-03-07T06:48:21.185 INFO:teuthology.orchestra.run.smithi161.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 652 kB/s | 32 kB 00:00 2024-03-07T06:48:21.210 INFO:teuthology.orchestra.run.smithi161.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 303 kB/s | 15 kB 00:00 2024-03-07T06:48:21.222 INFO:teuthology.orchestra.run.smithi133.stdout:(73/174): openblas-openmp-0.3.21-2.el9.x86_64.r 6.0 MB/s | 4.7 MB 00:00 2024-03-07T06:48:21.261 INFO:teuthology.orchestra.run.smithi161.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.3 MB/s | 172 kB 00:00 2024-03-07T06:48:21.272 INFO:teuthology.orchestra.run.smithi133.stdout:(74/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-03-07T06:48:21.344 INFO:teuthology.orchestra.run.smithi161.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-03-07T06:48:21.381 INFO:teuthology.orchestra.run.smithi133.stdout:(75/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.3 MB/s | 1.0 MB 00:00 2024-03-07T06:48:21.395 INFO:teuthology.orchestra.run.smithi161.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-03-07T06:48:21.454 INFO:teuthology.orchestra.run.smithi161.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 327 kB/s | 35 kB 00:00 2024-03-07T06:48:21.495 INFO:teuthology.orchestra.run.smithi133.stdout:(76/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.1 MB/s | 245 kB 00:00 2024-03-07T06:48:21.512 INFO:teuthology.orchestra.run.smithi161.stdout:(123/174): python3-google-auth-2.28.0-1.el9.noa 1.7 MB/s | 203 kB 00:00 2024-03-07T06:48:21.582 INFO:teuthology.orchestra.run.smithi133.stdout:(77/174): python3-jinja2-2.11.3-5.el9.noarch.rp 2.8 MB/s | 248 kB 00:00 2024-03-07T06:48:21.587 INFO:teuthology.orchestra.run.smithi161.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-03-07T06:48:21.623 INFO:teuthology.orchestra.run.smithi133.stdout:(78/174): python3-jsonpatch-1.21-16.el9.noarch. 627 kB/s | 26 kB 00:00 2024-03-07T06:48:21.629 INFO:teuthology.orchestra.run.smithi161.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 263 kB/s | 11 kB 00:00 2024-03-07T06:48:21.665 INFO:teuthology.orchestra.run.smithi133.stdout:(79/174): python3-jsonpointer-2.0-4.el9.noarch. 469 kB/s | 19 kB 00:00 2024-03-07T06:48:21.671 INFO:teuthology.orchestra.run.smithi161.stdout:(126/174): python3-jaraco-classes-3.2.1-5.el9.n 435 kB/s | 18 kB 00:00 2024-03-07T06:48:21.713 INFO:teuthology.orchestra.run.smithi161.stdout:(127/174): python3-jaraco-collections-3.0.0-8.e 569 kB/s | 23 kB 00:00 2024-03-07T06:48:21.732 INFO:teuthology.orchestra.run.smithi133.stdout:(80/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.6 MB/s | 172 kB 00:00 2024-03-07T06:48:21.755 INFO:teuthology.orchestra.run.smithi161.stdout:(128/174): python3-jaraco-functools-3.5.0-2.el9 476 kB/s | 19 kB 00:00 2024-03-07T06:48:21.774 INFO:teuthology.orchestra.run.smithi133.stdout:(81/174): python3-markupsafe-1.1.1-12.el9.x86_6 850 kB/s | 35 kB 00:00 2024-03-07T06:48:21.797 INFO:teuthology.orchestra.run.smithi161.stdout:(129/174): python3-jaraco-text-3.2.0-6.el9.noar 483 kB/s | 20 kB 00:00 2024-03-07T06:48:22.064 INFO:teuthology.orchestra.run.smithi161.stdout:(130/174): python3-grpcio-1.46.7-10.el9.x86_64. 3.3 MB/s | 2.0 MB 00:00 2024-03-07T06:48:22.106 INFO:teuthology.orchestra.run.smithi161.stdout:(131/174): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-03-07T06:48:22.148 INFO:teuthology.orchestra.run.smithi161.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 2.9 MB/s | 1.0 MB 00:00 2024-03-07T06:48:22.158 INFO:teuthology.orchestra.run.smithi133.stdout:(82/174): python3-babel-2.9.1-2.el9.noarch.rpm 6.7 MB/s | 6.0 MB 00:00 2024-03-07T06:48:22.216 INFO:teuthology.orchestra.run.smithi133.stdout:(83/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-03-07T06:48:22.240 INFO:teuthology.orchestra.run.smithi161.stdout:(133/174): libarrow-9.0.0-10.el9.x86_64.rpm 2.8 MB/s | 4.4 MB 00:01 2024-03-07T06:48:22.265 INFO:teuthology.orchestra.run.smithi161.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 499 kB/s | 79 kB 00:00 2024-03-07T06:48:22.266 INFO:teuthology.orchestra.run.smithi133.stdout:(84/174): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-03-07T06:48:22.282 INFO:teuthology.orchestra.run.smithi161.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 432 kB/s | 58 kB 00:00 2024-03-07T06:48:22.299 INFO:teuthology.orchestra.run.smithi161.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 503 kB/s | 16 kB 00:00 2024-03-07T06:48:22.309 INFO:teuthology.orchestra.run.smithi133.stdout:(85/174): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-03-07T06:48:22.333 INFO:teuthology.orchestra.run.smithi161.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-03-07T06:48:22.350 INFO:teuthology.orchestra.run.smithi133.stdout:(86/174): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-03-07T06:48:22.358 INFO:teuthology.orchestra.run.smithi161.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 490 kB/s | 28 kB 00:00 2024-03-07T06:48:22.383 INFO:teuthology.orchestra.run.smithi161.stdout:(139/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-03-07T06:48:22.408 INFO:teuthology.orchestra.run.smithi161.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 412 kB/s | 31 kB 00:00 2024-03-07T06:48:22.409 INFO:teuthology.orchestra.run.smithi133.stdout:(87/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-03-07T06:48:22.433 INFO:teuthology.orchestra.run.smithi161.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-07T06:48:22.459 INFO:teuthology.orchestra.run.smithi161.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 724 kB/s | 36 kB 00:00 2024-03-07T06:48:22.460 INFO:teuthology.orchestra.run.smithi133.stdout:(88/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-03-07T06:48:22.492 INFO:teuthology.orchestra.run.smithi161.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-03-07T06:48:22.517 INFO:teuthology.orchestra.run.smithi161.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 618 kB/s | 51 kB 00:00 2024-03-07T06:48:22.526 INFO:teuthology.orchestra.run.smithi133.stdout:(89/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-03-07T06:48:22.559 INFO:teuthology.orchestra.run.smithi161.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-03-07T06:48:22.568 INFO:teuthology.orchestra.run.smithi133.stdout:(90/174): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-03-07T06:48:22.584 INFO:teuthology.orchestra.run.smithi161.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-03-07T06:48:22.626 INFO:teuthology.orchestra.run.smithi161.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 301 kB/s | 20 kB 00:00 2024-03-07T06:48:22.631 INFO:teuthology.orchestra.run.smithi133.stdout:(91/174): python3-requests-oauthlib-1.3.0-12.el 879 kB/s | 54 kB 00:00 2024-03-07T06:48:22.693 INFO:teuthology.orchestra.run.smithi161.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-03-07T06:48:22.727 INFO:teuthology.orchestra.run.smithi161.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-03-07T06:48:22.877 INFO:teuthology.orchestra.run.smithi133.stdout:(92/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.6 MB/s | 5.1 MB 00:01 2024-03-07T06:48:22.936 INFO:teuthology.orchestra.run.smithi133.stdout:(93/174): python3-toml-0.10.2-6.el9.noarch.rpm 728 kB/s | 42 kB 00:00 2024-03-07T06:48:22.995 INFO:teuthology.orchestra.run.smithi133.stdout:(94/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 767 kB/s | 44 kB 00:00 2024-03-07T06:48:23.078 INFO:teuthology.orchestra.run.smithi133.stdout:(95/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.6 MB/s | 216 kB 00:00 2024-03-07T06:48:23.085 INFO:teuthology.orchestra.run.smithi161.stdout:(150/174): librados2-19.0.0-1848.g8b896a9e.el9. 8.7 MB/s | 3.4 MB 00:00 2024-03-07T06:48:23.137 INFO:teuthology.orchestra.run.smithi133.stdout:(96/174): qatlib-service-23.11.0-1.el9.x86_64.r 606 kB/s | 35 kB 00:00 2024-03-07T06:48:23.186 INFO:teuthology.orchestra.run.smithi161.stdout:(151/174): librbd1-19.0.0-1848.g8b896a9e.el9.x8 6.9 MB/s | 3.1 MB 00:00 2024-03-07T06:48:23.195 INFO:teuthology.orchestra.run.smithi133.stdout:(97/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 774 kB/s | 44 kB 00:00 2024-03-07T06:48:23.236 INFO:teuthology.orchestra.run.smithi161.stdout:(152/174): thrift-0.15.0-2.el9.x86_64.rpm 2.6 MB/s | 1.6 MB 00:00 2024-03-07T06:48:23.279 INFO:teuthology.orchestra.run.smithi133.stdout:(98/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-03-07T06:48:23.345 INFO:teuthology.orchestra.run.smithi161.stdout:(153/174): NetworkManager-team-1.46.0-1.el9.x86 393 kB/s | 42 kB 00:00 2024-03-07T06:48:23.388 INFO:teuthology.orchestra.run.smithi133.stdout:(99/174): postgresql-server-13.11-1.el9.x86_64. 2.4 MB/s | 5.7 MB 00:02 2024-03-07T06:48:23.430 INFO:teuthology.orchestra.run.smithi133.stdout:(100/174): uuid-1.6.2-55.el9.x86_64.rpm 387 kB/s | 58 kB 00:00 2024-03-07T06:48:23.453 INFO:teuthology.orchestra.run.smithi161.stdout:(154/174): NetworkManager-1.46.0-1.el9.x86_64.r 6.3 MB/s | 2.3 MB 00:00 2024-03-07T06:48:23.472 INFO:teuthology.orchestra.run.smithi133.stdout:(101/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.1 MB/s | 94 kB 00:00 2024-03-07T06:48:23.503 INFO:teuthology.orchestra.run.smithi161.stdout:(155/174): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-07T06:48:23.514 INFO:teuthology.orchestra.run.smithi133.stdout:(102/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 770 kB/s | 64 kB 00:00 2024-03-07T06:48:23.528 INFO:teuthology.orchestra.run.smithi161.stdout:(156/174): NetworkManager-tui-1.46.0-1.el9.x86_ 1.3 MB/s | 248 kB 00:00 2024-03-07T06:48:23.554 INFO:teuthology.orchestra.run.smithi161.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 739 kB/s | 36 kB 00:00 2024-03-07T06:48:23.564 INFO:teuthology.orchestra.run.smithi133.stdout:(103/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 494 kB/s | 45 kB 00:00 2024-03-07T06:48:23.587 INFO:teuthology.orchestra.run.smithi161.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-03-07T06:48:23.621 INFO:teuthology.orchestra.run.smithi161.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.8 MB/s | 256 kB 00:00 2024-03-07T06:48:23.646 INFO:teuthology.orchestra.run.smithi161.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 397 kB/s | 23 kB 00:00 2024-03-07T06:48:23.664 INFO:teuthology.orchestra.run.smithi161.stdout:(161/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.5 MB/s | 108 kB 00:00 2024-03-07T06:48:23.722 INFO:teuthology.orchestra.run.smithi161.stdout:(162/174): NetworkManager-libnm-1.46.0-1.el9.x8 3.4 MB/s | 1.8 MB 00:00 2024-03-07T06:48:23.747 INFO:teuthology.orchestra.run.smithi161.stdout:(163/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-03-07T06:48:23.794 INFO:teuthology.orchestra.run.smithi161.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.0 MB/s | 155 kB 00:00 2024-03-07T06:48:23.823 INFO:teuthology.orchestra.run.smithi161.stdout:(165/174): libuuid-2.37.4-18.el9.x86_64.rpm 385 kB/s | 29 kB 00:00 2024-03-07T06:48:23.848 INFO:teuthology.orchestra.run.smithi161.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 510 kB/s | 64 kB 00:00 2024-03-07T06:48:23.907 INFO:teuthology.orchestra.run.smithi161.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2024-03-07T06:48:23.974 INFO:teuthology.orchestra.run.smithi161.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2024-03-07T06:48:24.016 INFO:teuthology.orchestra.run.smithi161.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 4.2 MB/s | 465 kB 00:00 2024-03-07T06:48:24.041 INFO:teuthology.orchestra.run.smithi161.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 471 kB/s | 31 kB 00:00 2024-03-07T06:48:24.074 INFO:teuthology.orchestra.run.smithi161.stdout:(171/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.5 MB/s | 323 kB 00:00 2024-03-07T06:48:24.133 INFO:teuthology.orchestra.run.smithi161.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-03-07T06:48:24.242 INFO:teuthology.orchestra.run.smithi161.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 447 kB/s | 74 kB 00:00 2024-03-07T06:48:24.359 INFO:teuthology.orchestra.run.smithi161.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 4.5 MB/s | 2.3 MB 00:00 2024-03-07T06:48:24.371 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2024-03-07T06:48:24.372 INFO:teuthology.orchestra.run.smithi161.stdout:Total 13 MB/s | 226 MB 00:16 2024-03-07T06:48:24.565 INFO:teuthology.orchestra.run.smithi133.stdout:(104/174): lua-devel-5.4.4-4.el9.x86_64.rpm 21 kB/s | 22 kB 00:01 2024-03-07T06:48:24.665 INFO:teuthology.orchestra.run.smithi133.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 785 kB/s | 863 kB 00:01 2024-03-07T06:48:25.057 INFO:teuthology.orchestra.run.smithi133.stdout:(106/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.7 MB/s | 16 MB 00:02 2024-03-07T06:48:25.332 INFO:teuthology.orchestra.run.smithi133.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 71 kB/s | 19 kB 00:00 2024-03-07T06:48:25.490 INFO:teuthology.orchestra.run.smithi133.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 374 kB/s | 309 kB 00:00 2024-03-07T06:48:25.636 INFO:teuthology.orchestra.run.smithi133.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 180 kB/s | 26 kB 00:00 2024-03-07T06:48:25.811 INFO:teuthology.orchestra.run.smithi133.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 279 kB/s | 49 kB 00:00 2024-03-07T06:48:25.928 INFO:teuthology.orchestra.run.smithi133.stdout:(111/174): abseil-cpp-20211102.0-3.el9.x86_64.r 403 kB/s | 548 kB 00:01 2024-03-07T06:48:25.986 INFO:teuthology.orchestra.run.smithi133.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 386 kB/s | 67 kB 00:00 2024-03-07T06:48:26.087 INFO:teuthology.orchestra.run.smithi133.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-03-07T06:48:26.187 INFO:teuthology.orchestra.run.smithi133.stdout:(114/174): luarocks-3.9.2-1.el9.noarch.rpm 585 kB/s | 151 kB 00:00 2024-03-07T06:48:26.221 INFO:teuthology.orchestra.run.smithi133.stdout:(115/174): python3-asyncssh-2.13.2-4.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-03-07T06:48:26.339 INFO:teuthology.orchestra.run.smithi133.stdout:(116/174): python3-cachetools-4.2.4-1.el9.noarc 277 kB/s | 32 kB 00:00 2024-03-07T06:48:26.363 INFO:teuthology.orchestra.run.smithi133.stdout:(117/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 249 kB/s | 43 kB 00:00 2024-03-07T06:48:26.438 INFO:teuthology.orchestra.run.smithi133.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 151 kB/s | 15 kB 00:00 2024-03-07T06:48:26.472 INFO:teuthology.orchestra.run.smithi133.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-03-07T06:48:26.572 INFO:teuthology.orchestra.run.smithi133.stdout:(120/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-03-07T06:48:26.656 INFO:teuthology.orchestra.run.smithi133.stdout:(121/174): python3-colorama-0.4.4-12.el9.noarch 426 kB/s | 35 kB 00:00 2024-03-07T06:48:26.756 INFO:teuthology.orchestra.run.smithi133.stdout:(122/174): python3-google-auth-2.28.0-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-03-07T06:48:26.856 INFO:teuthology.orchestra.run.smithi133.stdout:(123/174): python3-cherrypy-18.6.1-2.el9.noarch 860 kB/s | 358 kB 00:00 2024-03-07T06:48:26.940 INFO:teuthology.orchestra.run.smithi133.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-03-07T06:48:27.007 INFO:teuthology.orchestra.run.smithi133.stdout:(125/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 822 kB/s | 838 kB 00:01 2024-03-07T06:48:27.032 INFO:teuthology.orchestra.run.smithi133.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-03-07T06:48:27.066 INFO:teuthology.orchestra.run.smithi133.stdout:(127/174): python3-grpcio-tools-1.46.7-10.el9.x 693 kB/s | 144 kB 00:00 2024-03-07T06:48:27.116 INFO:teuthology.orchestra.run.smithi133.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-03-07T06:48:27.133 INFO:teuthology.orchestra.run.smithi133.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-03-07T06:48:27.175 INFO:teuthology.orchestra.run.smithi133.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-03-07T06:48:27.225 INFO:teuthology.orchestra.run.smithi133.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-03-07T06:48:27.284 INFO:teuthology.orchestra.run.smithi133.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-03-07T06:48:27.309 INFO:teuthology.orchestra.run.smithi133.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 347 kB/s | 46 kB 00:00 2024-03-07T06:48:27.376 INFO:teuthology.orchestra.run.smithi133.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 526 kB/s | 79 kB 00:00 2024-03-07T06:48:27.402 INFO:teuthology.orchestra.run.smithi133.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 493 kB/s | 58 kB 00:00 2024-03-07T06:48:27.485 INFO:teuthology.orchestra.run.smithi133.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 152 kB/s | 16 kB 00:00 2024-03-07T06:48:27.511 INFO:teuthology.orchestra.run.smithi133.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 833 kB/s | 90 kB 00:00 2024-03-07T06:48:27.602 INFO:teuthology.orchestra.run.smithi133.stdout:(138/174): python3-pecan-1.4.2-3.el9.noarch.rpm 932 kB/s | 272 kB 00:00 2024-03-07T06:48:27.628 INFO:teuthology.orchestra.run.smithi133.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 199 kB/s | 28 kB 00:00 2024-03-07T06:48:27.653 INFO:teuthology.orchestra.run.smithi133.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 217 kB/s | 31 kB 00:00 2024-03-07T06:48:27.762 INFO:teuthology.orchestra.run.smithi133.stdout:(141/174): python3-tempora-5.0.0-2.el9.noarch.r 332 kB/s | 36 kB 00:00 2024-03-07T06:48:27.787 INFO:teuthology.orchestra.run.smithi133.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 374 kB/s | 59 kB 00:00 2024-03-07T06:48:27.820 INFO:teuthology.orchestra.run.smithi133.stdout:(143/174): python3-routes-2.5.1-5.el9.noarch.rp 869 kB/s | 188 kB 00:00 2024-03-07T06:48:27.854 INFO:teuthology.orchestra.run.smithi133.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 558 kB/s | 51 kB 00:00 2024-03-07T06:48:27.971 INFO:teuthology.orchestra.run.smithi133.stdout:(145/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-03-07T06:48:27.996 INFO:teuthology.orchestra.run.smithi133.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 512 kB/s | 90 kB 00:00 2024-03-07T06:48:28.038 INFO:teuthology.orchestra.run.smithi133.stdout:(147/174): python3-webob-1.8.7-6.el9.noarch.rpm 917 kB/s | 230 kB 00:00 2024-03-07T06:48:28.099 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2024-03-07T06:48:28.214 INFO:teuthology.orchestra.run.smithi133.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 83 kB/s | 20 kB 00:00 2024-03-07T06:48:28.247 INFO:teuthology.orchestra.run.smithi133.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 767 kB/s | 192 kB 00:00 2024-03-07T06:48:28.485 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2024-03-07T06:48:28.485 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2024-03-07T06:48:28.764 INFO:teuthology.orchestra.run.smithi133.stdout:(150/174): librbd1-19.0.0-1848.g8b896a9e.el9.x8 6.1 MB/s | 3.1 MB 00:00 2024-03-07T06:48:28.856 INFO:teuthology.orchestra.run.smithi133.stdout:(151/174): librados2-19.0.0-1848.g8b896a9e.el9. 5.3 MB/s | 3.4 MB 00:00 2024-03-07T06:48:29.239 INFO:teuthology.orchestra.run.smithi133.stdout:(152/174): thrift-0.15.0-2.el9.x86_64.rpm 1.3 MB/s | 1.6 MB 00:01 2024-03-07T06:48:29.990 INFO:teuthology.orchestra.run.smithi133.stdout:(153/174): NetworkManager-team-1.46.0-1.el9.x86 63 kB/s | 42 kB 00:00 2024-03-07T06:48:30.923 INFO:teuthology.orchestra.run.smithi133.stdout:(154/174): NetworkManager-tui-1.46.0-1.el9.x86_ 266 kB/s | 248 kB 00:00 2024-03-07T06:48:30.981 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2024-03-07T06:48:30.982 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2024-03-07T06:48:31.656 INFO:teuthology.orchestra.run.smithi133.stdout:(155/174): acl-2.3.1-4.el9.x86_64.rpm 97 kB/s | 71 kB 00:00 2024-03-07T06:48:31.998 INFO:teuthology.orchestra.run.smithi133.stdout:(156/174): NetworkManager-libnm-1.46.0-1.el9.x8 592 kB/s | 1.8 MB 00:03 2024-03-07T06:48:32.315 INFO:teuthology.orchestra.run.smithi133.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 55 kB/s | 36 kB 00:00 2024-03-07T06:48:32.532 INFO:teuthology.orchestra.run.smithi133.stdout:(158/174): NetworkManager-1.46.0-1.el9.x86_64.r 622 kB/s | 2.3 MB 00:03 2024-03-07T06:48:32.950 INFO:teuthology.orchestra.run.smithi133.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 202 kB/s | 192 kB 00:00 2024-03-07T06:48:33.266 INFO:teuthology.orchestra.run.smithi133.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 31 kB/s | 23 kB 00:00 2024-03-07T06:48:33.383 INFO:teuthology.orchestra.run.smithi133.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 240 kB/s | 256 kB 00:01 2024-03-07T06:48:33.801 INFO:teuthology.orchestra.run.smithi133.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 128 kB/s | 108 kB 00:00 2024-03-07T06:48:34.393 INFO:teuthology.orchestra.run.smithi133.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 138 kB/s | 155 kB 00:01 2024-03-07T06:48:34.434 INFO:teuthology.orchestra.run.smithi133.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 129 kB/s | 136 kB 00:01 2024-03-07T06:48:34.710 INFO:teuthology.orchestra.run.smithi133.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 71 kB/s | 64 kB 00:00 2024-03-07T06:48:35.102 INFO:teuthology.orchestra.run.smithi133.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 41 kB/s | 29 kB 00:00 2024-03-07T06:48:36.786 INFO:teuthology.orchestra.run.smithi133.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 527 kB/s | 1.2 MB 00:02 2024-03-07T06:48:38.202 INFO:teuthology.orchestra.run.smithi133.stdout:(168/174): util-linux-core-2.37.4-18.el9.x86_64 329 kB/s | 465 kB 00:01 2024-03-07T06:48:38.644 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2024-03-07T06:48:38.711 INFO:teuthology.orchestra.run.smithi133.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 552 kB/s | 2.2 MB 00:03 2024-03-07T06:48:38.795 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-07T06:48:38.928 INFO:teuthology.orchestra.run.smithi133.stdout:(170/174): util-linux-user-2.37.4-18.el9.x86_64 43 kB/s | 31 kB 00:00 2024-03-07T06:48:39.032 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-07T06:48:39.135 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-07T06:48:39.182 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-07T06:48:39.273 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-07T06:48:39.278 INFO:teuthology.orchestra.run.smithi133.stdout:(171/174): util-linux-2.37.4-18.el9.x86_64.rpm 560 kB/s | 2.3 MB 00:04 2024-03-07T06:48:39.381 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-07T06:48:39.387 INFO:teuthology.orchestra.run.smithi133.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 693 kB/s | 74 kB 00:00 2024-03-07T06:48:39.438 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-07T06:48:39.477 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-07T06:48:39.564 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-07T06:48:39.862 INFO:teuthology.orchestra.run.smithi133.stdout:(173/174): xfsdump-3.1.12-4.el9.x86_64.rpm 281 kB/s | 323 kB 00:01 2024-03-07T06:48:39.964 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-07T06:48:40.000 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T06:48:40.153 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T06:48:40.176 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T06:48:40.285 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T06:48:40.326 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-07T06:48:40.389 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-07T06:48:40.730 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-07T06:48:40.789 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-07T06:48:40.815 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T06:48:40.989 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T06:48:41.016 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-07T06:48:41.037 INFO:teuthology.orchestra.run.smithi133.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 521 kB/s | 1.1 MB 00:02 2024-03-07T06:48:41.048 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-03-07T06:48:41.048 INFO:teuthology.orchestra.run.smithi133.stdout:Total 6.6 MB/s | 226 MB 00:34 2024-03-07T06:48:41.114 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-07T06:48:41.184 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-07T06:48:41.209 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-07T06:48:41.268 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-07T06:48:41.579 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-07T06:48:41.626 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-07T06:48:41.717 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-07T06:48:42.078 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-07T06:48:42.292 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-07T06:48:42.406 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-07T06:48:42.626 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T06:48:42.712 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T06:48:43.088 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T06:48:43.262 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-07T06:48:43.412 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-07T06:48:43.451 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-07T06:48:43.499 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-07T06:48:43.592 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-03-07T06:48:43.637 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-03-07T06:48:43.724 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-03-07T06:48:43.768 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-03-07T06:48:43.799 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-03-07T06:48:43.883 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-03-07T06:48:43.943 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-03-07T06:48:43.975 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-03-07T06:48:44.026 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-03-07T06:48:44.146 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-03-07T06:48:44.188 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 39/198 2024-03-07T06:48:44.238 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 40/198 2024-03-07T06:48:44.270 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 41/198 2024-03-07T06:48:44.386 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-07T06:48:44.439 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-07T06:48:44.561 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-07T06:48:44.733 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-07T06:48:44.795 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-03-07T06:48:44.800 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-07T06:48:45.186 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-03-07T06:48:45.186 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-03-07T06:48:45.482 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-07T06:48:45.588 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-07T06:48:45.736 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-07T06:48:45.824 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-07T06:48:45.852 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-07T06:48:45.909 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-07T06:48:45.991 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-07T06:48:46.078 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 53/198 2024-03-07T06:48:46.134 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-07T06:48:46.557 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-07T06:48:46.820 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-07T06:48:46.920 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-07T06:48:46.980 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-07T06:48:47.184 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-07T06:48:47.258 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-07T06:48:47.335 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-07T06:48:47.427 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-07T06:48:47.465 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-07T06:48:47.504 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-07T06:48:47.570 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-07T06:48:47.637 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-07T06:48:47.702 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-07T06:48:47.770 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-07T06:48:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-03-07T06:48:47.931 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-03-07T06:48:47.976 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-07T06:48:48.060 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-07T06:48:48.098 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-07T06:48:48.150 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-07T06:48:48.299 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-07T06:48:48.338 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-07T06:48:48.384 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-07T06:48:48.421 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-07T06:48:48.471 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-07T06:48:48.634 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-07T06:48:48.802 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-07T06:48:48.877 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-07T06:48:48.915 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-07T06:48:48.952 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-03-07T06:48:49.011 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-07T06:48:49.066 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-03-07T06:48:49.114 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-03-07T06:48:49.159 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-03-07T06:48:49.209 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-03-07T06:48:49.313 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-03-07T06:48:49.359 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-03-07T06:48:49.504 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-03-07T06:48:49.563 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-03-07T06:48:49.606 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-03-07T06:48:49.687 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-03-07T06:48:49.833 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-03-07T06:48:49.894 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-03-07T06:48:50.791 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-03-07T06:48:50.860 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-03-07T06:48:50.893 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-03-07T06:48:51.021 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-03-07T06:48:51.080 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-03-07T06:48:51.175 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-03-07T06:48:51.229 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-03-07T06:48:51.454 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-03-07T06:48:51.492 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-03-07T06:48:51.583 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-03-07T06:48:52.383 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-03-07T06:48:52.520 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-03-07T06:48:54.041 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-03-07T06:48:54.140 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-03-07T06:48:54.174 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-03-07T06:48:54.529 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-03-07T06:48:54.560 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 112/198 2024-03-07T06:48:54.679 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 112/198 2024-03-07T06:48:54.707 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 113/198 2024-03-07T06:48:54.785 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-03-07T06:48:55.169 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-03-07T06:48:55.290 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-07T06:48:55.317 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-07T06:48:55.321 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 116/198 2024-03-07T06:48:55.384 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-07T06:48:55.412 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 116/198 2024-03-07T06:48:55.430 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-07T06:48:55.440 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 117/198 2024-03-07T06:48:55.475 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-07T06:48:55.539 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-07T06:48:55.622 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-07T06:48:55.667 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-07T06:48:55.711 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-07T06:48:55.793 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-07T06:48:56.215 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-07T06:48:56.252 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T06:48:56.405 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T06:48:56.436 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T06:48:56.556 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T06:48:56.600 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-07T06:48:56.675 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-07T06:48:56.973 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-07T06:48:57.049 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-07T06:48:57.075 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T06:48:57.239 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T06:48:57.268 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-07T06:48:57.366 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 17/198 2024-03-07T06:48:57.402 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-07T06:48:57.428 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-07T06:48:57.492 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-07T06:48:57.788 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-07T06:48:57.836 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-07T06:48:57.878 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-07T06:48:58.067 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T06:48:58.095 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T06:48:58.172 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-07T06:48:58.326 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-07T06:48:58.465 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 23/198 2024-03-07T06:48:58.696 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T06:48:58.718 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T06:48:58.737 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T06:48:58.764 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 119/198 2024-03-07T06:48:58.946 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 119/198 2024-03-07T06:48:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-07T06:48:58.946 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-07T06:48:58.946 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:48:58.999 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 120/198 2024-03-07T06:48:59.071 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.46.0-1.el9.x86_64 24/198 2024-03-07T06:48:59.237 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-07T06:48:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-07T06:48:59.402 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-07T06:48:59.449 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-07T06:48:59.533 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 29/198 2024-03-07T06:48:59.572 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 30/198 2024-03-07T06:48:59.629 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 31/198 2024-03-07T06:48:59.667 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 32/198 2024-03-07T06:48:59.708 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-5.4.4-4.el9.x86_64 33/198 2024-03-07T06:48:59.822 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 34/198 2024-03-07T06:48:59.898 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 35/198 2024-03-07T06:48:59.936 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 36/198 2024-03-07T06:48:59.995 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 37/198 2024-03-07T06:49:00.101 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 38/198 2024-03-07T06:49:00.139 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 39/198 2024-03-07T06:49:00.184 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 40/198 2024-03-07T06:49:00.220 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 41/198 2024-03-07T06:49:00.328 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-07T06:49:00.388 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-07T06:49:00.487 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-07T06:49:00.660 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-07T06:49:00.728 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-07T06:49:01.444 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-07T06:49:01.539 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-07T06:49:01.661 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-07T06:49:01.718 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-07T06:49:01.743 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-07T06:49:01.801 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-07T06:49:01.871 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-07T06:49:01.929 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 53/198 2024-03-07T06:49:01.976 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-07T06:49:02.360 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-07T06:49:02.601 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-07T06:49:02.682 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-07T06:49:02.752 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-07T06:49:02.957 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-07T06:49:03.027 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-07T06:49:03.115 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-07T06:49:03.217 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-07T06:49:03.327 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-07T06:49:03.368 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-07T06:49:03.449 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-07T06:49:03.493 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-07T06:49:03.556 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-07T06:49:03.618 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-07T06:49:03.810 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-07T06:49:03.863 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-07T06:49:03.901 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-07T06:49:03.949 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-07T06:49:04.076 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-07T06:49:04.117 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:2.28.0-1.el9.noarch 74/198 2024-03-07T06:49:04.161 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-07T06:49:04.191 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-07T06:49:04.223 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-07T06:49:04.378 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-07T06:49:04.556 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-07T06:49:04.596 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-07T06:49:04.633 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-07T06:49:04.670 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 82/198 2024-03-07T06:49:04.724 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-07T06:49:04.773 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 84/198 2024-03-07T06:49:04.815 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 85/198 2024-03-07T06:49:04.861 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 86/198 2024-03-07T06:49:04.902 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 87/198 2024-03-07T06:49:04.968 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 88/198 2024-03-07T06:49:05.011 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 89/198 2024-03-07T06:49:05.154 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 90/198 2024-03-07T06:49:05.213 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 91/198 2024-03-07T06:49:05.249 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 92/198 2024-03-07T06:49:05.309 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 93/198 2024-03-07T06:49:05.455 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-packaging-20.9-5.el9.noarch 94/198 2024-03-07T06:49:05.522 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 95/198 2024-03-07T06:49:06.489 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 96/198 2024-03-07T06:49:06.546 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 97/198 2024-03-07T06:49:06.580 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 98/198 2024-03-07T06:49:06.707 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 99/198 2024-03-07T06:49:06.765 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 100/198 2024-03-07T06:49:06.860 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 101/198 2024-03-07T06:49:06.898 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 102/198 2024-03-07T06:49:07.121 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.21-2.el9.x86_64 103/198 2024-03-07T06:49:07.153 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 104/198 2024-03-07T06:49:07.244 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 105/198 2024-03-07T06:49:08.061 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 106/198 2024-03-07T06:49:08.198 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 107/198 2024-03-07T06:49:09.715 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 108/198 2024-03-07T06:49:09.809 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 109/198 2024-03-07T06:49:09.843 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 110/198 2024-03-07T06:49:10.185 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 111/198 2024-03-07T06:49:10.212 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 112/198 2024-03-07T06:49:10.365 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 112/198 2024-03-07T06:49:10.393 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 113/198 2024-03-07T06:49:10.449 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 114/198 2024-03-07T06:49:10.949 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-07T06:49:10.982 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 116/198 2024-03-07T06:49:11.123 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 116/198 2024-03-07T06:49:11.150 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 117/198 2024-03-07T06:49:11.801 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 120/198 2024-03-07T06:49:11.802 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2024-03-07T06:49:11.802 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2024-03-07T06:49:11.802 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2024-03-07T06:49:11.802 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2024-03-07T06:49:11.802 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2024-03-07T06:49:11.802 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /root 2024-03-07T06:49:11.802 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2024-03-07T06:49:11.802 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:49:12.018 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 121/198 2024-03-07T06:49:12.068 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 121/198 2024-03-07T06:49:12.068 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:12.068 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-07T06:49:12.069 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:49:13.688 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 122/198 2024-03-07T06:49:13.768 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 122/198 2024-03-07T06:49:13.768 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:13.768 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-07T06:49:13.768 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-07T06:49:13.768 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-07T06:49:13.768 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:49:13.985 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T06:49:14.022 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 123/198 2024-03-07T06:49:14.030 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T06:49:14.089 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 123/198 2024-03-07T06:49:14.089 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:14.089 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-07T06:49:14.089 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-07T06:49:14.089 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-07T06:49:14.089 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:49:14.572 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-07T06:49:14.639 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-07T06:49:14.639 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:14.640 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-07T06:49:14.640 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-07T06:49:14.640 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-07T06:49:14.640 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:49:14.653 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 118/198 2024-03-07T06:49:14.701 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 119/198 2024-03-07T06:49:14.780 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-03-07T06:49:14.845 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-03-07T06:49:14.882 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 119/198 2024-03-07T06:49:14.883 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-07T06:49:14.883 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-07T06:49:14.883 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:49:14.927 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 120/198 2024-03-07T06:49:14.928 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-03-07T06:49:15.156 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-03-07T06:49:15.217 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-03-07T06:49:15.318 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-03-07T06:49:15.418 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-03-07T06:49:15.577 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-03-07T06:49:15.644 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 133/198 2024-03-07T06:49:15.691 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-03-07T06:49:15.724 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-03-07T06:49:15.763 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-03-07T06:49:15.792 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-03-07T06:49:16.395 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-07T06:49:16.462 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-07T06:49:16.518 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 139/198 2024-03-07T06:49:16.633 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 140/198 2024-03-07T06:49:16.663 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-07T06:49:16.993 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-07T06:49:17.031 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-07T06:49:18.118 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-07T06:49:18.148 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-07T06:49:18.289 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-07T06:49:18.468 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 144/198 2024-03-07T06:49:18.508 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-07T06:49:18.577 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-07T06:49:18.577 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:18.577 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-07T06:49:18.577 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-07T06:49:18.578 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-07T06:49:18.578 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:49:18.634 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-07T06:49:18.672 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-07T06:49:18.815 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 147/198 2024-03-07T06:49:23.575 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 148/198 2024-03-07T06:49:23.659 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 149/198 2024-03-07T06:49:24.580 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-03-07T06:49:24.617 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T06:49:24.712 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T06:49:24.712 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:24.712 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-07T06:49:24.713 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-07T06:49:24.713 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-07T06:49:24.713 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:49:24.768 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 152/198 2024-03-07T06:49:24.863 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1848.g8b896 152/198 2024-03-07T06:49:24.864 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:24.864 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-07T06:49:24.864 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:49:24.944 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T06:49:25.052 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T06:49:25.052 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:25.052 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-07T06:49:25.052 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-07T06:49:25.053 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-07T06:49:25.053 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:49:25.485 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-07T06:49:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-07T06:49:25.581 INFO:teuthology.orchestra.run.smithi161.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:25.581 INFO:teuthology.orchestra.run.smithi161.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-07T06:49:25.581 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-07T06:49:25.581 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-07T06:49:25.581 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:49:25.646 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 155/198 2024-03-07T06:49:25.695 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 156/198 2024-03-07T06:49:25.735 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-03-07T06:49:26.036 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 158/198 2024-03-07T06:49:26.107 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/198 2024-03-07T06:49:26.172 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-07T06:49:26.246 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-07T06:49:26.308 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-07T06:49:26.352 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 163/198 2024-03-07T06:49:26.396 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-07T06:49:26.446 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-07T06:49:26.530 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-07T06:49:26.640 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-07T06:49:26.712 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-07T06:49:27.486 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 120/198 2024-03-07T06:49:27.486 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2024-03-07T06:49:27.486 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2024-03-07T06:49:27.486 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2024-03-07T06:49:27.486 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2024-03-07T06:49:27.487 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2024-03-07T06:49:27.487 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /root 2024-03-07T06:49:27.487 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2024-03-07T06:49:27.487 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:49:27.626 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T06:49:27.675 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T06:49:27.711 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 121/198 2024-03-07T06:49:27.742 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T06:49:27.761 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 121/198 2024-03-07T06:49:27.761 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:27.761 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-07T06:49:27.761 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:49:29.170 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 170/198 2024-03-07T06:49:29.284 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-07T06:49:29.382 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 122/198 2024-03-07T06:49:29.393 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-03-07T06:49:29.461 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-03-07T06:49:29.475 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 122/198 2024-03-07T06:49:29.475 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:29.476 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-07T06:49:29.476 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-07T06:49:29.476 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-07T06:49:29.476 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:49:29.490 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-03-07T06:49:29.533 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-07T06:49:29.567 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-07T06:49:29.610 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-07T06:49:29.610 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-07T06:49:29.715 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 123/198 2024-03-07T06:49:29.754 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-07T06:49:29.786 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 123/198 2024-03-07T06:49:29.786 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:29.786 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-07T06:49:29.786 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-07T06:49:29.787 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-07T06:49:29.787 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:49:29.796 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-07T06:49:29.796 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-07T06:49:29.950 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-07T06:49:29.994 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-07T06:49:30.050 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-07T06:49:30.082 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-07T06:49:30.115 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-07T06:49:30.181 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-07T06:49:30.274 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-07T06:49:30.278 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-07T06:49:30.327 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-07T06:49:30.343 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 124/198 2024-03-07T06:49:30.343 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:30.343 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-07T06:49:30.343 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-07T06:49:30.343 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-07T06:49:30.343 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:49:30.360 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-07T06:49:30.395 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-07T06:49:30.428 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-07T06:49:30.461 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-07T06:49:30.474 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : automake-1.16.2-8.el9.noarch 125/198 2024-03-07T06:49:30.497 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T06:49:30.497 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T06:49:30.537 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ply-3.11-14.el9.noarch 126/198 2024-03-07T06:49:30.619 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 127/198 2024-03-07T06:49:30.828 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T06:49:30.828 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-07T06:49:30.853 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 128/198 2024-03-07T06:49:30.903 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 129/198 2024-03-07T06:49:30.959 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-07T06:49:30.989 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 130/198 2024-03-07T06:49:30.989 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-07T06:49:31.048 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-07T06:49:31.084 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-07T06:49:31.106 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 131/198 2024-03-07T06:49:31.146 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-07T06:49:31.267 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-07T06:49:31.274 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 132/198 2024-03-07T06:49:31.318 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 133/198 2024-03-07T06:49:31.359 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 134/198 2024-03-07T06:49:31.391 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 135/198 2024-03-07T06:49:31.438 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : jq-1.6-16.el9.x86_64 136/198 2024-03-07T06:49:31.478 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.9-15.el9.x86_64 137/198 2024-03-07T06:49:31.520 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-07T06:49:31.520 INFO:teuthology.orchestra.run.smithi161.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-07T06:49:31.520 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:49:32.225 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-07T06:49:32.273 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 138/198 2024-03-07T06:49:32.329 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 139/198 2024-03-07T06:49:32.407 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 140/198 2024-03-07T06:49:32.432 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-07T06:49:32.743 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 141/198 2024-03-07T06:49:32.783 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-07T06:49:33.888 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 142/198 2024-03-07T06:49:33.925 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-07T06:49:34.065 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 143/198 2024-03-07T06:49:34.239 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 144/198 2024-03-07T06:49:34.277 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-07T06:49:34.361 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 145/198 2024-03-07T06:49:34.361 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:34.361 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-07T06:49:34.361 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-07T06:49:34.361 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-07T06:49:34.361 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:49:34.419 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-07T06:49:34.457 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 146/198 2024-03-07T06:49:34.599 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 147/198 2024-03-07T06:49:36.559 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 1/198 2024-03-07T06:49:36.560 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2/198 2024-03-07T06:49:36.560 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 3/198 2024-03-07T06:49:36.560 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 4/198 2024-03-07T06:49:36.561 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 5/198 2024-03-07T06:49:36.561 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 6/198 2024-03-07T06:49:36.561 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 7/198 2024-03-07T06:49:36.561 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 8/198 2024-03-07T06:49:36.561 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 9/198 2024-03-07T06:49:36.561 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T06:49:36.561 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T06:49:36.561 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-07T06:49:36.561 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 13/198 2024-03-07T06:49:36.561 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 14/198 2024-03-07T06:49:36.561 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 15/198 2024-03-07T06:49:36.561 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T06:49:36.562 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 17/198 2024-03-07T06:49:36.562 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 18/198 2024-03-07T06:49:36.562 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 19/198 2024-03-07T06:49:36.562 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 20/198 2024-03-07T06:49:36.562 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 21/198 2024-03-07T06:49:36.562 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 22/198 2024-03-07T06:49:36.562 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 23/198 2024-03-07T06:49:36.562 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 24/198 2024-03-07T06:49:36.562 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 25/198 2024-03-07T06:49:36.562 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 26/198 2024-03-07T06:49:36.562 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 27/198 2024-03-07T06:49:36.562 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 28/198 2024-03-07T06:49:36.562 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 29/198 2024-03-07T06:49:36.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 30/198 2024-03-07T06:49:36.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 31/198 2024-03-07T06:49:36.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 32/198 2024-03-07T06:49:36.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 33/198 2024-03-07T06:49:36.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 34/198 2024-03-07T06:49:36.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 35/198 2024-03-07T06:49:36.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 36/198 2024-03-07T06:49:36.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 37/198 2024-03-07T06:49:36.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 38/198 2024-03-07T06:49:36.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 39/198 2024-03-07T06:49:36.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-03-07T06:49:36.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-03-07T06:49:36.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-03-07T06:49:36.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-03-07T06:49:36.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-03-07T06:49:36.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-03-07T06:49:36.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-03-07T06:49:36.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-03-07T06:49:36.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-03-07T06:49:36.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-03-07T06:49:36.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-03-07T06:49:36.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-03-07T06:49:36.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-03-07T06:49:36.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-03-07T06:49:36.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-03-07T06:49:36.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-03-07T06:49:36.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-03-07T06:49:36.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/198 2024-03-07T06:49:36.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/198 2024-03-07T06:49:36.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/198 2024-03-07T06:49:36.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/198 2024-03-07T06:49:36.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-03-07T06:49:36.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-03-07T06:49:36.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-03-07T06:49:36.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-03-07T06:49:36.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-03-07T06:49:36.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-03-07T06:49:36.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-03-07T06:49:36.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-03-07T06:49:36.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-03-07T06:49:36.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-03-07T06:49:36.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/198 2024-03-07T06:49:36.567 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-03-07T06:49:36.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-03-07T06:49:36.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-03-07T06:49:36.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-03-07T06:49:36.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-03-07T06:49:36.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-07T06:49:36.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-07T06:49:36.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-07T06:49:36.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-07T06:49:36.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-07T06:49:36.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-07T06:49:36.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-07T06:49:36.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-07T06:49:36.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-07T06:49:36.568 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-07T06:49:36.569 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-07T06:49:36.569 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-07T06:49:36.569 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-07T06:49:36.569 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-07T06:49:36.569 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-07T06:49:36.569 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-07T06:49:36.569 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-07T06:49:36.569 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-07T06:49:36.569 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-07T06:49:36.569 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-07T06:49:36.569 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-07T06:49:36.569 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-07T06:49:36.569 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-07T06:49:36.569 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-07T06:49:36.570 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-07T06:49:36.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-07T06:49:36.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-07T06:49:36.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-07T06:49:36.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-07T06:49:36.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-07T06:49:36.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-07T06:49:36.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-07T06:49:36.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T06:49:36.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-07T06:49:36.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T06:49:36.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-07T06:49:36.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 155/198 2024-03-07T06:49:36.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-07T06:49:36.571 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 157/198 2024-03-07T06:49:36.572 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-07T06:49:36.572 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 159/198 2024-03-07T06:49:36.572 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-07T06:49:36.572 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 161/198 2024-03-07T06:49:36.572 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-07T06:49:36.572 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-03-07T06:49:36.572 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-03-07T06:49:36.572 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-03-07T06:49:36.572 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-03-07T06:49:36.572 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-03-07T06:49:36.572 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-03-07T06:49:36.572 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-03-07T06:49:36.572 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-03-07T06:49:36.572 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-03-07T06:49:36.573 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-03-07T06:49:36.573 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-03-07T06:49:36.573 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-03-07T06:49:36.573 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-03-07T06:49:36.573 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-03-07T06:49:36.573 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-03-07T06:49:36.573 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-03-07T06:49:36.573 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-03-07T06:49:36.573 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-03-07T06:49:36.573 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-03-07T06:49:36.573 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-03-07T06:49:36.573 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-03-07T06:49:36.573 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-03-07T06:49:36.574 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-03-07T06:49:36.574 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-03-07T06:49:36.574 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-03-07T06:49:36.574 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-03-07T06:49:36.574 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-03-07T06:49:36.574 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-03-07T06:49:36.574 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-03-07T06:49:36.574 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-03-07T06:49:36.574 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-03-07T06:49:36.574 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-03-07T06:49:36.574 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-07T06:49:36.574 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-07T06:49:36.574 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-07T06:49:39.198 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-07T06:49:39.198 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:49:39.198 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2024-03-07T06:49:39.198 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-07T06:49:39.198 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-07T06:49:39.198 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-07T06:49:39.198 INFO:teuthology.orchestra.run.smithi161.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-07T06:49:39.198 INFO:teuthology.orchestra.run.smithi161.stdout: acl-2.3.1-4.el9.x86_64 2024-03-07T06:49:39.199 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-07T06:49:39.199 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-07T06:49:39.199 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-07T06:49:39.199 INFO:teuthology.orchestra.run.smithi161.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-07T06:49:39.199 INFO:teuthology.orchestra.run.smithi161.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-07T06:49:39.199 INFO:teuthology.orchestra.run.smithi161.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-07T06:49:39.199 INFO:teuthology.orchestra.run.smithi161.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-07T06:49:39.199 INFO:teuthology.orchestra.run.smithi161.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-07T06:49:39.199 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.199 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.199 INFO:teuthology.orchestra.run.smithi161.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-07T06:49:39.199 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-07T06:49:39.199 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-07T06:49:39.199 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-07T06:49:39.200 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-07T06:49:39.200 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-07T06:49:39.200 INFO:teuthology.orchestra.run.smithi161.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-07T06:49:39.200 INFO:teuthology.orchestra.run.smithi161.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-07T06:49:39.200 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-07T06:49:39.200 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2024-03-07T06:49:39.200 INFO:teuthology.orchestra.run.smithi161.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-07T06:49:39.200 INFO:teuthology.orchestra.run.smithi161.stdout: automake-1.16.2-8.el9.noarch 2024-03-07T06:49:39.200 INFO:teuthology.orchestra.run.smithi161.stdout: bison-3.7.4-5.el9.x86_64 2024-03-07T06:49:39.200 INFO:teuthology.orchestra.run.smithi161.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-07T06:49:39.200 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.200 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.200 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.201 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.201 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:39.201 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.201 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.201 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.201 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:39.201 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:39.201 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:39.201 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:39.201 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:39.201 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.201 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.201 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:39.201 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.202 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.202 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.202 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:39.202 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:39.202 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.202 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.202 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:39.202 INFO:teuthology.orchestra.run.smithi161.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-07T06:49:39.202 INFO:teuthology.orchestra.run.smithi161.stdout: flex-2.6.4-9.el9.x86_64 2024-03-07T06:49:39.202 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-07T06:49:39.202 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-07T06:49:39.202 INFO:teuthology.orchestra.run.smithi161.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-07T06:49:39.202 INFO:teuthology.orchestra.run.smithi161.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-07T06:49:39.202 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-07T06:49:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-07T06:49:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: jq-1.6-16.el9.x86_64 2024-03-07T06:49:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-07T06:49:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-07T06:49:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-07T06:49:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-07T06:49:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-07T06:49:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-07T06:49:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-07T06:49:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-07T06:49:39.203 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.204 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.204 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-07T06:49:39.204 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.204 INFO:teuthology.orchestra.run.smithi161.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-07T06:49:39.204 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-07T06:49:39.204 INFO:teuthology.orchestra.run.smithi161.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-07T06:49:39.204 INFO:teuthology.orchestra.run.smithi161.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-07T06:49:39.204 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-07T06:49:39.204 INFO:teuthology.orchestra.run.smithi161.stdout: lua-5.4.4-4.el9.x86_64 2024-03-07T06:49:39.204 INFO:teuthology.orchestra.run.smithi161.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-07T06:49:39.204 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-07T06:49:39.204 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-07T06:49:39.204 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-07T06:49:39.204 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-07T06:49:39.205 INFO:teuthology.orchestra.run.smithi161.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-07T06:49:39.205 INFO:teuthology.orchestra.run.smithi161.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-07T06:49:39.205 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-13.11-1.el9.x86_64 2024-03-07T06:49:39.205 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-07T06:49:39.205 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-07T06:49:39.205 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-07T06:49:39.205 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-07T06:49:39.205 INFO:teuthology.orchestra.run.smithi161.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-07T06:49:39.205 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-07T06:49:39.205 INFO:teuthology.orchestra.run.smithi161.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-07T06:49:39.205 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-07T06:49:39.205 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-07T06:49:39.205 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-07T06:49:39.206 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.206 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.206 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.206 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-07T06:49:39.206 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-07T06:49:39.206 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-07T06:49:39.206 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-07T06:49:39.206 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-07T06:49:39.206 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-07T06:49:39.206 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-07T06:49:39.206 INFO:teuthology.orchestra.run.smithi161.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-07T06:49:39.206 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-07T06:49:39.206 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-07T06:49:39.206 INFO:teuthology.orchestra.run.smithi161.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-07T06:49:39.207 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-07T06:49:39.207 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-07T06:49:39.207 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-07T06:49:39.207 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-07T06:49:39.207 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-07T06:49:39.207 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-07T06:49:39.207 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-07T06:49:39.207 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-07T06:49:39.207 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-07T06:49:39.207 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-07T06:49:39.207 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-07T06:49:39.207 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-07T06:49:39.207 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-07T06:49:39.208 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-07T06:49:39.208 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-07T06:49:39.208 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-07T06:49:39.208 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-07T06:49:39.208 INFO:teuthology.orchestra.run.smithi161.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-07T06:49:39.208 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-07T06:49:39.208 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ply-3.11-14.el9.noarch 2024-03-07T06:49:39.208 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-07T06:49:39.208 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-07T06:49:39.208 INFO:teuthology.orchestra.run.smithi161.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-07T06:49:39.208 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-07T06:49:39.208 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-07T06:49:39.208 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-07T06:49:39.209 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-07T06:49:39.209 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-07T06:49:39.209 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-07T06:49:39.209 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.209 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.209 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-07T06:49:39.209 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-07T06:49:39.209 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-07T06:49:39.209 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.209 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-07T06:49:39.209 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-07T06:49:39.209 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-07T06:49:39.209 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-07T06:49:39.209 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-07T06:49:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-07T06:49:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-07T06:49:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-07T06:49:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-07T06:49:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-07T06:49:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-07T06:49:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-07T06:49:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-07T06:49:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-07T06:49:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-07T06:49:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:39.210 INFO:teuthology.orchestra.run.smithi161.stdout: re2-1:20211101-3.el9.x86_64 2024-03-07T06:49:39.211 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-07T06:49:39.211 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-07T06:49:39.211 INFO:teuthology.orchestra.run.smithi161.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-07T06:49:39.211 INFO:teuthology.orchestra.run.smithi161.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-07T06:49:39.211 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-07T06:49:39.211 INFO:teuthology.orchestra.run.smithi161.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-07T06:49:39.211 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:49:39.211 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2024-03-07T06:49:39.306 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 148/198 2024-03-07T06:49:39.388 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 149/198 2024-03-07T06:49:39.875 DEBUG:teuthology.parallel:result is None 2024-03-07T06:49:40.324 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libtool-2.4.6-45.el9.x86_64 150/198 2024-03-07T06:49:40.359 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T06:49:40.437 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T06:49:40.437 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:40.438 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-07T06:49:40.438 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-07T06:49:40.438 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-07T06:49:40.438 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:49:40.496 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 152/198 2024-03-07T06:49:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1848.g8b896 152/198 2024-03-07T06:49:40.576 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:40.576 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-07T06:49:40.576 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:49:40.653 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T06:49:40.729 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T06:49:40.729 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:40.729 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-07T06:49:40.729 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-07T06:49:40.729 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-07T06:49:40.729 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:49:41.103 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-07T06:49:41.179 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 154/198 2024-03-07T06:49:41.179 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-07T06:49:41.179 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-07T06:49:41.179 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-07T06:49:41.179 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-07T06:49:41.179 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:49:41.243 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 155/198 2024-03-07T06:49:41.278 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 156/198 2024-03-07T06:49:41.317 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/198 2024-03-07T06:49:41.609 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 158/198 2024-03-07T06:49:41.705 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/198 2024-03-07T06:49:41.767 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-07T06:49:41.846 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-07T06:49:41.917 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-07T06:49:41.961 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 163/198 2024-03-07T06:49:42.021 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-07T06:49:42.073 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-team-1:1.46.0-1.el9.x86_64 165/198 2024-03-07T06:49:42.182 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-tui-1:1.46.0-1.el9.x86_64 166/198 2024-03-07T06:49:42.316 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-07T06:49:42.379 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-07T06:49:43.451 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T06:49:43.501 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T06:49:43.557 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-07T06:49:44.887 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 170/198 2024-03-07T06:49:45.031 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-07T06:49:45.165 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flex-2.6.4-9.el9.x86_64 172/198 2024-03-07T06:49:45.207 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : bison-3.7.4-5.el9.x86_64 173/198 2024-03-07T06:49:45.232 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 174/198 2024-03-07T06:49:45.266 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-07T06:49:45.301 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-07T06:49:45.344 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-07T06:49:45.344 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-07T06:49:45.487 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-07T06:49:45.530 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-07T06:49:45.530 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-07T06:49:45.668 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-07T06:49:45.720 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-07T06:49:45.774 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-07T06:49:45.808 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-07T06:49:45.840 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-07T06:49:45.906 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-07T06:49:45.979 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-07T06:49:46.037 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-07T06:49:46.072 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-07T06:49:46.114 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-07T06:49:46.172 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-07T06:49:46.206 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-07T06:49:46.242 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T06:49:46.242 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T06:49:46.572 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-07T06:49:46.573 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-07T06:49:46.712 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-07T06:49:46.742 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-07T06:49:46.775 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-07T06:49:46.812 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-07T06:49:46.882 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-07T06:49:47.002 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-07T06:49:47.248 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-07T06:49:47.248 INFO:teuthology.orchestra.run.smithi133.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-07T06:49:47.248 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:49:52.296 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 1/198 2024-03-07T06:49:52.296 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2/198 2024-03-07T06:49:52.296 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 3/198 2024-03-07T06:49:52.296 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 4/198 2024-03-07T06:49:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1848.g8b896 5/198 2024-03-07T06:49:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 6/198 2024-03-07T06:49:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 7/198 2024-03-07T06:49:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 8/198 2024-03-07T06:49:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 9/198 2024-03-07T06:49:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 10/198 2024-03-07T06:49:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 11/198 2024-03-07T06:49:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 12/198 2024-03-07T06:49:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 13/198 2024-03-07T06:49:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 14/198 2024-03-07T06:49:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_ 15/198 2024-03-07T06:49:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 16/198 2024-03-07T06:49:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 17/198 2024-03-07T06:49:52.297 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_6 18/198 2024-03-07T06:49:52.298 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86 19/198 2024-03-07T06:49:52.298 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 20/198 2024-03-07T06:49:52.298 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el 21/198 2024-03-07T06:49:52.298 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9. 22/198 2024-03-07T06:49:52.298 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_6 23/198 2024-03-07T06:49:52.298 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 24/198 2024-03-07T06:49:52.298 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 25/198 2024-03-07T06:49:52.298 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 26/198 2024-03-07T06:49:52.298 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 27/198 2024-03-07T06:49:52.298 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 28/198 2024-03-07T06:49:52.298 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 29/198 2024-03-07T06:49:52.314 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e. 30/198 2024-03-07T06:49:52.314 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noa 31/198 2024-03-07T06:49:52.314 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.n 32/198 2024-03-07T06:49:52.314 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b8 33/198 2024-03-07T06:49:52.314 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el 34/198 2024-03-07T06:49:52.314 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 35/198 2024-03-07T06:49:52.314 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.e 36/198 2024-03-07T06:49:52.314 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 37/198 2024-03-07T06:49:52.314 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 38/198 2024-03-07T06:49:52.314 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 39/198 2024-03-07T06:49:52.314 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 40/198 2024-03-07T06:49:52.314 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 41/198 2024-03-07T06:49:52.315 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : jq-1.6-16.el9.x86_64 42/198 2024-03-07T06:49:52.315 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 43/198 2024-03-07T06:49:52.315 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 44/198 2024-03-07T06:49:52.315 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 45/198 2024-03-07T06:49:52.315 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 46/198 2024-03-07T06:49:52.315 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 47/198 2024-03-07T06:49:52.315 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ply-3.11-14.el9.noarch 48/198 2024-03-07T06:49:52.315 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 49/198 2024-03-07T06:49:52.315 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 50/198 2024-03-07T06:49:52.315 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 51/198 2024-03-07T06:49:52.315 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : automake-1.16.2-8.el9.noarch 52/198 2024-03-07T06:49:52.315 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : bison-3.7.4-5.el9.x86_64 53/198 2024-03-07T06:49:52.315 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 54/198 2024-03-07T06:49:52.316 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 55/198 2024-03-07T06:49:52.316 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flex-2.6.4-9.el9.x86_64 56/198 2024-03-07T06:49:52.316 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 57/198 2024-03-07T06:49:52.316 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 58/198 2024-03-07T06:49:52.316 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 59/198 2024-03-07T06:49:52.316 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 60/198 2024-03-07T06:49:52.316 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 61/198 2024-03-07T06:49:52.316 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 62/198 2024-03-07T06:49:52.316 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 63/198 2024-03-07T06:49:52.316 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 64/198 2024-03-07T06:49:52.316 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 65/198 2024-03-07T06:49:52.316 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 66/198 2024-03-07T06:49:52.316 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-5.4.4-4.el9.x86_64 67/198 2024-03-07T06:49:52.317 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 68/198 2024-03-07T06:49:52.317 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 69/198 2024-03-07T06:49:52.317 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 70/198 2024-03-07T06:49:52.317 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-13.11-1.el9.x86_64 71/198 2024-03-07T06:49:52.317 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 72/198 2024-03-07T06:49:52.317 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 73/198 2024-03-07T06:49:52.317 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 74/198 2024-03-07T06:49:52.317 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 75/198 2024-03-07T06:49:52.317 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 76/198 2024-03-07T06:49:52.317 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/198 2024-03-07T06:49:52.317 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 78/198 2024-03-07T06:49:52.317 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 79/198 2024-03-07T06:49:52.317 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/198 2024-03-07T06:49:52.318 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/198 2024-03-07T06:49:52.318 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/198 2024-03-07T06:49:52.318 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/198 2024-03-07T06:49:52.318 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/198 2024-03-07T06:49:52.318 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/198 2024-03-07T06:49:52.318 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/198 2024-03-07T06:49:52.318 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/198 2024-03-07T06:49:52.318 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/198 2024-03-07T06:49:52.318 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/198 2024-03-07T06:49:52.318 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/198 2024-03-07T06:49:52.318 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/198 2024-03-07T06:49:52.318 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/198 2024-03-07T06:49:52.318 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/198 2024-03-07T06:49:52.318 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/198 2024-03-07T06:49:52.319 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/198 2024-03-07T06:49:52.319 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/198 2024-03-07T06:49:52.319 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 97/198 2024-03-07T06:49:52.319 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 98/198 2024-03-07T06:49:52.319 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 99/198 2024-03-07T06:49:52.319 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 100/198 2024-03-07T06:49:52.319 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/198 2024-03-07T06:49:52.319 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 102/198 2024-03-07T06:49:52.319 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/198 2024-03-07T06:49:52.319 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 104/198 2024-03-07T06:49:52.319 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-07T06:49:52.319 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-07T06:49:52.319 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-07T06:49:52.319 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-07T06:49:52.320 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-07T06:49:52.320 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-07T06:49:52.320 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-07T06:49:52.320 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-07T06:49:52.320 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-07T06:49:52.320 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-07T06:49:52.320 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-07T06:49:52.320 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/198 2024-03-07T06:49:52.320 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-07T06:49:52.320 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-07T06:49:52.320 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-07T06:49:52.320 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-07T06:49:52.320 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-07T06:49:52.320 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:2.28.0-1.el9.noarch 124/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-07T06:49:52.321 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 151/198 2024-03-07T06:49:52.322 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 153/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-1:1.46.0-1.el9.x86_64 155/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-libnm-1:1.46.0-1.el9.x86_64 157/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-team-1:1.46.0-1.el9.x86_64 159/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-tui-1:1.46.0-1.el9.x86_64 161/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : acl-2.3.1-4.el9.x86_64 163/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : acl-2.3.1-3.el9.x86_64 164/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 166/198 2024-03-07T06:49:52.323 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 167/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 168/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 169/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 170/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 171/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 172/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 173/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 174/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 175/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 176/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 177/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 178/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 179/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 180/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 181/198 2024-03-07T06:49:52.324 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 182/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 183/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 184/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 185/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 186/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 187/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 188/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 189/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 190/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 191/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 192/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 193/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 194/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-07T06:49:52.325 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-07T06:49:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-07T06:49:54.855 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:49:54.856 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2024-03-07T06:49:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-1:1.46.0-1.el9.x86_64 2024-03-07T06:49:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-libnm-1:1.46.0-1.el9.x86_64 2024-03-07T06:49:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-team-1:1.46.0-1.el9.x86_64 2024-03-07T06:49:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-tui-1:1.46.0-1.el9.x86_64 2024-03-07T06:49:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: acl-2.3.1-4.el9.x86_64 2024-03-07T06:49:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-07T06:49:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-07T06:49:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-07T06:49:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-07T06:49:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-07T06:49:54.856 INFO:teuthology.orchestra.run.smithi133.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-07T06:49:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-07T06:49:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-07T06:49:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-07T06:49:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-07T06:49:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-07T06:49:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-07T06:49:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-07T06:49:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-07T06:49:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-07T06:49:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-07T06:49:54.857 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-07T06:49:54.858 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-03-07T06:49:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-07T06:49:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: automake-1.16.2-8.el9.noarch 2024-03-07T06:49:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: bison-3.7.4-5.el9.x86_64 2024-03-07T06:49:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-07T06:49:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.858 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.859 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top-2:19.0.0-1848.g8b896a9e.el9.noarch 2024-03-07T06:49:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-07T06:49:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: flex-2.6.4-9.el9.x86_64 2024-03-07T06:49:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-07T06:49:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-07T06:49:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-07T06:49:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-07T06:49:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-07T06:49:54.860 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-07T06:49:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: jq-1.6-16.el9.x86_64 2024-03-07T06:49:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-03-07T06:49:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-07T06:49:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-07T06:49:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-07T06:49:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-07T06:49:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-07T06:49:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-07T06:49:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-07T06:49:54.861 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: lua-5.4.4-4.el9.x86_64 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-07T06:49:54.862 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-07T06:49:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-07T06:49:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-13.11-1.el9.x86_64 2024-03-07T06:49:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-07T06:49:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-07T06:49:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-07T06:49:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-07T06:49:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-07T06:49:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-03-07T06:49:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-07T06:49:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-07T06:49:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-07T06:49:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-07T06:49:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.863 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-07T06:49:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-07T06:49:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-07T06:49:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-07T06:49:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-07T06:49:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-07T06:49:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-07T06:49:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-07T06:49:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:2.28.0-1.el9.noarch 2024-03-07T06:49:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-07T06:49:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-07T06:49:54.864 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-07T06:49:54.865 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-07T06:49:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-07T06:49:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-07T06:49:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-07T06:49:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply-3.11-14.el9.noarch 2024-03-07T06:49:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-07T06:49:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-07T06:49:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-07T06:49:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-07T06:49:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-07T06:49:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-07T06:49:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-07T06:49:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-07T06:49:54.866 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-07T06:49:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-07T06:49:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-07T06:49:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-07T06:49:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-07T06:49:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-07T06:49:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-07T06:49:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-07T06:49:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-07T06:49:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-07T06:49:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-07T06:49:54.867 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-07T06:49:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-07T06:49:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-07T06:49:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-07T06:49:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-07T06:49:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-07T06:49:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-07T06:49:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-07T06:49:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:19.0.0-1848.g8b896a9e.el9.x86_64 2024-03-07T06:49:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: re2-1:20211101-3.el9.x86_64 2024-03-07T06:49:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-07T06:49:54.868 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-07T06:49:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-07T06:49:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-07T06:49:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-07T06:49:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-07T06:49:54.869 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:49:54.869 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-03-07T06:49:55.586 DEBUG:teuthology.parallel:result is None 2024-03-07T06:49:55.587 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T06:49:55.732 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-07T06:49:55.785 INFO:teuthology.orchestra.run.smithi133.stdout:19.0.0-1848.g8b896a9e.el9 2024-03-07T06:49:55.786 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1848.g8b896a9e.el9 2024-03-07T06:49:55.786 INFO:teuthology.task.install:The correct ceph version 19.0.0-1848.g8b896a9e is installed. 2024-03-07T06:49:55.788 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8b896a9e145796119f0451201dc4d53ddac97db0 2024-03-07T06:49:56.003 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-07T06:49:56.072 INFO:teuthology.orchestra.run.smithi161.stdout:19.0.0-1848.g8b896a9e.el9 2024-03-07T06:49:56.072 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1848.g8b896a9e.el9 2024-03-07T06:49:56.072 INFO:teuthology.task.install:The correct ceph version 19.0.0-1848.g8b896a9e is installed. 2024-03-07T06:49:56.074 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-07T06:49:56.075 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:49:56.075 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-07T06:49:56.160 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:49:56.161 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-07T06:49:56.263 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-07T06:49:56.263 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:49:56.263 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-03-07T06:49:56.334 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-07T06:49:56.443 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:49:56.443 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2024-03-07T06:49:56.543 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-07T06:49:56.681 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-07T06:49:56.681 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:49:56.681 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-07T06:49:56.754 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-07T06:49:56.869 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:49:56.869 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-07T06:49:56.967 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-07T06:49:57.106 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-07T06:49:57.106 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:49:57.106 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-03-07T06:49:57.182 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-07T06:49:57.295 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:49:57.295 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2024-03-07T06:49:57.396 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-07T06:49:57.488 INFO:teuthology.run_tasks:Running task ceph... 2024-03-07T06:49:57.574 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-07T06:49:57.574 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /var/log/ceph 2024-03-07T06:49:57.579 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /var/log/ceph 2024-03-07T06:49:57.676 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-07T06:49:57.677 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-07T06:49:57.681 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-07T06:49:57.780 INFO:tasks.ceph:Creating extra log directories... 2024-03-07T06:49:57.781 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-07T06:49:57.785 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-07T06:49:57.885 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-07T06:49:57.886 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-07T06:49:57.886 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-07T06:49:57.886 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:49:57.886 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-07T06:49:57.930 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-07T06:49:58.175 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-07T06:49:58.269 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:49:58.269 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-07T06:49:58.325 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-07T06:49:58.591 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-07T06:49:58.694 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-07T06:49:58.694 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-07T06:49:58.694 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926', 'branch': 'wip-lusov-qdb', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/strays}', 'email': None, 'first_in_suite': False, 'job_id': '7583926', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lusov-qdb'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-qdb', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5072, 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'sleep_before_teardown': 0, 'subset': '246/10000', 'suite': 'fs', 'suite_branch': 'wip-lusov-qdb', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi133.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNnQT/fxUBs5Q0elCOxjRtpReucIfdQ7XZT28EvF4NqbJJWKkBq9sNyVNk0kzyXIw+udH5hrcAWgun/AQdgp6tc=', 'smithi161.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMTx5NOlXi6N3VtHmq0Qr1XTvtJezbU/EhMYstAczFirOyXNqu+yRMicxrcRDOcv8elfpQq53o9nBxC/te7xEos='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '8b896a9e145796119f0451201dc4d53ddac97db0', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_strays']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-07_05:44:13', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-07T06:49:58.695 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-07T06:49:58.705 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-07T06:49:58.757 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/run/ceph 2024-03-07T06:49:58.792 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/run/ceph 2024-03-07T06:49:58.890 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:49:58.891 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-03-07T06:49:58.931 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-07T06:49:58.931 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-03-07T06:49:59.011 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-07T06:49:59.011 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T06:49:59.011 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 634 Links: 1 2024-03-07T06:49:59.011 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T06:49:59.011 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T06:49:59.011 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-03-07 06:49:47.662815447 +0000 2024-03-07T06:49:59.011 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-03-07 06:46:44.350449791 +0000 2024-03-07T06:49:59.011 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-03-07 06:46:44.351449772 +0000 2024-03-07T06:49:59.011 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-03-07 06:46:44.350449791 +0000 2024-03-07T06:49:59.012 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-07T06:49:59.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-03-07T06:49:59.116 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-03-07T06:49:59.117 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000373615 s, 1.4 MB/s 2024-03-07T06:49:59.119 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-07T06:49:59.159 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-03-07T06:49:59.248 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-07T06:49:59.248 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T06:49:59.249 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-07T06:49:59.249 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T06:49:59.249 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T06:49:59.249 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-03-07 06:49:47.663815429 +0000 2024-03-07T06:49:59.249 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-03-07 06:46:44.928439209 +0000 2024-03-07T06:49:59.249 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-03-07 06:46:44.928439209 +0000 2024-03-07T06:49:59.249 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-03-07 06:46:44.928439209 +0000 2024-03-07T06:49:59.249 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-07T06:49:59.334 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-03-07T06:49:59.334 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-03-07T06:49:59.334 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000303857 s, 1.7 MB/s 2024-03-07T06:49:59.339 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-07T06:49:59.393 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-03-07T06:49:59.441 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-07T06:49:59.441 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T06:49:59.441 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 671 Links: 1 2024-03-07T06:49:59.441 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T06:49:59.442 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T06:49:59.442 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-03-07 06:49:47.663815429 +0000 2024-03-07T06:49:59.442 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-03-07 06:46:45.493428865 +0000 2024-03-07T06:49:59.442 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-03-07 06:46:45.494428846 +0000 2024-03-07T06:49:59.442 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-03-07 06:46:45.493428865 +0000 2024-03-07T06:49:59.442 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-07T06:49:59.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-03-07T06:49:59.525 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-03-07T06:49:59.525 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000344014 s, 1.5 MB/s 2024-03-07T06:49:59.529 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-07T06:49:59.581 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-03-07T06:49:59.628 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-07T06:49:59.629 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T06:49:59.629 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 691 Links: 1 2024-03-07T06:49:59.629 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T06:49:59.629 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T06:49:59.629 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-03-07 06:49:47.665815393 +0000 2024-03-07T06:49:59.629 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-03-07 06:46:46.055418575 +0000 2024-03-07T06:49:59.629 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-03-07 06:46:46.055418575 +0000 2024-03-07T06:49:59.629 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-03-07 06:46:46.055418575 +0000 2024-03-07T06:49:59.629 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-07T06:49:59.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-03-07T06:49:59.709 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-03-07T06:49:59.709 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000344839 s, 1.5 MB/s 2024-03-07T06:49:59.713 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-07T06:49:59.767 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-07T06:49:59.767 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:49:59.767 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2024-03-07T06:49:59.823 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-07T06:49:59.824 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2024-03-07T06:49:59.870 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-07T06:49:59.870 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T06:49:59.871 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 633 Links: 1 2024-03-07T06:49:59.871 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T06:49:59.871 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T06:49:59.871 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-03-07 06:49:31.938593857 +0000 2024-03-07T06:49:59.871 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-03-07 06:46:43.591110799 +0000 2024-03-07T06:49:59.871 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-03-07 06:46:43.591110799 +0000 2024-03-07T06:49:59.871 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-03-07 06:46:43.591110799 +0000 2024-03-07T06:49:59.871 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-07T06:50:00.002 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-03-07T06:50:00.003 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-03-07T06:50:00.003 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.00036408 s, 1.4 MB/s 2024-03-07T06:50:00.007 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-07T06:50:00.062 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2024-03-07T06:50:00.110 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-07T06:50:00.110 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T06:50:00.110 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-07T06:50:00.110 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T06:50:00.110 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T06:50:00.110 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-03-07 06:49:31.939593837 +0000 2024-03-07T06:50:00.110 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-03-07 06:46:44.191098292 +0000 2024-03-07T06:50:00.111 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-03-07 06:46:44.191098292 +0000 2024-03-07T06:50:00.111 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-03-07 06:46:44.191098292 +0000 2024-03-07T06:50:00.111 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-07T06:50:00.195 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-03-07T06:50:00.195 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-03-07T06:50:00.195 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000281899 s, 1.8 MB/s 2024-03-07T06:50:00.199 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-07T06:50:00.251 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2024-03-07T06:50:00.299 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-07T06:50:00.299 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T06:50:00.299 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 669 Links: 1 2024-03-07T06:50:00.299 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T06:50:00.299 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T06:50:00.299 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-03-07 06:49:31.940593816 +0000 2024-03-07T06:50:00.299 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-03-07 06:46:44.769086244 +0000 2024-03-07T06:50:00.300 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-03-07 06:46:44.769086244 +0000 2024-03-07T06:50:00.300 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-03-07 06:46:44.769086244 +0000 2024-03-07T06:50:00.300 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-07T06:50:00.381 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-03-07T06:50:00.381 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-03-07T06:50:00.381 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000348768 s, 1.5 MB/s 2024-03-07T06:50:00.384 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-07T06:50:00.437 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2024-03-07T06:50:00.487 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-07T06:50:00.487 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-07T06:50:00.487 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-03-07T06:50:00.488 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-07T06:50:00.488 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2024-03-07T06:50:00.488 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2024-03-07 06:49:31.941593795 +0000 2024-03-07T06:50:00.488 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2024-03-07 06:46:45.369073738 +0000 2024-03-07T06:50:00.488 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2024-03-07 06:46:45.369073738 +0000 2024-03-07T06:50:00.488 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: 2024-03-07 06:46:45.369073738 +0000 2024-03-07T06:50:00.488 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-07T06:50:00.572 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2024-03-07T06:50:00.573 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2024-03-07T06:50:00.573 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000297574 s, 1.7 MB/s 2024-03-07T06:50:00.576 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-07T06:50:00.629 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-07T06:50:00.629 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-07T06:50:00.629 INFO:tasks.ceph:Generating config... 2024-03-07T06:50:00.631 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-07T06:50:00.631 INFO:tasks.ceph:[client] debug client = 20 2024-03-07T06:50:00.631 INFO:tasks.ceph:[client] debug ms = 1 2024-03-07T06:50:00.631 INFO:tasks.ceph:[client] pid file = 2024-03-07T06:50:00.631 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-07T06:50:00.631 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-07T06:50:00.631 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-07T06:50:00.632 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-07T06:50:00.632 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-07T06:50:00.632 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-07T06:50:00.632 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-07T06:50:00.632 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-07T06:50:00.632 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-07T06:50:00.633 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-07T06:50:00.633 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-07T06:50:00.633 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-07T06:50:00.633 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-07T06:50:00.633 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-07T06:50:00.633 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-07T06:50:00.633 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-07T06:50:00.633 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-07T06:50:00.633 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-07T06:50:00.634 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-07T06:50:00.634 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-07T06:50:00.634 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-07T06:50:00.634 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-07T06:50:00.634 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-07T06:50:00.634 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-07T06:50:00.634 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-07T06:50:00.634 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-07T06:50:00.634 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-07T06:50:00.634 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-07T06:50:00.634 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-07T06:50:00.634 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-07T06:50:00.634 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-07T06:50:00.634 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-07T06:50:00.634 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-07T06:50:00.635 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-07T06:50:00.635 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-07T06:50:00.635 INFO:tasks.ceph:Setting up mon.a... 2024-03-07T06:50:00.635 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-07T06:50:00.777 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.keyring 2024-03-07T06:50:00.785 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-07T06:50:00.911 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-07T06:50:00.994 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.133'), ('mon.b', '172.21.15.161'), ('mon.c', '[v2:172.21.15.161:3301,v1:172.21.15.161:6790]')] 2024-03-07T06:50:00.994 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.133,172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-07T06:50:00.996 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:00.996 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-07T06:50:01.049 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.133 --add b 172.21.15.161 --addv c '[v2:172.21.15.161:3301,v1:172.21.15.161:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-07T06:50:01.188 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-07T06:50:01.188 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: generated fsid 1655279d-73c2-44d4-ba1b-3e8a15ae005a 2024-03-07T06:50:01.188 INFO:teuthology.orchestra.run.smithi133.stdout:setting min_mon_release = quincy 2024-03-07T06:50:01.188 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 0 2024-03-07T06:50:01.188 INFO:teuthology.orchestra.run.smithi133.stdout:fsid 1655279d-73c2-44d4-ba1b-3e8a15ae005a 2024-03-07T06:50:01.189 INFO:teuthology.orchestra.run.smithi133.stdout:last_changed 2024-03-07T06:50:01.190010+0000 2024-03-07T06:50:01.189 INFO:teuthology.orchestra.run.smithi133.stdout:created 2024-03-07T06:50:01.190010+0000 2024-03-07T06:50:01.189 INFO:teuthology.orchestra.run.smithi133.stdout:min_mon_release 17 (quincy) 2024-03-07T06:50:01.189 INFO:teuthology.orchestra.run.smithi133.stdout:election_strategy: 1 2024-03-07T06:50:01.189 INFO:teuthology.orchestra.run.smithi133.stdout:0: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.a 2024-03-07T06:50:01.189 INFO:teuthology.orchestra.run.smithi133.stdout:1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2024-03-07T06:50:01.189 INFO:teuthology.orchestra.run.smithi133.stdout:2: [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] mon.c 2024-03-07T06:50:01.189 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-07T06:50:01.191 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-07T06:50:01.245 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1655279d-73c2-44d4-ba1b-3e8a15ae005a... 2024-03-07T06:50:01.246 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-07T06:50:01.252 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-07T06:50:01.411 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-03-07T06:50:01.411 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-03-07T06:50:01.411 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-07T06:50:01.411 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-03-07T06:50:01.411 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.411 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-03-07T06:50:01.411 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.411 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-03-07T06:50:01.412 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.412 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-03-07T06:50:01.412 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-03-07T06:50:01.412 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.412 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-03-07T06:50:01.412 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-03-07T06:50:01.412 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.412 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-07T06:50:01.412 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-03-07T06:50:01.412 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.412 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-03-07T06:50:01.412 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-07T06:50:01.412 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.412 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.413 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.133,172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790]" 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on skipped message = False 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 1655279d-73c2-44d4-ba1b-3e8a15ae005a 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-03-07T06:50:01.414 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug trim objects = true 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.415 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2024-03-07T06:50:01.416 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-03-07T06:50:01.417 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-03-07T06:50:01.418 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-03-07T06:50:01.418 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-03-07T06:50:01.418 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-03-07T06:50:01.418 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.418 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-03-07T06:50:01.418 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-03-07T06:50:01.418 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-03-07T06:50:01.418 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-03-07T06:50:01.418 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-03-07T06:50:01.418 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-03-07T06:50:01.418 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-03-07T06:50:01.418 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-03-07T06:50:01.418 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.419 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-03-07T06:50:01.420 INFO:teuthology.orchestra.run.smithi133.stdout: mon op complaint time = 120 2024-03-07T06:50:01.420 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:01.420 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-03-07T06:50:01.420 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-03-07T06:50:01.420 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-03-07T06:50:01.420 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-03-07T06:50:01.420 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-07T06:50:01.420 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-07T06:50:01.420 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 600 2024-03-07T06:50:01.420 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-03-07T06:50:01.420 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-03-07T06:50:01.420 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2024-03-07T06:50:01.420 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-03-07T06:50:01.420 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds = 20 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout: debug mds balancer = 20 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug frag = True 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = True 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout: mds op complaint time = 180 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = True 2024-03-07T06:50:01.421 INFO:teuthology.orchestra.run.smithi133.stdout: osd op complaint time = 180 2024-03-07T06:50:01.422 INFO:teuthology.orchestra.run.smithi133.stdout: rados mon op timeout = 900 2024-03-07T06:50:01.422 INFO:teuthology.orchestra.run.smithi133.stdout: rados osd op timeout = 900 2024-03-07T06:50:01.429 INFO:teuthology.orchestra.run.smithi161.stdout:[global] 2024-03-07T06:50:01.429 INFO:teuthology.orchestra.run.smithi161.stdout: chdir = "" 2024-03-07T06:50:01.429 INFO:teuthology.orchestra.run.smithi161.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-07T06:50:01.429 INFO:teuthology.orchestra.run.smithi161.stdout: auth supported = cephx 2024-03-07T06:50:01.429 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.429 INFO:teuthology.orchestra.run.smithi161.stdout: filestore xattr use omap = true 2024-03-07T06:50:01.429 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.429 INFO:teuthology.orchestra.run.smithi161.stdout: mon clock drift allowed = 1.000 2024-03-07T06:50:01.429 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.429 INFO:teuthology.orchestra.run.smithi161.stdout: osd crush chooseleaf type = 0 2024-03-07T06:50:01.429 INFO:teuthology.orchestra.run.smithi161.stdout: auth debug = true 2024-03-07T06:50:01.429 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.430 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on old message = true 2024-03-07T06:50:01.430 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on bug = true 2024-03-07T06:50:01.430 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.430 INFO:teuthology.orchestra.run.smithi161.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-07T06:50:01.430 INFO:teuthology.orchestra.run.smithi161.stdout: mon pg warn max object skew = 0 2024-03-07T06:50:01.430 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.430 INFO:teuthology.orchestra.run.smithi161.stdout: # disable pg_autoscaler by default for new pools 2024-03-07T06:50:01.430 INFO:teuthology.orchestra.run.smithi161.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-07T06:50:01.430 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.430 INFO:teuthology.orchestra.run.smithi161.stdout: osd pool default size = 2 2024-03-07T06:50:01.430 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.430 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd allow primary affinity = true 2024-03-07T06:50:01.430 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd allow pg remap = true 2024-03-07T06:50:01.430 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on legacy crush tunables = false 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on crush straw calc version zero = false 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on no sortbitwise = false 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on osd down out interval zero = false 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on too few osds = false 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: mon_allow_pool_size_one = true 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: osd default data pool replay window = 5 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: mon allow pool delete = true 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.431 INFO:teuthology.orchestra.run.smithi161.stdout: mon cluster log file level = debug 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout: debug asserts on shutdown = true 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout: mon health detail to clog = false 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout: mon host = "172.21.15.133,172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790]" 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on skipped message = False 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout: fsid = 1655279d-73c2-44d4-ba1b-3e8a15ae005a 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout:[osd] 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout: osd journal size = 100 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout: osd scrub load threshold = 5.0 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout: osd scrub max interval = 600 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout: osd mclock profile = high_recovery_ops 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout: osd recover clone overlap = true 2024-03-07T06:50:01.432 INFO:teuthology.orchestra.run.smithi161.stdout: osd recovery max chunk = 1048576 2024-03-07T06:50:01.433 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.433 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug shutdown = true 2024-03-07T06:50:01.433 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug op order = true 2024-03-07T06:50:01.433 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug verify stray on activate = true 2024-03-07T06:50:01.433 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug trim objects = true 2024-03-07T06:50:01.433 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.433 INFO:teuthology.orchestra.run.smithi161.stdout: osd open classes on start = true 2024-03-07T06:50:01.433 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug pg log writeout = true 2024-03-07T06:50:01.433 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.433 INFO:teuthology.orchestra.run.smithi161.stdout: osd deep scrub update digest min age = 30 2024-03-07T06:50:01.433 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: osd map max advance = 10 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: journal zero on create = true 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: filestore ondisk finisher threads = 3 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: filestore apply finisher threads = 3 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: bdev debug aio = true 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug misdirected ops = true 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: bdev async discard = True 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: bdev enable discard = True 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore allocator = bitmap 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore block size = 96636764160 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore fsck on mount = True 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: debug bluefs = 1/20 2024-03-07T06:50:01.434 INFO:teuthology.orchestra.run.smithi161.stdout: debug bluestore = 1/20 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: debug osd = 20 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: debug rocksdb = 4/10 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd backfillfull_ratio = 0.85 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd full ratio = 0.9 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd nearfull ratio = 0.8 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: osd failsafe full ratio = 0.95 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: osd objectstore = bluestore 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: osd op complaint time = 180 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr] 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: debug mgr = 20 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: debug mon = 20 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: debug auth = 20 2024-03-07T06:50:01.435 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min pgs per osd = 4 2024-03-07T06:50:01.436 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min bytes per osd = 10 2024-03-07T06:50:01.436 INFO:teuthology.orchestra.run.smithi161.stdout: mgr/telemetry/nag = false 2024-03-07T06:50:01.436 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.436 INFO:teuthology.orchestra.run.smithi161.stdout:[mon] 2024-03-07T06:50:01.436 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-03-07T06:50:01.436 INFO:teuthology.orchestra.run.smithi161.stdout: debug mon = 20 2024-03-07T06:50:01.436 INFO:teuthology.orchestra.run.smithi161.stdout: debug paxos = 20 2024-03-07T06:50:01.436 INFO:teuthology.orchestra.run.smithi161.stdout: debug auth = 20 2024-03-07T06:50:01.436 INFO:teuthology.orchestra.run.smithi161.stdout: mon data avail warn = 5 2024-03-07T06:50:01.436 INFO:teuthology.orchestra.run.smithi161.stdout: mon mgr mkfs grace = 240 2024-03-07T06:50:01.436 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min pgs per osd = 4 2024-03-07T06:50:01.436 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd reporter subtree level = osd 2024-03-07T06:50:01.436 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd prime pg temp = true 2024-03-07T06:50:01.436 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min bytes per osd = 10 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: auth mon ticket ttl = 660 # 11m 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: auth service ticket ttl = 240 # 4m 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: # don't complain about insecure global_id in the test suite 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: # 1m isn't quite enough 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: mon_down_mkfs_grace = 2m 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_filestore_osds = false 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: mon op complaint time = 120 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout: 2024-03-07T06:50:01.437 INFO:teuthology.orchestra.run.smithi161.stdout:[client] 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout: rgw cache enabled = true 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout: rgw enable ops log = true 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout: rgw enable usage log = true 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout: client mount timeout = 600 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout: debug client = 20 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout: pid file = "" 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout: rados mon op timeout = 900 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout: rados osd op timeout = 900 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.a] 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.b] 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.c] 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.a] 2024-03-07T06:50:01.438 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.c] 2024-03-07T06:50:01.439 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.b] 2024-03-07T06:50:01.439 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.d] 2024-03-07T06:50:01.439 INFO:teuthology.orchestra.run.smithi161.stdout:[mds] 2024-03-07T06:50:01.439 INFO:teuthology.orchestra.run.smithi161.stdout: debug mds = 20 2024-03-07T06:50:01.439 INFO:teuthology.orchestra.run.smithi161.stdout: debug mds balancer = 20 2024-03-07T06:50:01.439 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2024-03-07T06:50:01.439 INFO:teuthology.orchestra.run.smithi161.stdout: mds debug frag = True 2024-03-07T06:50:01.439 INFO:teuthology.orchestra.run.smithi161.stdout: mds debug scatterstat = True 2024-03-07T06:50:01.439 INFO:teuthology.orchestra.run.smithi161.stdout: mds op complaint time = 180 2024-03-07T06:50:01.439 INFO:teuthology.orchestra.run.smithi161.stdout: mds verify scatter = True 2024-03-07T06:50:01.439 INFO:teuthology.orchestra.run.smithi161.stdout: osd op complaint time = 180 2024-03-07T06:50:01.439 INFO:teuthology.orchestra.run.smithi161.stdout: rados mon op timeout = 900 2024-03-07T06:50:01.439 INFO:teuthology.orchestra.run.smithi161.stdout: rados osd op timeout = 900 2024-03-07T06:50:01.472 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-07T06:50:01.472 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-07T06:50:01.616 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-07T06:50:01.616 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:01.616 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-07T06:50:01.661 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:01.661 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-07T06:50:01.756 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:50:01.757 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:01.757 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-07T06:50:01.757 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-07T06:50:01.891 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:01.891 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-07T06:50:01.986 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi161.front.sepia.ceph.com 2024-03-07T06:50:01.987 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:01.987 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-07T06:50:01.987 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-07T06:50:02.137 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:02.137 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-07T06:50:02.232 INFO:tasks.ceph:Setting up mon nodes... 2024-03-07T06:50:02.232 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-07T06:50:02.232 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-07T06:50:02.415 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-07T06:50:02.425 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-07T06:50:02.631 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-07T06:50:02.643 INFO:tasks.ceph:Setting up mds nodes... 2024-03-07T06:50:02.643 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-07T06:50:02.812 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-07T06:50:02.822 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-07T06:50:02.920 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-07T06:50:03.101 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-07T06:50:03.111 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-07T06:50:03.204 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-07T06:50:03.389 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-07T06:50:03.398 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-07T06:50:03.493 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-07T06:50:03.665 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-07T06:50:03.675 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-07T06:50:03.765 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-07T06:50:03.766 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-07T06:50:03.896 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-07T06:50:03.954 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-07T06:50:04.077 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-07T06:50:04.140 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-07T06:50:04.270 INFO:teuthology.orchestra.run.smithi161.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-07T06:50:04.332 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-07T06:50:04.457 INFO:teuthology.orchestra.run.smithi161.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-07T06:50:04.514 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-07T06:50:04.515 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-07T06:50:04.515 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-07T06:50:04.619 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-07T06:50:04.620 INFO:tasks.ceph:role: osd.0 2024-03-07T06:50:04.620 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:50:04.620 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-07T06:50:04.720 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T06:50:04.720 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T06:50:04.720 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T06:50:04.720 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T06:50:04.720 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T06:50:04.721 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-03-07T06:50:04.721 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T06:50:04.721 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T06:50:04.721 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T06:50:04.721 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T06:50:05.229 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-03-07T06:50:05.238 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-03-07T06:50:05.238 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-07T06:50:05.426 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-07T06:50:05.529 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-07T06:50:05.616 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-07T06:50:05.616 INFO:tasks.ceph:role: osd.1 2024-03-07T06:50:05.617 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:50:05.617 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-07T06:50:05.703 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T06:50:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T06:50:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T06:50:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T06:50:05.703 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T06:50:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-03-07T06:50:05.703 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T06:50:05.703 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T06:50:05.703 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T06:50:05.703 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T06:50:06.164 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-03-07T06:50:06.169 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-03-07T06:50:06.169 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-07T06:50:06.287 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-07T06:50:06.380 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-07T06:50:06.463 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-07T06:50:06.463 INFO:tasks.ceph:role: osd.2 2024-03-07T06:50:06.463 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:50:06.463 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-07T06:50:06.546 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T06:50:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T06:50:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T06:50:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T06:50:06.547 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T06:50:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-03-07T06:50:06.547 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T06:50:06.547 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T06:50:06.547 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T06:50:06.547 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T06:50:07.066 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-03-07T06:50:07.073 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-03-07T06:50:07.073 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-07T06:50:07.251 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-07T06:50:07.349 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-07T06:50:07.444 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-07T06:50:07.445 INFO:tasks.ceph:role: osd.3 2024-03-07T06:50:07.445 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:50:07.445 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-07T06:50:07.543 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T06:50:07.543 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T06:50:07.543 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T06:50:07.543 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T06:50:07.543 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T06:50:07.543 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-03-07T06:50:07.544 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T06:50:07.544 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T06:50:07.544 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T06:50:07.544 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T06:50:08.005 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-03-07T06:50:08.013 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-03-07T06:50:08.013 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-07T06:50:08.162 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-07T06:50:08.254 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T06:50:08.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:08.393+0000 7f7e5778a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-07T06:50:08.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:08.398+0000 7f7e5778a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-07T06:50:08.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:08.398+0000 7f7e5778a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-07T06:50:08.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:08.398+0000 7f7e5778a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-07T06:50:08.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:08.398+0000 7f7e5778a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-07T06:50:08.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:08.401+0000 7f7e5778a640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-07T06:50:11.339 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-07T06:50:11.430 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T06:50:11.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:11.568+0000 7fd82739a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-07T06:50:11.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:11.569+0000 7fd82739a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-07T06:50:11.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:11.569+0000 7fd82739a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-07T06:50:11.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:11.569+0000 7fd82739a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-07T06:50:11.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:11.569+0000 7fd82739a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-07T06:50:11.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:11.571+0000 7fd82739a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-07T06:50:14.739 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-07T06:50:14.837 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T06:50:14.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:14.974+0000 7fce56817640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-07T06:50:14.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:14.975+0000 7fce56817640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-07T06:50:14.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:14.975+0000 7fce56817640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-07T06:50:14.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:14.975+0000 7fce56817640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-07T06:50:14.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:14.975+0000 7fce56817640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-07T06:50:14.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:14.977+0000 7fce56817640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-07T06:50:18.143 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-07T06:50:18.235 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T06:50:18.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:18.367+0000 7f590dc0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-07T06:50:18.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:18.367+0000 7f590dc0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-07T06:50:18.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:18.367+0000 7f590dc0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-07T06:50:18.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:18.367+0000 7f590dc0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-07T06:50:18.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:18.367+0000 7f590dc0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-07T06:50:18.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:18.369+0000 7f590dc0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-07T06:50:21.540 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-07T06:50:21.634 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-07T06:50:21.735 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-07T06:50:21.735 INFO:tasks.ceph:role: osd.4 2024-03-07T06:50:21.735 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com 2024-03-07T06:50:21.735 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-07T06:50:21.831 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T06:50:21.831 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T06:50:21.831 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T06:50:21.831 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T06:50:21.831 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T06:50:21.831 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-03-07T06:50:21.831 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T06:50:21.831 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T06:50:21.831 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T06:50:21.831 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T06:50:22.345 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2024-03-07T06:50:22.353 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-03-07T06:50:22.353 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-07T06:50:22.546 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-07T06:50:22.652 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-07T06:50:22.738 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-07T06:50:22.738 INFO:tasks.ceph:role: osd.5 2024-03-07T06:50:22.738 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com 2024-03-07T06:50:22.738 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-07T06:50:22.826 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T06:50:22.826 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T06:50:22.826 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T06:50:22.826 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T06:50:22.826 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T06:50:22.826 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-03-07T06:50:22.826 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T06:50:22.826 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T06:50:22.826 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T06:50:22.826 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T06:50:23.301 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2024-03-07T06:50:23.308 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-03-07T06:50:23.309 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-07T06:50:23.434 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-07T06:50:23.516 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-07T06:50:23.596 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-07T06:50:23.596 INFO:tasks.ceph:role: osd.6 2024-03-07T06:50:23.596 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com 2024-03-07T06:50:23.596 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-07T06:50:23.684 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T06:50:23.684 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T06:50:23.684 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T06:50:23.684 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T06:50:23.684 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T06:50:23.684 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-03-07T06:50:23.684 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T06:50:23.684 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T06:50:23.684 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T06:50:23.684 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T06:50:24.201 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2024-03-07T06:50:24.206 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-03-07T06:50:24.206 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-07T06:50:24.392 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-07T06:50:24.488 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-07T06:50:24.576 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-07T06:50:24.576 INFO:tasks.ceph:role: osd.7 2024-03-07T06:50:24.576 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com 2024-03-07T06:50:24.576 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-07T06:50:24.675 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-07T06:50:24.676 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-07T06:50:24.676 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-07T06:50:24.676 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-07T06:50:24.676 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-07T06:50:24.676 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2024-03-07T06:50:24.676 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-07T06:50:24.676 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-07T06:50:24.676 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-07T06:50:24.676 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-07T06:50:25.136 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2024-03-07T06:50:25.142 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2024-03-07T06:50:25.143 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-07T06:50:25.286 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-07T06:50:25.373 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T06:50:25.519 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:25.520+0000 7f29ec60e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-07T06:50:25.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:25.521+0000 7f29ec60e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-07T06:50:25.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:25.521+0000 7f29ec60e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-07T06:50:25.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:25.521+0000 7f29ec60e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-07T06:50:25.521 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:25.521+0000 7f29ec60e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-07T06:50:25.525 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:25.526+0000 7f29ec60e640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-07T06:50:28.696 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:50:28.704 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:50:28.712 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-07T06:50:28.843 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T06:50:28.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:28.980+0000 7f030a58f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-07T06:50:28.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:28.981+0000 7f030a58f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-07T06:50:28.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:28.981+0000 7f030a58f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-07T06:50:28.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:28.981+0000 7f030a58f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-07T06:50:28.981 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:28.981+0000 7f030a58f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-07T06:50:28.983 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:28.984+0000 7f030a58f640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-07T06:50:31.907 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-07T06:50:32.004 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T06:50:32.145 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:32.145+0000 7f6657e19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-07T06:50:32.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:32.146+0000 7f6657e19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-07T06:50:32.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:32.146+0000 7f6657e19640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-07T06:50:32.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:32.146+0000 7f6657e19640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-07T06:50:32.146 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:32.146+0000 7f6657e19640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-07T06:50:32.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:32.148+0000 7f6657e19640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-07T06:50:35.329 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-07T06:50:35.422 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-07T06:50:35.560 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:35.560+0000 7fb43ef0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-07T06:50:35.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:35.561+0000 7fb43ef0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-07T06:50:35.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:35.561+0000 7fb43ef0f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-07T06:50:35.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:35.561+0000 7fb43ef0f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-07T06:50:35.561 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:35.561+0000 7fb43ef0f640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-07T06:50:35.563 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:35.563+0000 7fb43ef0f640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-07T06:50:38.486 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-07T06:50:38.580 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-07T06:50:38.581 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:38.581 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-07T06:50:38.680 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:38.680 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-07T06:50:38.766 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:38.766 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-07T06:50:38.904 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:38.904 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-07T06:50:39.003 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:39.003 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-07T06:50:39.093 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:39.093 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-07T06:50:39.188 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:39.188 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-07T06:50:39.326 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:39.326 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-07T06:50:39.426 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:39.426 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-07T06:50:39.520 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:39.520 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-07T06:50:39.617 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:39.618 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-07T06:50:39.710 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:39.710 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-07T06:50:39.802 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:39.802 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-07T06:50:39.896 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:39.897 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-07T06:50:39.990 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:39.991 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-07T06:50:40.046 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:40.046 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-07T06:50:40.095 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:40.095 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-07T06:50:40.150 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:40.150 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-07T06:50:40.201 INFO:tasks.ceph:Adding keys to all mons... 2024-03-07T06:50:40.202 DEBUG:teuthology.orchestra.run.smithi133:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-07T06:50:40.209 DEBUG:teuthology.orchestra.run.smithi161:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-07T06:50:40.294 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.y] 2024-03-07T06:50:40.294 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCaY+llosjXGBAAjyuWL3GnYWP6nHe9jnyUNg== 2024-03-07T06:50:40.294 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.a] 2024-03-07T06:50:40.294 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCaY+llrqSIMBAAy1d9K8CCdERld7xCmKOutQ== 2024-03-07T06:50:40.294 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.c] 2024-03-07T06:50:40.294 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCbY+ll0XgoBhAAXJiu001WO6mHv/UaVuHoJA== 2024-03-07T06:50:40.294 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.0] 2024-03-07T06:50:40.294 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCgY+llgbaKFxAAUNMCRhWDof+qhuedrCY57Q== 2024-03-07T06:50:40.294 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.1] 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCjY+llQx/1IRAA5yMfKhH4CxK5751ymJYSzQ== 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.2] 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCmY+lluw4uOhAAvy/SoCA7TWJGid53bj1fhA== 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.3] 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCqY+llf5L0FRAA/WlR9r+5bSmu+Y4ggoobdw== 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.x] 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCaY+llqK68JRAAuPudks7z1/uL593N8/VyGA== 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.b] 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCbY+llx7hIFxAAaEpBpdI7eaGAU9GuxPPfhg== 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.d] 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCbY+llrDLKJxAAASZPgaXrsJYHB3pzksPcSA== 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.4] 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCxY+llYmATHxAA+eGqKm4rlXZtcYgV3bZDDg== 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.5] 2024-03-07T06:50:40.295 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQC0Y+llQJ+EOhAAs7gC9YWEap3xhX/Qiyt0lg== 2024-03-07T06:50:40.296 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.6] 2024-03-07T06:50:40.296 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQC4Y+llpADECBAA42RfVF05S45UiaHJyPCRzA== 2024-03-07T06:50:40.296 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.7] 2024-03-07T06:50:40.296 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQC7Y+llC4N/IRAAcnaB9/yZj2fz9nL6hS5mSA== 2024-03-07T06:50:40.296 INFO:teuthology.orchestra.run.smithi161.stdout:[client.0] 2024-03-07T06:50:40.296 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCbY+ll6xWENRAArKbw8ID7PLMFVCO3UlKDOg== 2024-03-07T06:50:40.296 INFO:teuthology.orchestra.run.smithi161.stdout:[client.2] 2024-03-07T06:50:40.296 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCcY+llvYWxBBAA32ACXxhIhSDI+wI6zZiafA== 2024-03-07T06:50:40.296 INFO:teuthology.orchestra.run.smithi161.stdout:[client.1] 2024-03-07T06:50:40.296 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCcY+llCvgxEBAA+PC1o+JD9S0G7IUJVLVh+g== 2024-03-07T06:50:40.296 INFO:teuthology.orchestra.run.smithi161.stdout:[client.3] 2024-03-07T06:50:40.296 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCcY+llwgZYGxAAtbCo73WlwL2ftJ4yyFEzXA== 2024-03-07T06:50:40.297 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.y] 2024-03-07T06:50:40.297 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCaY+llosjXGBAAjyuWL3GnYWP6nHe9jnyUNg== 2024-03-07T06:50:40.297 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-03-07T06:50:40.297 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCaY+llrqSIMBAAy1d9K8CCdERld7xCmKOutQ== 2024-03-07T06:50:40.297 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-03-07T06:50:40.297 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCbY+ll0XgoBhAAXJiu001WO6mHv/UaVuHoJA== 2024-03-07T06:50:40.297 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.0] 2024-03-07T06:50:40.297 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCgY+llgbaKFxAAUNMCRhWDof+qhuedrCY57Q== 2024-03-07T06:50:40.297 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.1] 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCjY+llQx/1IRAA5yMfKhH4CxK5751ymJYSzQ== 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.2] 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCmY+lluw4uOhAAvy/SoCA7TWJGid53bj1fhA== 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.3] 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCqY+llf5L0FRAA/WlR9r+5bSmu+Y4ggoobdw== 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.x] 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCaY+llqK68JRAAuPudks7z1/uL593N8/VyGA== 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCbY+llx7hIFxAAaEpBpdI7eaGAU9GuxPPfhg== 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCbY+llrDLKJxAAASZPgaXrsJYHB3pzksPcSA== 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.4] 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCxY+llYmATHxAA+eGqKm4rlXZtcYgV3bZDDg== 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.5] 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQC0Y+llQJ+EOhAAs7gC9YWEap3xhX/Qiyt0lg== 2024-03-07T06:50:40.298 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.6] 2024-03-07T06:50:40.299 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQC4Y+llpADECBAA42RfVF05S45UiaHJyPCRzA== 2024-03-07T06:50:40.299 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.7] 2024-03-07T06:50:40.299 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQC7Y+llC4N/IRAAcnaB9/yZj2fz9nL6hS5mSA== 2024-03-07T06:50:40.299 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-03-07T06:50:40.299 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCbY+ll6xWENRAArKbw8ID7PLMFVCO3UlKDOg== 2024-03-07T06:50:40.299 INFO:teuthology.orchestra.run.smithi133.stdout:[client.2] 2024-03-07T06:50:40.299 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCcY+llvYWxBBAA32ACXxhIhSDI+wI6zZiafA== 2024-03-07T06:50:40.299 INFO:teuthology.orchestra.run.smithi133.stdout:[client.1] 2024-03-07T06:50:40.299 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCcY+llCvgxEBAA+PC1o+JD9S0G7IUJVLVh+g== 2024-03-07T06:50:40.299 INFO:teuthology.orchestra.run.smithi133.stdout:[client.3] 2024-03-07T06:50:40.299 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCcY+llwgZYGxAAtbCo73WlwL2ftJ4yyFEzXA== 2024-03-07T06:50:40.301 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-07T06:50:40.308 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-07T06:50:40.479 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T06:50:40.486 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T06:50:40.633 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T06:50:40.641 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T06:50:40.787 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:40.792 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:40.934 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:40.940 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:41.083 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:41.088 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:41.233 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:41.238 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:41.379 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-07T06:50:41.385 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-07T06:50:41.529 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T06:50:41.535 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T06:50:41.678 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T06:50:41.683 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-07T06:50:41.830 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:41.835 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:41.975 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:41.982 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:42.123 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:42.132 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:42.269 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:42.274 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-07T06:50:42.415 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T06:50:42.420 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T06:50:42.563 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T06:50:42.567 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T06:50:42.714 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T06:50:42.719 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T06:50:42.866 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T06:50:42.871 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-07T06:50:43.009 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-07T06:50:43.009 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-07T06:50:43.105 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-07T06:50:43.260 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-07T06:50:43.356 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-07T06:50:43.459 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-07T06:50:43.618 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-07T06:50:43.703 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-07T06:50:43.796 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-07T06:50:43.951 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-07T06:50:44.031 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-07T06:50:44.038 DEBUG:teuthology.orchestra.run.smithi161:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-07T06:50:44.088 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-07T06:50:44.089 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-07T06:50:44.089 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-07T06:50:44.096 INFO:tasks.ceph.mon.a:Started 2024-03-07T06:50:44.096 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-07T06:50:44.096 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-07T06:50:44.101 INFO:tasks.ceph.mon.b:Started 2024-03-07T06:50:44.101 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-07T06:50:44.101 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-07T06:50:44.107 INFO:tasks.ceph.mon.c:Started 2024-03-07T06:50:44.107 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-07T06:50:44.107 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-07T06:50:44.108 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-07T06:50:44.115 INFO:tasks.ceph.mgr.y:Started 2024-03-07T06:50:44.115 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-07T06:50:44.116 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-07T06:50:44.121 INFO:tasks.ceph.mgr.x:Started 2024-03-07T06:50:44.121 INFO:tasks.ceph:Setting crush tunables to default 2024-03-07T06:50:44.122 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd crush tunables default 2024-03-07T06:50:44.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.300+0000 7efd5e000640 1 Processor -- start 2024-03-07T06:50:44.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.301+0000 7efd5e000640 1 -- start start 2024-03-07T06:50:44.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.302+0000 7efd5e000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd58071550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:44.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.302+0000 7efd5e000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd580d0060 con 0x7efd580ce1a0 2024-03-07T06:50:44.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.302+0000 7efd5e000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd580d0890 con 0x7efd580ce580 2024-03-07T06:50:44.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.302+0000 7efd5e000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd580d02d0 con 0x7efd580cfbd0 2024-03-07T06:50:44.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.302+0000 7efd57400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7efd580ce1a0 0x7efd5815bee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:58756/0 (socket says 172.21.15.133:58756) 2024-03-07T06:50:44.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.302+0000 7efd57e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd58071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:44.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.302+0000 7efd57400640 1 -- 172.21.15.133:0/119858570 learned_addr learned my addr 172.21.15.133:0/119858570 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:50:44.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.303+0000 7efd57e00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 msgr2=0x7efd58071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-07T06:50:44.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.303+0000 7efd57e00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 msgr2=0x7efd58071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T06:50:44.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.303+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd58071550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T06:50:44.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.303+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd58071550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-07T06:50:44.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.504+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd58071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:44.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.505+0000 7efd57e00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 msgr2=0x7efd58071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-07T06:50:44.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.505+0000 7efd57e00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 msgr2=0x7efd58071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T06:50:44.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.505+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd58071550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T06:50:44.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.505+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd58071550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-07T06:50:44.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.906+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd58071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:44.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.907+0000 7efd57e00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 msgr2=0x7efd58071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-07T06:50:44.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.907+0000 7efd57e00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 msgr2=0x7efd58071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T06:50:44.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.907+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd58071550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T06:50:44.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:44.907+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd58071550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-07T06:50:45.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:45.708+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd58071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:45.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:45.709+0000 7efd57e00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 msgr2=0x7efd58071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-07T06:50:45.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:45.709+0000 7efd57e00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 msgr2=0x7efd58071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T06:50:45.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:45.709+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd58071550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T06:50:45.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:45.709+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd58071550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-07T06:50:47.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.302+0000 7efd55600640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 msgr2=0x7efd58071550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-07T06:50:47.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.302+0000 7efd55600640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd58071550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:47.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.302+0000 7efd55600640 1 -- 172.21.15.133:0/119858570 >> v1:172.21.15.161:6789/0 conn(0x7efd580ce580 legacy=0x7efd5815f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:47.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.302+0000 7efd55600640 1 -- 172.21.15.133:0/119858570 >> v1:172.21.15.133:6789/0 conn(0x7efd580ce1a0 legacy=0x7efd5815bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:47.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.302+0000 7efd55600640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 0x7efd34005b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:47.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.302+0000 7efd55600640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 0x7efd340086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:47.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.302+0000 7efd55600640 1 -- 172.21.15.133:0/119858570 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd580d02d0 con 0x7efd340061f0 2024-03-07T06:50:47.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.302+0000 7efd55600640 1 -- 172.21.15.133:0/119858570 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd34008d70 con 0x7efd34003730 2024-03-07T06:50:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.303+0000 7efd4fe00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 0x7efd340086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.303+0000 7efd57400640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 0x7efd34005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.304+0000 7efd4fe00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 msgr2=0x7efd340086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-07T06:50:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.304+0000 7efd4fe00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 msgr2=0x7efd340086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T06:50:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.304+0000 7efd4fe00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 0x7efd340086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T06:50:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.304+0000 7efd57400640 1 -- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 msgr2=0x7efd34005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-07T06:50:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.304+0000 7efd57400640 1 -- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 msgr2=0x7efd34005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T06:50:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.304+0000 7efd57400640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 0x7efd34005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T06:50:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.304+0000 7efd4fe00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 0x7efd340086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-07T06:50:47.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.304+0000 7efd57400640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 0x7efd34005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-07T06:50:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.505+0000 7efd4fe00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 0x7efd340086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.505+0000 7efd57400640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 0x7efd34005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.505+0000 7efd4fe00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 msgr2=0x7efd340086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-07T06:50:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.505+0000 7efd4fe00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 msgr2=0x7efd340086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T06:50:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.505+0000 7efd4fe00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 0x7efd340086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T06:50:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.506+0000 7efd4fe00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 0x7efd340086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-07T06:50:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.506+0000 7efd57400640 1 -- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 msgr2=0x7efd34005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-07T06:50:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.506+0000 7efd57400640 1 -- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 msgr2=0x7efd34005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T06:50:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.506+0000 7efd57400640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 0x7efd34005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T06:50:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.506+0000 7efd57400640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 0x7efd34005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-07T06:50:47.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.907+0000 7efd4fe00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 0x7efd340086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:47.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.907+0000 7efd4fe00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 msgr2=0x7efd340086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-07T06:50:47.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.907+0000 7efd4fe00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 msgr2=0x7efd340086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T06:50:47.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.907+0000 7efd4fe00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 0x7efd340086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T06:50:47.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.907+0000 7efd57400640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 0x7efd34005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:47.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.907+0000 7efd4fe00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 0x7efd340086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-07T06:50:47.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.908+0000 7efd57400640 1 -- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 msgr2=0x7efd34005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-07T06:50:47.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.908+0000 7efd57400640 1 -- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 msgr2=0x7efd34005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T06:50:47.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.908+0000 7efd57400640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 0x7efd34005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T06:50:47.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:47.908+0000 7efd57400640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 0x7efd34005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-07T06:50:48.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:48.709+0000 7efd4fe00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 0x7efd340086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:48.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:48.709+0000 7efd4fe00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 msgr2=0x7efd340086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-07T06:50:48.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:48.709+0000 7efd4fe00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 msgr2=0x7efd340086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T06:50:48.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:48.709+0000 7efd4fe00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 0x7efd340086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T06:50:48.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:48.709+0000 7efd57400640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 0x7efd34005b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:48.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:48.709+0000 7efd4fe00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 0x7efd340086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-07T06:50:48.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:48.710+0000 7efd57400640 1 -- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 msgr2=0x7efd34005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-07T06:50:48.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:48.710+0000 7efd57400640 1 -- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 msgr2=0x7efd34005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-07T06:50:48.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:48.710+0000 7efd57400640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 0x7efd34005b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-07T06:50:48.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:48.710+0000 7efd57400640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 0x7efd34005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-07T06:50:49.998 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:49.998+0000 7f63afe14040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-07T06:50:50.065 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:50.066+0000 7f63afe14040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-07T06:50:50.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.303+0000 7efd55600640 1 -- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 msgr2=0x7efd34005b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-07T06:50:50.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.303+0000 7efd55600640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd34003730 0x7efd34005b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:50.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.303+0000 7efd55600640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 msgr2=0x7efd340086b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-07T06:50:50.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.303+0000 7efd55600640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.133:3300/0 conn(0x7efd340061f0 0x7efd340086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:50.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.303+0000 7efd57e00640 1 -- 172.21.15.133:0/119858570 reap_dead start 2024-03-07T06:50:50.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.303+0000 7efd55600640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd34012100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:50.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.303+0000 7efd55600640 1 -- 172.21.15.133:0/119858570 --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd580d05b0 con 0x7efd580ce1a0 2024-03-07T06:50:50.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.303+0000 7efd55600640 1 -- 172.21.15.133:0/119858570 --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd340076d0 con 0x7efd580ce580 2024-03-07T06:50:50.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.303+0000 7efd55600640 1 -- 172.21.15.133:0/119858570 --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd580d02d0 con 0x7efd580cfbd0 2024-03-07T06:50:50.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.303+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd34012100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:50.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.304+0000 7efd56a00640 1 -- 172.21.15.133:0/119858570 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1627862536 0 0) 0x7efd580d05b0 con 0x7efd580ce1a0 2024-03-07T06:50:50.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.304+0000 7efd56a00640 1 -- 172.21.15.133:0/119858570 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd3c003520 con 0x7efd580ce1a0 2024-03-07T06:50:50.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.304+0000 7efd56a00640 1 -- 172.21.15.133:0/119858570 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1904296489 0 0) 0x7efd3c003520 con 0x7efd580ce1a0 2024-03-07T06:50:50.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.304+0000 7efd56a00640 1 -- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 msgr2=0x7efd34012100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:50:50.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.304+0000 7efd56a00640 1 --2- 172.21.15.133:0/119858570 >> v2:172.21.15.161:3300/0 conn(0x7efd580cfbd0 0x7efd34012100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:50.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.304+0000 7efd56a00640 1 -- 172.21.15.133:0/119858570 >> v1:172.21.15.161:6789/0 conn(0x7efd580ce580 legacy=0x7efd3400e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:50.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.304+0000 7efd56a00640 1 -- 172.21.15.133:0/119858570 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd340088b0 con 0x7efd580ce1a0 2024-03-07T06:50:50.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.304+0000 7efd56a00640 1 -- 172.21.15.133:0/119858570 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7efd48001460 con 0x7efd580ce1a0 2024-03-07T06:50:50.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.304+0000 7efd56a00640 1 -- 172.21.15.133:0/119858570 >> v1:172.21.15.133:6789/0 conn(0x7efd580ce1a0 legacy=0x7efd3400b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:50.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.304+0000 7efd56a00640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd3c0039b0 0x7efd3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:50.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.304+0000 7efd56a00640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd3c004800 0x7efd3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:50.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.304+0000 7efd56a00640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c0051c0 0x7efd3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:50.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.305+0000 7efd56a00640 1 -- 172.21.15.133:0/119858570 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd580d02d0 con 0x7efd3c0051c0 2024-03-07T06:50:50.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.305+0000 7efd56a00640 1 -- 172.21.15.133:0/119858570 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c00c160 con 0x7efd3c004800 2024-03-07T06:50:50.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.305+0000 7efd56a00640 1 -- 172.21.15.133:0/119858570 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c00c360 con 0x7efd3c0039b0 2024-03-07T06:50:50.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.305+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c0051c0 0x7efd3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:50.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.305+0000 7efd4fe00640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd3c004800 0x7efd3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:50.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.305+0000 7efd57400640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd3c0039b0 0x7efd3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:50.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.305+0000 7efd57e00640 1 -- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd3c0039b0 msgr2=0x7efd3c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:50:50.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.305+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd3c0039b0 0x7efd3c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:50.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.305+0000 7efd57e00640 1 -- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd3c004800 msgr2=0x7efd3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:50:50.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.305+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd3c004800 0x7efd3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:50.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.305+0000 7efd57e00640 1 -- 172.21.15.133:0/119858570 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c003520 con 0x7efd3c0051c0 2024-03-07T06:50:50.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.305+0000 7efd57400640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd3c0039b0 0x7efd3c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:50:50.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.306+0000 7efd57e00640 1 -- 172.21.15.133:0/119858570 reap_dead start 2024-03-07T06:50:50.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.307+0000 7efd57e00640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c0051c0 0x7efd3c00b9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efd34008cb0 tx=0x7efd3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f79b1877f3170a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:50.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.307+0000 7efd56a00640 1 -- 172.21.15.133:0/119858570 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50033070 con 0x7efd3c0051c0 2024-03-07T06:50:50.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.307+0000 7efd56a00640 1 -- 172.21.15.133:0/119858570 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd50031070 con 0x7efd3c0051c0 2024-03-07T06:50:50.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.307+0000 7efd56a00640 1 -- 172.21.15.133:0/119858570 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd50004060 con 0x7efd3c0051c0 2024-03-07T06:50:50.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.308+0000 7efd5e000640 1 -- 172.21.15.133:0/119858570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c0051c0 msgr2=0x7efd3c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:50.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.308+0000 7efd5e000640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c0051c0 0x7efd3c00b9c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efd34008cb0 tx=0x7efd3c005830 comp rx=0 tx=0).stop 2024-03-07T06:50:50.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.308+0000 7efd5e000640 1 -- 172.21.15.133:0/119858570 shutdown_connections 2024-03-07T06:50:50.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.308+0000 7efd5e000640 1 --2- 172.21.15.133:0/119858570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c0051c0 0x7efd3c00b9c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:50.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.308+0000 7efd5e000640 1 -- 172.21.15.133:0/119858570 >> 172.21.15.133:0/119858570 conn(0x7efd58072ae0 msgr2=0x7efd58072ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:50.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.308+0000 7efd5e000640 1 -- 172.21.15.133:0/119858570 shutdown_connections 2024-03-07T06:50:50.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.308+0000 7efd5e000640 1 -- 172.21.15.133:0/119858570 wait complete. 2024-03-07T06:50:50.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.311+0000 7efd5e000640 1 Processor -- start 2024-03-07T06:50:50.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.311+0000 7efd5e000640 1 -- start start 2024-03-07T06:50:50.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.313+0000 7efd5e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c0051c0 0x7efd58159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:50.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.313+0000 7efd5e000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd5814e3c0 0x7efd58156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:50.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.313+0000 7efd5e000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd58157410 0x7efd58157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:50.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.313+0000 7efd5e000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5815f3b0 con 0x7efd3c0051c0 2024-03-07T06:50:50.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.313+0000 7efd5e000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5815b9c0 con 0x7efd5814e3c0 2024-03-07T06:50:50.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.313+0000 7efd5e000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5815f530 con 0x7efd58157410 2024-03-07T06:50:50.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.313+0000 7efd57400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c0051c0 0x7efd58159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:50.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.313+0000 7efd57400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c0051c0 0x7efd58159430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37814/0 (socket says 172.21.15.133:37814) 2024-03-07T06:50:50.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.313+0000 7efd57400640 1 -- 172.21.15.133:0/3917464237 learned_addr learned my addr 172.21.15.133:0/3917464237 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:50:50.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.313+0000 7efd4fe00640 1 --2- 172.21.15.133:0/3917464237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd5814e3c0 0x7efd58156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:50.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.313+0000 7efd57e00640 1 --2- 172.21.15.133:0/3917464237 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd58157410 0x7efd58157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:50.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.314+0000 7efd57400640 1 -- 172.21.15.133:0/3917464237 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd58157410 msgr2=0x7efd58157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:50.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.314+0000 7efd57400640 1 --2- 172.21.15.133:0/3917464237 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd58157410 0x7efd58157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:50.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.314+0000 7efd57400640 1 -- 172.21.15.133:0/3917464237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd5814e3c0 msgr2=0x7efd58156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:50.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.314+0000 7efd57400640 1 --2- 172.21.15.133:0/3917464237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd5814e3c0 0x7efd58156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.314+0000 7efd57400640 1 -- 172.21.15.133:0/3917464237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5815ab20 con 0x7efd3c0051c0 2024-03-07T06:50:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.314+0000 7efd57400640 1 --2- 172.21.15.133:0/3917464237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c0051c0 0x7efd58159430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efd48005e10 tx=0x7efd48007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.314+0000 7efd4f400640 1 -- 172.21.15.133:0/3917464237 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48017640 con 0x7efd3c0051c0 2024-03-07T06:50:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.314+0000 7efd4f400640 1 -- 172.21.15.133:0/3917464237 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd480177e0 con 0x7efd3c0051c0 2024-03-07T06:50:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.314+0000 7efd5e000640 1 -- 172.21.15.133:0/3917464237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd58159a80 con 0x7efd3c0051c0 2024-03-07T06:50:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.314+0000 7efd4f400640 1 -- 172.21.15.133:0/3917464237 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48017b90 con 0x7efd3c0051c0 2024-03-07T06:50:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.315+0000 7efd5e000640 1 -- 172.21.15.133:0/3917464237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5815a040 con 0x7efd3c0051c0 2024-03-07T06:50:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.315+0000 7efd4f400640 1 -- 172.21.15.133:0/3917464237 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7efd48013070 con 0x7efd3c0051c0 2024-03-07T06:50:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.315+0000 7efd4f400640 1 -- 172.21.15.133:0/3917464237 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efd48015a90 con 0x7efd3c0051c0 2024-03-07T06:50:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.316+0000 7efd5e000640 1 -- 172.21.15.133:0/3917464237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd58062c90 con 0x7efd3c0051c0 2024-03-07T06:50:50.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.319+0000 7efd4f400640 1 -- 172.21.15.133:0/3917464237 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7efd4801b070 con 0x7efd3c0051c0 2024-03-07T06:50:50.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.393+0000 7efd5e000640 1 -- 172.21.15.133:0/3917464237 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7efd5806f840 con 0x7efd3c0051c0 2024-03-07T06:50:50.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.397+0000 7efd4f400640 1 -- 172.21.15.133:0/3917464237 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7efd48017d30 con 0x7efd3c0051c0 2024-03-07T06:50:50.397 INFO:teuthology.orchestra.run.smithi133.stderr:adjusted tunables profile to default 2024-03-07T06:50:50.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.398+0000 7efd5e000640 1 -- 172.21.15.133:0/3917464237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c0051c0 msgr2=0x7efd58159430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:50.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.398+0000 7efd5e000640 1 --2- 172.21.15.133:0/3917464237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c0051c0 0x7efd58159430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efd48005e10 tx=0x7efd48007420 comp rx=0 tx=0).stop 2024-03-07T06:50:50.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.398+0000 7efd5e000640 1 -- 172.21.15.133:0/3917464237 shutdown_connections 2024-03-07T06:50:50.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.398+0000 7efd5e000640 1 --2- 172.21.15.133:0/3917464237 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efd58157410 0x7efd58157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:50.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.398+0000 7efd5e000640 1 --2- 172.21.15.133:0/3917464237 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efd5814e3c0 0x7efd58156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:50.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.398+0000 7efd5e000640 1 --2- 172.21.15.133:0/3917464237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c0051c0 0x7efd58159430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:50.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.398+0000 7efd5e000640 1 -- 172.21.15.133:0/3917464237 >> 172.21.15.133:0/3917464237 conn(0x7efd58072ae0 msgr2=0x7efd580b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:50.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.398+0000 7efd5e000640 1 -- 172.21.15.133:0/3917464237 shutdown_connections 2024-03-07T06:50:50.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:50.399+0000 7efd5e000640 1 -- 172.21.15.133:0/3917464237 wait complete. 2024-03-07T06:50:50.416 INFO:tasks.ceph:check_enable_crimson: False 2024-03-07T06:50:50.416 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-07T06:50:50.416 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:50.416 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-07T06:50:50.495 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:50.495 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-07T06:50:50.619 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:50.619 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-07T06:50:50.687 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:50.687+0000 7f63afe14040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-07T06:50:50.742 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:50:50.742 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-07T06:50:50.750 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:50.750+0000 7f63afe14040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-07T06:50:50.800 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:50.801+0000 7f63afe14040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-07T06:50:50.829 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:50.830+0000 7f396b241040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-07T06:50:50.862 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:50.863+0000 7f63afe14040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-07T06:50:50.869 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:50.869 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-07T06:50:50.896 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:50.895+0000 7f396b241040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-07T06:50:50.914 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:50.915+0000 7f63afe14040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-07T06:50:50.955 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:50.955 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-07T06:50:51.051 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:51.052+0000 7f63afe14040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-07T06:50:51.078 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:51.078 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-07T06:50:51.135 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:51.136+0000 7f63afe14040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-07T06:50:51.192 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:51.192+0000 7f63afe14040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-07T06:50:51.209 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:50:51.209 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-07T06:50:51.257 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:51.256+0000 7f63afe14040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-07T06:50:51.314 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:51.315+0000 7f63afe14040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-07T06:50:51.335 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b 0 2024-03-07T06:50:51.487 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:51.487+0000 7f396b241040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-07T06:50:51.551 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:51.551+0000 7f396b241040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-07T06:50:51.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.570+0000 7fbd13200640 1 Processor -- start 2024-03-07T06:50:51.570 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.570+0000 7fbd13200640 1 -- start start 2024-03-07T06:50:51.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.572+0000 7fbd13200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbd140cf1a0 0x7fbd140d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:51.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.572+0000 7fbd13200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbd140b9680 0x7fbd140d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:51.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.572+0000 7fbd13200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd140b9bd0 con 0x7fbd1415def0 2024-03-07T06:50:51.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.572+0000 7fbd13200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd14070e10 con 0x7fbd140b9680 2024-03-07T06:50:51.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.572+0000 7fbd13200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd14071060 con 0x7fbd140cf1a0 2024-03-07T06:50:51.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.572+0000 7fbd11400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbd140cf1a0 0x7fbd140d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:51.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.572+0000 7fbd11400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbd140cf1a0 0x7fbd140d1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55364/0 (socket says 172.21.15.161:55364) 2024-03-07T06:50:51.572 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.572+0000 7fbd11400640 1 -- 172.21.15.161:0/1829554434 learned_addr learned my addr 172.21.15.161:0/1829554434 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-07T06:50:51.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.572+0000 7fbd12800640 1 --2- 172.21.15.161:0/1829554434 >> v2:172.21.15.133:3300/0 conn(0x7fbd140b9680 0x7fbd140d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:51.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.572+0000 7fbd11400640 1 -- 172.21.15.161:0/1829554434 >> v2:172.21.15.133:3300/0 conn(0x7fbd140b9680 msgr2=0x7fbd140d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:50:51.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.572+0000 7fbd11400640 1 --2- 172.21.15.161:0/1829554434 >> v2:172.21.15.133:3300/0 conn(0x7fbd140b9680 0x7fbd140d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.572+0000 7fbd11400640 1 -- 172.21.15.161:0/1829554434 >> v1:172.21.15.133:6789/0 conn(0x7fbd1415def0 legacy=0x7fbd140cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:50:51.573 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.572+0000 7fbd11400640 1 -- 172.21.15.161:0/1829554434 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd14071340 con 0x7fbd140cf1a0 2024-03-07T06:50:51.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.573+0000 7fbd11400640 1 --2- 172.21.15.161:0/1829554434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd140cf1a0 0x7fbd140d1ed0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbd14070f90 tx=0x7fbd0002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfc8b9df47267ebb server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:51.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.574+0000 7fbd10a00640 1 -- 172.21.15.161:0/1829554434 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd00035020 con 0x7fbd140cf1a0 2024-03-07T06:50:51.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.574+0000 7fbd10a00640 1 -- 172.21.15.161:0/1829554434 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd00031870 con 0x7fbd140cf1a0 2024-03-07T06:50:51.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.574+0000 7fbd10a00640 1 -- 172.21.15.161:0/1829554434 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd00031c40 con 0x7fbd140cf1a0 2024-03-07T06:50:51.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.574+0000 7fbd13200640 1 -- 172.21.15.161:0/1829554434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd140cf1a0 msgr2=0x7fbd140d1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:51.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.574+0000 7fbd13200640 1 --2- 172.21.15.161:0/1829554434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd140cf1a0 0x7fbd140d1ed0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbd14070f90 tx=0x7fbd0002b910 comp rx=0 tx=0).stop 2024-03-07T06:50:51.574 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.575+0000 7fbd13200640 1 -- 172.21.15.161:0/1829554434 shutdown_connections 2024-03-07T06:50:51.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.575+0000 7fbd13200640 1 --2- 172.21.15.161:0/1829554434 >> v2:172.21.15.133:3300/0 conn(0x7fbd140b9680 0x7fbd140d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.575+0000 7fbd13200640 1 --2- 172.21.15.161:0/1829554434 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd140cf1a0 0x7fbd140d1ed0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.575+0000 7fbd13200640 1 -- 172.21.15.161:0/1829554434 >> 172.21.15.161:0/1829554434 conn(0x7fbd140d0880 msgr2=0x7fbd140d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:51.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.575+0000 7fbd13200640 1 -- 172.21.15.161:0/1829554434 shutdown_connections 2024-03-07T06:50:51.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.575+0000 7fbd13200640 1 -- 172.21.15.161:0/1829554434 wait complete. 2024-03-07T06:50:51.575 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.576+0000 7fbd13200640 1 Processor -- start 2024-03-07T06:50:51.576 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.576+0000 7fbd13200640 1 -- start start 2024-03-07T06:50:51.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.577+0000 7fbd13200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd140b9680 0x7fbd1415b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:51.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.577+0000 7fbd13200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd140cf1a0 0x7fbd141f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:51.578 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.577+0000 7fbd13200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd1415def0 0x7fbd140b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.577+0000 7fbd13200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1415e2d0 con 0x7fbd140b9680 2024-03-07T06:50:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.577+0000 7fbd13200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd140b3230 con 0x7fbd1415def0 2024-03-07T06:50:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.577+0000 7fbd13200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1415e450 con 0x7fbd140cf1a0 2024-03-07T06:50:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.577+0000 7fbd12800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd1415def0 0x7fbd140b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.577+0000 7fbd11400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd140cf1a0 0x7fbd141f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.577+0000 7fbd12800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd1415def0 0x7fbd140b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55370/0 (socket says 172.21.15.161:55370) 2024-03-07T06:50:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.577+0000 7fbd12800640 1 -- 172.21.15.161:0/2286449488 learned_addr learned my addr 172.21.15.161:0/2286449488 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-07T06:50:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.578+0000 7fbd11e00640 1 --2- 172.21.15.161:0/2286449488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd140b9680 0x7fbd1415b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.578+0000 7fbd12800640 1 -- 172.21.15.161:0/2286449488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd140cf1a0 msgr2=0x7fbd141f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.578+0000 7fbd12800640 1 --2- 172.21.15.161:0/2286449488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd140cf1a0 0x7fbd141f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.578+0000 7fbd12800640 1 -- 172.21.15.161:0/2286449488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd140b9680 msgr2=0x7fbd1415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.578+0000 7fbd12800640 1 --2- 172.21.15.161:0/2286449488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd140b9680 0x7fbd1415b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.578+0000 7fbd12800640 1 -- 172.21.15.161:0/2286449488 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1415bae0 con 0x7fbd1415def0 2024-03-07T06:50:51.579 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.578+0000 7fbd11400640 1 --2- 172.21.15.161:0/2286449488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd140cf1a0 0x7fbd141f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T06:50:51.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.578+0000 7fbd12800640 1 --2- 172.21.15.161:0/2286449488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd1415def0 0x7fbd140b4570 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00efc0 tx=0x7fbd0c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:51.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.578+0000 7fbd0a000640 1 -- 172.21.15.161:0/2286449488 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c018020 con 0x7fbd1415def0 2024-03-07T06:50:51.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.578+0000 7fbd0a000640 1 -- 172.21.15.161:0/2286449488 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0c014480 con 0x7fbd1415def0 2024-03-07T06:50:51.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.579+0000 7fbd0a000640 1 -- 172.21.15.161:0/2286449488 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c014870 con 0x7fbd1415def0 2024-03-07T06:50:51.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.579+0000 7fbd13200640 1 -- 172.21.15.161:0/2286449488 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd140b37b0 con 0x7fbd1415def0 2024-03-07T06:50:51.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.579+0000 7fbd13200640 1 -- 172.21.15.161:0/2286449488 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd140b3d70 con 0x7fbd1415def0 2024-03-07T06:50:51.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.579+0000 7fbd0a000640 1 -- 172.21.15.161:0/2286449488 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbd0c005d10 con 0x7fbd1415def0 2024-03-07T06:50:51.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.579+0000 7fbd0a000640 1 -- 172.21.15.161:0/2286449488 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbd0c0133e0 con 0x7fbd1415def0 2024-03-07T06:50:51.580 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.580+0000 7fbd13200640 1 -- 172.21.15.161:0/2286449488 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd14062c90 con 0x7fbd1415def0 2024-03-07T06:50:51.582 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.583+0000 7fbd0a000640 1 -- 172.21.15.161:0/2286449488 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbd0c022030 con 0x7fbd1415def0 2024-03-07T06:50:51.602 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:51.603+0000 7f396b241040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-07T06:50:51.662 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.662+0000 7fbd13200640 1 -- 172.21.15.161:0/2286449488 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b", "id": 0} v 0) v1 -- 0x7fbd140d0cf0 con 0x7fbd1415def0 2024-03-07T06:50:51.665 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:51.665+0000 7f396b241040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-07T06:50:51.669 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.669+0000 7fbd0a000640 1 -- 172.21.15.161:0/2286449488 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbd0c010040 con 0x7fbd1415def0 2024-03-07T06:50:51.669 INFO:teuthology.orchestra.run.smithi161.stdout:0 2024-03-07T06:50:51.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.671+0000 7fbd13200640 1 -- 172.21.15.161:0/2286449488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd1415def0 msgr2=0x7fbd140b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:51.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.671+0000 7fbd13200640 1 --2- 172.21.15.161:0/2286449488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd1415def0 0x7fbd140b4570 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00efc0 tx=0x7fbd0c00c490 comp rx=0 tx=0).stop 2024-03-07T06:50:51.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.671+0000 7fbd13200640 1 -- 172.21.15.161:0/2286449488 shutdown_connections 2024-03-07T06:50:51.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.671+0000 7fbd13200640 1 --2- 172.21.15.161:0/2286449488 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd1415def0 0x7fbd140b4570 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.671+0000 7fbd13200640 1 --2- 172.21.15.161:0/2286449488 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd140cf1a0 0x7fbd141f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.671+0000 7fbd13200640 1 --2- 172.21.15.161:0/2286449488 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd140b9680 0x7fbd1415b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.671 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.671+0000 7fbd13200640 1 -- 172.21.15.161:0/2286449488 >> 172.21.15.161:0/2286449488 conn(0x7fbd140d0880 msgr2=0x7fbd1415d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:51.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.672+0000 7fbd13200640 1 -- 172.21.15.161:0/2286449488 shutdown_connections 2024-03-07T06:50:51.672 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.672+0000 7fbd13200640 1 -- 172.21.15.161:0/2286449488 wait complete. 2024-03-07T06:50:51.688 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d 1 2024-03-07T06:50:51.715 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:51.715+0000 7f396b241040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-07T06:50:51.789 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:51.789+0000 7f63afe14040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-07T06:50:51.845 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:51.845+0000 7f396b241040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-07T06:50:51.877 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.878+0000 7fb682a00640 1 Processor -- start 2024-03-07T06:50:51.878 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.878+0000 7fb682a00640 1 -- start start 2024-03-07T06:50:51.879 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.880+0000 7fb682a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb67c144d20 0x7fb67c0b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:51.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.880+0000 7fb682a00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c0b46e0 con 0x7fb67c0cd9c0 2024-03-07T06:50:51.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.880+0000 7fb682a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c0b49c0 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.880+0000 7fb682a00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67c071b30 con 0x7fb67c144d20 2024-03-07T06:50:51.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.880+0000 7fb681600640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb67c0cd5e0 0x7fb67c144610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53716/0 (socket says 172.21.15.161:53716) 2024-03-07T06:50:51.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.880+0000 7fb681600640 1 -- 172.21.15.161:0/2393648592 learned_addr learned my addr 172.21.15.161:0/2393648592 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-03-07T06:50:51.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.880+0000 7fb680c00640 1 --2- 172.21.15.161:0/2393648592 >> v2:172.21.15.133:3300/0 conn(0x7fb67c144d20 0x7fb67c0b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:51.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.880+0000 7fb67be00640 1 -- 172.21.15.161:0/2393648592 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550003699 0 0) 0x7fb67c0b49c0 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.880 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.880+0000 7fb67be00640 1 -- 172.21.15.161:0/2393648592 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb664003520 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.881+0000 7fb680c00640 1 -- 172.21.15.161:0/2393648592 >> v1:172.21.15.161:6789/0 conn(0x7fb67c0cd5e0 legacy=0x7fb67c144610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:51.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.881+0000 7fb67be00640 1 -- 172.21.15.161:0/2393648592 <== mon.1 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1966143192 0 0) 0x7fb664003520 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.881+0000 7fb680c00640 1 -- 172.21.15.161:0/2393648592 >> v1:172.21.15.133:6789/0 conn(0x7fb67c0cd9c0 legacy=0x7fb67c0722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:51.881 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.881+0000 7fb680c00640 1 -- 172.21.15.161:0/2393648592 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c0b4ca0 con 0x7fb67c144d20 2024-03-07T06:50:51.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.882+0000 7fb680c00640 1 --2- 172.21.15.161:0/2393648592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb67c144d20 0x7fb67c0b4050 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb67c071cb0 tx=0x7fb66802b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8097fe38730873a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:51.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.882+0000 7fb67be00640 1 -- 172.21.15.161:0/2393648592 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668038070 con 0x7fb67c144d20 2024-03-07T06:50:51.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.882+0000 7fb67be00640 1 -- 172.21.15.161:0/2393648592 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb668031510 con 0x7fb67c144d20 2024-03-07T06:50:51.882 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.882+0000 7fb67be00640 1 -- 172.21.15.161:0/2393648592 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670001670 con 0x7fb67c144d20 2024-03-07T06:50:51.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.883+0000 7fb682a00640 1 -- 172.21.15.161:0/2393648592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb67c144d20 msgr2=0x7fb67c0b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:51.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.883+0000 7fb682a00640 1 --2- 172.21.15.161:0/2393648592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb67c144d20 0x7fb67c0b4050 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb67c071cb0 tx=0x7fb66802b7b0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.883+0000 7fb682a00640 1 -- 172.21.15.161:0/2393648592 shutdown_connections 2024-03-07T06:50:51.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.883+0000 7fb682a00640 1 --2- 172.21.15.161:0/2393648592 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb67c144d20 0x7fb67c0b4050 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.883+0000 7fb682a00640 1 -- 172.21.15.161:0/2393648592 >> 172.21.15.161:0/2393648592 conn(0x7fb67c0b1d90 msgr2=0x7fb67c0b21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:51.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.883+0000 7fb682a00640 1 -- 172.21.15.161:0/2393648592 shutdown_connections 2024-03-07T06:50:51.883 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.883+0000 7fb682a00640 1 -- 172.21.15.161:0/2393648592 wait complete. 2024-03-07T06:50:51.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.885+0000 7fb682a00640 1 Processor -- start 2024-03-07T06:50:51.884 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.885+0000 7fb682a00640 1 -- start start 2024-03-07T06:50:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.886+0000 7fb682a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb67c0cd5e0 0x7fb67c13ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.886+0000 7fb682a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb67c0cd9c0 0x7fb67c1399b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.886+0000 7fb682a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb67c144d20 0x7fb67c139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.886+0000 7fb682a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0cf620 con 0x7fb67c0cd9c0 2024-03-07T06:50:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.886+0000 7fb682a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0cf4a0 con 0x7fb67c144d20 2024-03-07T06:50:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.886+0000 7fb682a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0cf7a0 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.886 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.886+0000 7fb681600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb67c0cd5e0 0x7fb67c13ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.886+0000 7fb681600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb67c0cd5e0 0x7fb67c13ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:55938/0 (socket says 172.21.15.161:55938) 2024-03-07T06:50:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.886+0000 7fb681600640 1 -- 172.21.15.161:0/575931698 learned_addr learned my addr 172.21.15.161:0/575931698 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-07T06:50:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.886+0000 7fb680c00640 1 --2- 172.21.15.161:0/575931698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb67c0cd9c0 0x7fb67c1399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.886+0000 7fb682000640 1 --2- 172.21.15.161:0/575931698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb67c144d20 0x7fb67c139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.886+0000 7fb681600640 1 -- 172.21.15.161:0/575931698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb67c144d20 msgr2=0x7fb67c139ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.886+0000 7fb681600640 1 --2- 172.21.15.161:0/575931698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb67c144d20 0x7fb67c139ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.886+0000 7fb681600640 1 -- 172.21.15.161:0/575931698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb67c0cd9c0 msgr2=0x7fb67c1399b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.887+0000 7fb681600640 1 --2- 172.21.15.161:0/575931698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb67c0cd9c0 0x7fb67c1399b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.887+0000 7fb681600640 1 -- 172.21.15.161:0/575931698 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c13c3e0 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.887+0000 7fb682000640 1 --2- 172.21.15.161:0/575931698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb67c144d20 0x7fb67c139ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:50:51.887 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.887+0000 7fb681600640 1 --2- 172.21.15.161:0/575931698 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb67c0cd5e0 0x7fb67c13ba50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb67000c780 tx=0x7fb67000cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:51.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.887+0000 7fb679600640 1 -- 172.21.15.161:0/575931698 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670018070 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.887+0000 7fb679600640 1 -- 172.21.15.161:0/575931698 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6700072d0 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.888 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.887+0000 7fb682a00640 1 -- 172.21.15.161:0/575931698 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c13c6c0 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.887+0000 7fb679600640 1 -- 172.21.15.161:0/575931698 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6700047e0 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.887+0000 7fb682a00640 1 -- 172.21.15.161:0/575931698 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c13cc80 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.887+0000 7fb679600640 1 -- 172.21.15.161:0/575931698 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb67001b070 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.887+0000 7fb679600640 1 -- 172.21.15.161:0/575931698 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fb67001d440 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.889 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.888+0000 7fb682a00640 1 -- 172.21.15.161:0/575931698 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67c067c80 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.891+0000 7fb679600640 1 -- 172.21.15.161:0/575931698 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb67000a640 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.926 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:51.926+0000 7f396b241040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-07T06:50:51.957 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:51.958+0000 7f63afe14040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-07T06:50:51.970 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.971+0000 7fb682a00640 1 -- 172.21.15.161:0/575931698 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d", "id": 1} v 0) v1 -- 0x7fb67c062c90 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.977 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.978+0000 7fb679600640 1 -- 172.21.15.161:0/575931698 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb670004980 con 0x7fb67c0cd5e0 2024-03-07T06:50:51.977 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-03-07T06:50:51.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.979+0000 7fb682a00640 1 -- 172.21.15.161:0/575931698 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb67c0cd5e0 msgr2=0x7fb67c13ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:51.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.979+0000 7fb682a00640 1 --2- 172.21.15.161:0/575931698 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb67c0cd5e0 0x7fb67c13ba50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb67000c780 tx=0x7fb67000cc40 comp rx=0 tx=0).stop 2024-03-07T06:50:51.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.979+0000 7fb682a00640 1 -- 172.21.15.161:0/575931698 shutdown_connections 2024-03-07T06:50:51.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.979+0000 7fb682a00640 1 --2- 172.21.15.161:0/575931698 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb67c144d20 0x7fb67c139ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.979+0000 7fb682a00640 1 --2- 172.21.15.161:0/575931698 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb67c0cd9c0 0x7fb67c1399b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.979+0000 7fb682a00640 1 --2- 172.21.15.161:0/575931698 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb67c0cd5e0 0x7fb67c13ba50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:51.979 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.979+0000 7fb682a00640 1 -- 172.21.15.161:0/575931698 >> 172.21.15.161:0/575931698 conn(0x7fb67c0b1d90 msgr2=0x7fb67c0728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:51.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.979+0000 7fb682a00640 1 -- 172.21.15.161:0/575931698 shutdown_connections 2024-03-07T06:50:51.980 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:51.979+0000 7fb682a00640 1 -- 172.21.15.161:0/575931698 wait complete. 2024-03-07T06:50:51.981 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:51.981+0000 7f396b241040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-07T06:50:51.995 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new bf595492-1c7c-41bd-a9fa-a3d855305317 2 2024-03-07T06:50:52.042 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:52.042+0000 7f396b241040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-07T06:50:52.096 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:52.097+0000 7f396b241040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-07T06:50:52.100 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:52.100+0000 7f63afe14040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-07T06:50:52.156 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:52.156+0000 7f63afe14040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-07T06:50:52.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.187+0000 7f396f200640 1 Processor -- start 2024-03-07T06:50:52.187 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.187+0000 7f396f200640 1 -- start start 2024-03-07T06:50:52.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.188+0000 7f396f200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39701583c0 0x7f397015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.188+0000 7f396f200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f39700cefd0 0x7f3970071360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.189 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.188+0000 7f396f200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39700705a0 con 0x7f397015ee60 2024-03-07T06:50:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.188+0000 7f396f200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f397015f3e0 con 0x7f39700cefd0 2024-03-07T06:50:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.188+0000 7f396f200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3970070ed0 con 0x7f39701583c0 2024-03-07T06:50:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.189+0000 7f396de00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39701583c0 0x7f397015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.189+0000 7f396e800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f39700cefd0 0x7f3970071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.189+0000 7f396de00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39701583c0 0x7f397015f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:55954/0 (socket says 172.21.15.161:55954) 2024-03-07T06:50:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.189+0000 7f396de00640 1 -- 172.21.15.161:0/2382011100 learned_addr learned my addr 172.21.15.161:0/2382011100 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-07T06:50:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.189+0000 7f396de00640 1 -- 172.21.15.161:0/2382011100 >> v2:172.21.15.133:3300/0 conn(0x7f39700cefd0 msgr2=0x7f3970071360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:50:52.190 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.189+0000 7f396de00640 1 --2- 172.21.15.161:0/2382011100 >> v2:172.21.15.133:3300/0 conn(0x7f39700cefd0 0x7f3970071360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.189+0000 7f396de00640 1 -- 172.21.15.161:0/2382011100 >> v1:172.21.15.161:6789/0 conn(0x7f397015ee60 legacy=0x7f39700ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.189+0000 7f396de00640 1 -- 172.21.15.161:0/2382011100 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3970070880 con 0x7f39701583c0 2024-03-07T06:50:52.191 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.189+0000 7f396d400640 1 -- 172.21.15.161:0/2382011100 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12788966 0 0) 0x7f39700705a0 con 0x7f397015ee60 2024-03-07T06:50:52.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.190+0000 7f396de00640 1 --2- 172.21.15.161:0/2382011100 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39701583c0 0x7f397015f5a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3960009870 tx=0x7f396002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9c6cba68db07117 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:52.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.191+0000 7f396d400640 1 -- 172.21.15.161:0/2382011100 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960034070 con 0x7f39701583c0 2024-03-07T06:50:52.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.191+0000 7f396d400640 1 -- 172.21.15.161:0/2382011100 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3960030840 con 0x7f39701583c0 2024-03-07T06:50:52.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.191+0000 7f396d400640 1 -- 172.21.15.161:0/2382011100 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960030c10 con 0x7f39701583c0 2024-03-07T06:50:52.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.191+0000 7f396f200640 1 -- 172.21.15.161:0/2382011100 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39701583c0 msgr2=0x7f397015f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.197 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.191+0000 7f396f200640 1 --2- 172.21.15.161:0/2382011100 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39701583c0 0x7f397015f5a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3960009870 tx=0x7f396002b980 comp rx=0 tx=0).stop 2024-03-07T06:50:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.191+0000 7f396f200640 1 -- 172.21.15.161:0/2382011100 shutdown_connections 2024-03-07T06:50:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.191+0000 7f396f200640 1 --2- 172.21.15.161:0/2382011100 >> v2:172.21.15.133:3300/0 conn(0x7f39700cefd0 0x7f3970071360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.191+0000 7f396f200640 1 --2- 172.21.15.161:0/2382011100 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39701583c0 0x7f397015f5a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.191+0000 7f396f200640 1 -- 172.21.15.161:0/2382011100 >> 172.21.15.161:0/2382011100 conn(0x7f3970072080 msgr2=0x7f3970072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.191+0000 7f396f200640 1 -- 172.21.15.161:0/2382011100 shutdown_connections 2024-03-07T06:50:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.191+0000 7f396f200640 1 -- 172.21.15.161:0/2382011100 wait complete. 2024-03-07T06:50:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.193+0000 7f396f200640 1 Processor -- start 2024-03-07T06:50:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.193+0000 7f396f200640 1 -- start start 2024-03-07T06:50:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.194+0000 7f396f200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39700cefd0 0x7f397013ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.194+0000 7f396f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39701583c0 0x7f397014f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.198 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.194+0000 7f396f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f397015ee60 0x7f397014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.194+0000 7f396f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39701587a0 con 0x7f39701583c0 2024-03-07T06:50:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.194+0000 7f396f200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39700b2fe0 con 0x7f397015ee60 2024-03-07T06:50:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.194+0000 7f396f200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3970158920 con 0x7f39700cefd0 2024-03-07T06:50:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.194+0000 7f396de00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39700cefd0 0x7f397013ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.194+0000 7f396de00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39700cefd0 0x7f397013ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:55968/0 (socket says 172.21.15.161:55968) 2024-03-07T06:50:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.194+0000 7f396de00640 1 -- 172.21.15.161:0/855575226 learned_addr learned my addr 172.21.15.161:0/855575226 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-07T06:50:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.194+0000 7f396e800640 1 --2- 172.21.15.161:0/855575226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f397015ee60 0x7f397014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.194+0000 7f3967e00640 1 --2- 172.21.15.161:0/855575226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39701583c0 0x7f397014f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.195+0000 7f396de00640 1 -- 172.21.15.161:0/855575226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f397015ee60 msgr2=0x7f397014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.195+0000 7f396de00640 1 --2- 172.21.15.161:0/855575226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f397015ee60 0x7f397014fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.195+0000 7f396de00640 1 -- 172.21.15.161:0/855575226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39701583c0 msgr2=0x7f397014f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.195+0000 7f396de00640 1 --2- 172.21.15.161:0/855575226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39701583c0 0x7f397014f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.195+0000 7f396de00640 1 -- 172.21.15.161:0/855575226 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397013f220 con 0x7f39700cefd0 2024-03-07T06:50:52.199 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.195+0000 7f396de00640 1 --2- 172.21.15.161:0/855575226 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39700cefd0 0x7f397013ece0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3960000c00 tx=0x7f39600046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.195+0000 7f3966000640 1 -- 172.21.15.161:0/855575226 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960034040 con 0x7f39700cefd0 2024-03-07T06:50:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.195+0000 7f3966000640 1 -- 172.21.15.161:0/855575226 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3960030640 con 0x7f39700cefd0 2024-03-07T06:50:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.195+0000 7f3966000640 1 -- 172.21.15.161:0/855575226 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3960039d40 con 0x7f39700cefd0 2024-03-07T06:50:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.195+0000 7f396f200640 1 -- 172.21.15.161:0/855575226 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f397014d4a0 con 0x7f39700cefd0 2024-03-07T06:50:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.195+0000 7f396f200640 1 -- 172.21.15.161:0/855575226 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f397014b8b0 con 0x7f39700cefd0 2024-03-07T06:50:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.196+0000 7f3966000640 1 -- 172.21.15.161:0/855575226 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f396004b060 con 0x7f39700cefd0 2024-03-07T06:50:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.196+0000 7f3966000640 1 -- 172.21.15.161:0/855575226 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f3960004030 con 0x7f39700cefd0 2024-03-07T06:50:52.200 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.196+0000 7f396f200640 1 -- 172.21.15.161:0/855575226 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3970062c90 con 0x7f39700cefd0 2024-03-07T06:50:52.201 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.201+0000 7f3966000640 1 -- 172.21.15.161:0/855575226 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3960063dd0 con 0x7f39700cefd0 2024-03-07T06:50:52.279 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.279+0000 7f396f200640 1 -- 172.21.15.161:0/855575226 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bf595492-1c7c-41bd-a9fa-a3d855305317", "id": 2} v 0) v1 -- 0x7f39700730e0 con 0x7f39700cefd0 2024-03-07T06:50:52.286 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.286+0000 7f3966000640 1 -- 172.21.15.161:0/855575226 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bf595492-1c7c-41bd-a9fa-a3d855305317", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3960052030 con 0x7f39700cefd0 2024-03-07T06:50:52.286 INFO:teuthology.orchestra.run.smithi161.stdout:2 2024-03-07T06:50:52.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.288+0000 7f396f200640 1 -- 172.21.15.161:0/855575226 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39700cefd0 msgr2=0x7f397013ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.288+0000 7f396f200640 1 --2- 172.21.15.161:0/855575226 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39700cefd0 0x7f397013ece0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3960000c00 tx=0x7f39600046c0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.288+0000 7f396f200640 1 -- 172.21.15.161:0/855575226 shutdown_connections 2024-03-07T06:50:52.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.288+0000 7f396f200640 1 --2- 172.21.15.161:0/855575226 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f397015ee60 0x7f397014fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.288+0000 7f396f200640 1 --2- 172.21.15.161:0/855575226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39701583c0 0x7f397014f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.288+0000 7f396f200640 1 --2- 172.21.15.161:0/855575226 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39700cefd0 0x7f397013ece0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.288+0000 7f396f200640 1 -- 172.21.15.161:0/855575226 >> 172.21.15.161:0/855575226 conn(0x7f3970072080 msgr2=0x7f3970071360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:52.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.288+0000 7f396f200640 1 -- 172.21.15.161:0/855575226 shutdown_connections 2024-03-07T06:50:52.288 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.288+0000 7f396f200640 1 -- 172.21.15.161:0/855575226 wait complete. 2024-03-07T06:50:52.293 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:52.294+0000 7f63afe14040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-07T06:50:52.304 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 3af6d127-f3e2-4465-b13e-914b7c50fb33 3 2024-03-07T06:50:52.415 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:52.415+0000 7f63afe14040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-07T06:50:52.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.495+0000 7f03ef000640 1 Processor -- start 2024-03-07T06:50:52.495 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.495+0000 7f03ef000640 1 -- start start 2024-03-07T06:50:52.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.497+0000 7f03ef000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f03e80b9680 0x7f03e80d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.497+0000 7f03ef000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e80b9bd0 con 0x7f03e815def0 2024-03-07T06:50:52.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.497+0000 7f03ef000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e8071060 con 0x7f03e80cf1a0 2024-03-07T06:50:52.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.497+0000 7f03ef000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e8070e10 con 0x7f03e80b9680 2024-03-07T06:50:52.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.497+0000 7f03ed200640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f03e80cf1a0 0x7f03e80d1ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53734/0 (socket says 172.21.15.161:53734) 2024-03-07T06:50:52.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.497+0000 7f03ed200640 1 -- 172.21.15.161:0/1731280499 learned_addr learned my addr 172.21.15.161:0/1731280499 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-03-07T06:50:52.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.497+0000 7f03ee600640 1 --2- 172.21.15.161:0/1731280499 >> v2:172.21.15.133:3300/0 conn(0x7f03e80b9680 0x7f03e80d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:52.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.498+0000 7f03e7e00640 1 -- 172.21.15.161:0/1731280499 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314118629 0 0) 0x7f03e8071060 con 0x7f03e80cf1a0 2024-03-07T06:50:52.497 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.498+0000 7f03e7e00640 1 -- 172.21.15.161:0/1731280499 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03d0003520 con 0x7f03e80cf1a0 2024-03-07T06:50:52.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.498+0000 7f03ee600640 1 -- 172.21.15.161:0/1731280499 >> v1:172.21.15.161:6789/0 conn(0x7f03e80cf1a0 legacy=0x7f03e80d1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.498+0000 7f03ee600640 1 -- 172.21.15.161:0/1731280499 >> v1:172.21.15.133:6789/0 conn(0x7f03e815def0 legacy=0x7f03e80cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.498 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.498+0000 7f03ee600640 1 -- 172.21.15.161:0/1731280499 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e8071340 con 0x7f03e80b9680 2024-03-07T06:50:52.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.499+0000 7f03ee600640 1 --2- 172.21.15.161:0/1731280499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e80b9680 0x7f03e80d25e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009d90 tx=0x7f03e002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c91f2b15186bb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:52.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.499+0000 7f03e7e00640 1 -- 172.21.15.161:0/1731280499 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0039070 con 0x7f03e80b9680 2024-03-07T06:50:52.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.499+0000 7f03e7e00640 1 -- 172.21.15.161:0/1731280499 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e002ccf0 con 0x7f03e80b9680 2024-03-07T06:50:52.499 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.499+0000 7f03e7e00640 1 -- 172.21.15.161:0/1731280499 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0034640 con 0x7f03e80b9680 2024-03-07T06:50:52.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.500+0000 7f03ef000640 1 -- 172.21.15.161:0/1731280499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e80b9680 msgr2=0x7f03e80d25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.500+0000 7f03ef000640 1 --2- 172.21.15.161:0/1731280499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e80b9680 0x7f03e80d25e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009d90 tx=0x7f03e002bf40 comp rx=0 tx=0).stop 2024-03-07T06:50:52.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.500+0000 7f03ef000640 1 -- 172.21.15.161:0/1731280499 shutdown_connections 2024-03-07T06:50:52.500 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.500+0000 7f03ef000640 1 --2- 172.21.15.161:0/1731280499 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e80b9680 0x7f03e80d25e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.500+0000 7f03ef000640 1 -- 172.21.15.161:0/1731280499 >> 172.21.15.161:0/1731280499 conn(0x7f03e80d0880 msgr2=0x7f03e80d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:52.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.500+0000 7f03ef000640 1 -- 172.21.15.161:0/1731280499 shutdown_connections 2024-03-07T06:50:52.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.500+0000 7f03ef000640 1 -- 172.21.15.161:0/1731280499 wait complete. 2024-03-07T06:50:52.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.502+0000 7f03ef000640 1 Processor -- start 2024-03-07T06:50:52.501 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.502+0000 7f03ef000640 1 -- start start 2024-03-07T06:50:52.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.503+0000 7f03ef000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e80b9680 0x7f03e815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.503+0000 7f03ef000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03e80cf1a0 0x7f03e80cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.503+0000 7f03ef000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e815def0 0x7f03e815ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.503 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.503+0000 7f03ef000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e815e2d0 con 0x7f03e815def0 2024-03-07T06:50:52.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.503+0000 7f03ef000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e80b3230 con 0x7f03e80b9680 2024-03-07T06:50:52.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.503+0000 7f03ef000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e815e450 con 0x7f03e80cf1a0 2024-03-07T06:50:52.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.503+0000 7f03edc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e80b9680 0x7f03e815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:52.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.503+0000 7f03ed200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03e80cf1a0 0x7f03e80cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:52.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.503+0000 7f03edc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e80b9680 0x7f03e815bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55386/0 (socket says 172.21.15.161:55386) 2024-03-07T06:50:52.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.503+0000 7f03edc00640 1 -- 172.21.15.161:0/2077981740 learned_addr learned my addr 172.21.15.161:0/2077981740 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-07T06:50:52.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.504+0000 7f03edc00640 1 -- 172.21.15.161:0/2077981740 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03e80cf1a0 msgr2=0x7f03e80cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.504+0000 7f03edc00640 1 --2- 172.21.15.161:0/2077981740 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03e80cf1a0 0x7f03e80cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.504+0000 7f03edc00640 1 -- 172.21.15.161:0/2077981740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e815def0 msgr2=0x7f03e815ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.504+0000 7f03edc00640 1 --2- 172.21.15.161:0/2077981740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e815def0 0x7f03e815ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.504+0000 7f03edc00640 1 -- 172.21.15.161:0/2077981740 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e80cd590 con 0x7f03e80b9680 2024-03-07T06:50:52.504 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.504+0000 7f03edc00640 1 --2- 172.21.15.161:0/2077981740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e80b9680 0x7f03e815bbb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00c400 tx=0x7f03dc00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:52.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.504+0000 7f03e5600640 1 -- 172.21.15.161:0/2077981740 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc017070 con 0x7f03e80b9680 2024-03-07T06:50:52.505 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.504+0000 7f03e5600640 1 -- 172.21.15.161:0/2077981740 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc0052d0 con 0x7f03e80b9680 2024-03-07T06:50:52.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.504+0000 7f03e5600640 1 -- 172.21.15.161:0/2077981740 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc003830 con 0x7f03e80b9680 2024-03-07T06:50:52.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.504+0000 7f03ef000640 1 -- 172.21.15.161:0/2077981740 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e80cd870 con 0x7f03e80b9680 2024-03-07T06:50:52.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.504+0000 7f03ef000640 1 -- 172.21.15.161:0/2077981740 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e80cde30 con 0x7f03e80b9680 2024-03-07T06:50:52.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.505+0000 7f03e5600640 1 -- 172.21.15.161:0/2077981740 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f03dc01a070 con 0x7f03e80b9680 2024-03-07T06:50:52.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.505+0000 7f03e5600640 1 -- 172.21.15.161:0/2077981740 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f03dc011590 con 0x7f03e80b9680 2024-03-07T06:50:52.506 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.505+0000 7f03ef000640 1 -- 172.21.15.161:0/2077981740 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e8062c90 con 0x7f03e80b9680 2024-03-07T06:50:52.507 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.508+0000 7f03e5600640 1 -- 172.21.15.161:0/2077981740 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f03dc021070 con 0x7f03e80b9680 2024-03-07T06:50:52.554 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:52.554+0000 7f396b241040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-07T06:50:52.586 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.586+0000 7f03ef000640 1 -- 172.21.15.161:0/2077981740 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3af6d127-f3e2-4465-b13e-914b7c50fb33", "id": 3} v 0) v1 -- 0x7f03e80d1a80 con 0x7f03e80b9680 2024-03-07T06:50:52.592 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.593+0000 7f03e5600640 1 -- 172.21.15.161:0/2077981740 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3af6d127-f3e2-4465-b13e-914b7c50fb33", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f03dc011b80 con 0x7f03e80b9680 2024-03-07T06:50:52.593 INFO:teuthology.orchestra.run.smithi161.stdout:3 2024-03-07T06:50:52.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.594+0000 7f03ef000640 1 -- 172.21.15.161:0/2077981740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e80b9680 msgr2=0x7f03e815bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.594+0000 7f03ef000640 1 --2- 172.21.15.161:0/2077981740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e80b9680 0x7f03e815bbb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f03dc00c400 tx=0x7f03dc00c8c0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.594 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.594+0000 7f03ef000640 1 -- 172.21.15.161:0/2077981740 shutdown_connections 2024-03-07T06:50:52.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.594+0000 7f03ef000640 1 --2- 172.21.15.161:0/2077981740 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03e815def0 0x7f03e815ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.594+0000 7f03ef000640 1 --2- 172.21.15.161:0/2077981740 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f03e80cf1a0 0x7f03e80cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.594+0000 7f03ef000640 1 --2- 172.21.15.161:0/2077981740 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03e80b9680 0x7f03e815bbb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.594+0000 7f03ef000640 1 -- 172.21.15.161:0/2077981740 >> 172.21.15.161:0/2077981740 conn(0x7f03e80d0880 msgr2=0x7f03e80c9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:52.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.595+0000 7f03ef000640 1 -- 172.21.15.161:0/2077981740 shutdown_connections 2024-03-07T06:50:52.595 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.595+0000 7f03ef000640 1 -- 172.21.15.161:0/2077981740 wait complete. 2024-03-07T06:50:52.598 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:52.599+0000 7f63afe14040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-07T06:50:52.611 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 163f8723-31be-4a08-bd17-c9d68108167f 4 2024-03-07T06:50:52.652 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:52.652+0000 7f63afe14040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-07T06:50:52.714 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:52.715+0000 7f396b241040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-07T06:50:52.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.799+0000 7fcdb3200640 1 Processor -- start 2024-03-07T06:50:52.799 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.799+0000 7fcdb3200640 1 -- start start 2024-03-07T06:50:52.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.801+0000 7fcdb3200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb4147720 0x7fcdb40b5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.801+0000 7fcdb3200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcdb4071550 0x7fcdb415ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.801+0000 7fcdb3200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcdb415f230 0x7fcdb4161b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.801+0000 7fcdb3200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb415e550 con 0x7fcdb415f230 2024-03-07T06:50:52.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.801+0000 7fcdb3200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb4071d10 con 0x7fcdb4071550 2024-03-07T06:50:52.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.801+0000 7fcdb3200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb4071ff0 con 0x7fcdb4147720 2024-03-07T06:50:52.801 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.801+0000 7fcdb1400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcdb4071550 0x7fcdb415ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:52.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.801+0000 7fcdb1e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb4147720 0x7fcdb40b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:52.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.801+0000 7fcdb1400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcdb4071550 0x7fcdb415ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55400/0 (socket says 172.21.15.161:55400) 2024-03-07T06:50:52.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.801+0000 7fcdb1400640 1 -- 172.21.15.161:0/483986695 learned_addr learned my addr 172.21.15.161:0/483986695 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-07T06:50:52.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.801+0000 7fcdb2800640 1 --2- 172.21.15.161:0/483986695 >> v2:172.21.15.133:3300/0 conn(0x7fcdb415f230 0x7fcdb4161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:52.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.802+0000 7fcdb1400640 1 -- 172.21.15.161:0/483986695 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb4147720 msgr2=0x7fcdb40b5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:50:52.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.802+0000 7fcdb1400640 1 --2- 172.21.15.161:0/483986695 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb4147720 0x7fcdb40b5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.802+0000 7fcdb1400640 1 -- 172.21.15.161:0/483986695 >> v2:172.21.15.133:3300/0 conn(0x7fcdb415f230 msgr2=0x7fcdb4161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:50:52.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.802+0000 7fcdb1400640 1 --2- 172.21.15.161:0/483986695 >> v2:172.21.15.133:3300/0 conn(0x7fcdb415f230 0x7fcdb4161b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.802 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.802+0000 7fcdb1400640 1 -- 172.21.15.161:0/483986695 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb40722d0 con 0x7fcdb4071550 2024-03-07T06:50:52.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.803+0000 7fcdb1400640 1 --2- 172.21.15.161:0/483986695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4071550 0x7fcdb415ecf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40736b0 tx=0x7fcd9c02b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1616e36e18ff601b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:52.803 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.803+0000 7fcdb0a00640 1 -- 172.21.15.161:0/483986695 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c02e070 con 0x7fcdb4071550 2024-03-07T06:50:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.803+0000 7fcdb0a00640 1 -- 172.21.15.161:0/483986695 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9c037070 con 0x7fcdb4071550 2024-03-07T06:50:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.803+0000 7fcdb0a00640 1 -- 172.21.15.161:0/483986695 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c031a00 con 0x7fcdb4071550 2024-03-07T06:50:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.804+0000 7fcdb3200640 1 -- 172.21.15.161:0/483986695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4071550 msgr2=0x7fcdb415ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.804+0000 7fcdb3200640 1 --2- 172.21.15.161:0/483986695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4071550 0x7fcdb415ecf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40736b0 tx=0x7fcd9c02b880 comp rx=0 tx=0).stop 2024-03-07T06:50:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.804+0000 7fcdb3200640 1 -- 172.21.15.161:0/483986695 shutdown_connections 2024-03-07T06:50:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.804+0000 7fcdb3200640 1 --2- 172.21.15.161:0/483986695 >> v2:172.21.15.133:3300/0 conn(0x7fcdb415f230 0x7fcdb4161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.804+0000 7fcdb3200640 1 --2- 172.21.15.161:0/483986695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4071550 0x7fcdb415ecf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.804+0000 7fcdb3200640 1 --2- 172.21.15.161:0/483986695 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb4147720 0x7fcdb40b5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.804+0000 7fcdb3200640 1 -- 172.21.15.161:0/483986695 >> 172.21.15.161:0/483986695 conn(0x7fcdb4072a30 msgr2=0x7fcdb4072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.804+0000 7fcdb3200640 1 -- 172.21.15.161:0/483986695 shutdown_connections 2024-03-07T06:50:52.804 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.804+0000 7fcdb3200640 1 -- 172.21.15.161:0/483986695 wait complete. 2024-03-07T06:50:52.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.806+0000 7fcdb3200640 1 Processor -- start 2024-03-07T06:50:52.805 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.806+0000 7fcdb3200640 1 -- start start 2024-03-07T06:50:52.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb3200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4071550 0x7fcdb4131ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb3200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb4147720 0x7fcdb40af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb415f230 0x7fcdb40afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:52.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb3200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb4147b00 con 0x7fcdb415f230 2024-03-07T06:50:52.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb3200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb40719b0 con 0x7fcdb4071550 2024-03-07T06:50:52.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb3200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb4147c80 con 0x7fcdb4147720 2024-03-07T06:50:52.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb1400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb4147720 0x7fcdb40af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:52.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4071550 0x7fcdb4131ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:52.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb1e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4071550 0x7fcdb4131ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55410/0 (socket says 172.21.15.161:55410) 2024-03-07T06:50:52.807 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb1e00640 1 -- 172.21.15.161:0/1842495782 learned_addr learned my addr 172.21.15.161:0/1842495782 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-07T06:50:52.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb1400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb4147720 0x7fcdb40af500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:55994/0 (socket says 172.21.15.161:55994) 2024-03-07T06:50:52.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb2800640 1 --2- 172.21.15.161:0/1842495782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb415f230 0x7fcdb40afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:52.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb1e00640 1 -- 172.21.15.161:0/1842495782 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb4147720 msgr2=0x7fcdb40af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb1e00640 1 --2- 172.21.15.161:0/1842495782 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb4147720 0x7fcdb40af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb1e00640 1 -- 172.21.15.161:0/1842495782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb415f230 msgr2=0x7fcdb40afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb1e00640 1 --2- 172.21.15.161:0/1842495782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb415f230 0x7fcdb40afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.807+0000 7fcdb1e00640 1 -- 172.21.15.161:0/1842495782 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb40ae460 con 0x7fcdb4071550 2024-03-07T06:50:52.808 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.808+0000 7fcdb1e00640 1 --2- 172.21.15.161:0/1842495782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4071550 0x7fcdb4131ce0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00eed0 tx=0x7fcdac00b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:52.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.808+0000 7fcdaa000640 1 -- 172.21.15.161:0/1842495782 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac018020 con 0x7fcdb4071550 2024-03-07T06:50:52.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.808+0000 7fcdaa000640 1 -- 172.21.15.161:0/1842495782 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdac00bd90 con 0x7fcdb4071550 2024-03-07T06:50:52.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.808+0000 7fcdaa000640 1 -- 172.21.15.161:0/1842495782 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac014620 con 0x7fcdb4071550 2024-03-07T06:50:52.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.808+0000 7fcdb3200640 1 -- 172.21.15.161:0/1842495782 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb40ae740 con 0x7fcdb4071550 2024-03-07T06:50:52.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.808+0000 7fcdb3200640 1 -- 172.21.15.161:0/1842495782 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb40aed00 con 0x7fcdb4071550 2024-03-07T06:50:52.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.808+0000 7fcdaa000640 1 -- 172.21.15.161:0/1842495782 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fcdac005d10 con 0x7fcdb4071550 2024-03-07T06:50:52.809 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.808+0000 7fcdaa000640 1 -- 172.21.15.161:0/1842495782 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fcdac01db30 con 0x7fcdb4071550 2024-03-07T06:50:52.810 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.810+0000 7fcdb3200640 1 -- 172.21.15.161:0/1842495782 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb4062c90 con 0x7fcdb4071550 2024-03-07T06:50:52.811 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.812+0000 7fcdaa000640 1 -- 172.21.15.161:0/1842495782 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fcdac022030 con 0x7fcdb4071550 2024-03-07T06:50:52.844 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:52.844+0000 7f63afe14040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-07T06:50:52.869 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:52.870+0000 7f396b241040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-07T06:50:52.890 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.891+0000 7fcdb3200640 1 -- 172.21.15.161:0/1842495782 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "163f8723-31be-4a08-bd17-c9d68108167f", "id": 4} v 0) v1 -- 0x7fcdb4070e20 con 0x7fcdb4071550 2024-03-07T06:50:52.896 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.896+0000 7fcdaa000640 1 -- 172.21.15.161:0/1842495782 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "163f8723-31be-4a08-bd17-c9d68108167f", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcdac01d400 con 0x7fcdb4071550 2024-03-07T06:50:52.896 INFO:teuthology.orchestra.run.smithi161.stdout:4 2024-03-07T06:50:52.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.898+0000 7fcdb3200640 1 -- 172.21.15.161:0/1842495782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4071550 msgr2=0x7fcdb4131ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:52.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.898+0000 7fcdb3200640 1 --2- 172.21.15.161:0/1842495782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4071550 0x7fcdb4131ce0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00eed0 tx=0x7fcdac00b480 comp rx=0 tx=0).stop 2024-03-07T06:50:52.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.898+0000 7fcdb3200640 1 -- 172.21.15.161:0/1842495782 shutdown_connections 2024-03-07T06:50:52.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.898+0000 7fcdb3200640 1 --2- 172.21.15.161:0/1842495782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdb415f230 0x7fcdb40afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.898+0000 7fcdb3200640 1 --2- 172.21.15.161:0/1842495782 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcdb4147720 0x7fcdb40af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.898+0000 7fcdb3200640 1 --2- 172.21.15.161:0/1842495782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcdb4071550 0x7fcdb4131ce0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:52.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.898+0000 7fcdb3200640 1 -- 172.21.15.161:0/1842495782 >> 172.21.15.161:0/1842495782 conn(0x7fcdb4072a30 msgr2=0x7fcdb4072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:52.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.898+0000 7fcdb3200640 1 -- 172.21.15.161:0/1842495782 shutdown_connections 2024-03-07T06:50:52.898 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:52.898+0000 7fcdb3200640 1 -- 172.21.15.161:0/1842495782 wait complete. 2024-03-07T06:50:52.914 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 972747df-dbbf-4ae5-8101-be5d39fde1ed 5 2024-03-07T06:50:52.926 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:52.926+0000 7f396b241040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-07T06:50:53.054 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:53.054+0000 7f396b241040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-07T06:50:53.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.148+0000 7f8d25800640 1 Processor -- start 2024-03-07T06:50:53.148 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.148+0000 7f8d25800640 1 -- start start 2024-03-07T06:50:53.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.149+0000 7f8d25800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8d20071b20 0x7f8d20168e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:53.149 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.149+0000 7f8d25800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d200b8180 con 0x7f8d200d3be0 2024-03-07T06:50:53.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.150+0000 7f8d25800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d200d28b0 con 0x7f8d2015e3e0 2024-03-07T06:50:53.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.150+0000 7f8d25800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2015e7c0 con 0x7f8d20071b20 2024-03-07T06:50:53.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.150+0000 7f8d1ea00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f8d2015e3e0 0x7f8d200d21a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53744/0 (socket says 172.21.15.161:53744) 2024-03-07T06:50:53.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.150+0000 7f8d1ea00640 1 -- 172.21.15.161:0/2097901310 learned_addr learned my addr 172.21.15.161:0/2097901310 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-03-07T06:50:53.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.150+0000 7f8d1f400640 1 --2- 172.21.15.161:0/2097901310 >> v2:172.21.15.133:3300/0 conn(0x7f8d20071b20 0x7f8d20168e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:53.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.150+0000 7f8d1e000640 1 -- 172.21.15.161:0/2097901310 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375360462 0 0) 0x7f8d200d28b0 con 0x7f8d2015e3e0 2024-03-07T06:50:53.150 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.150+0000 7f8d1e000640 1 -- 172.21.15.161:0/2097901310 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d04003520 con 0x7f8d2015e3e0 2024-03-07T06:50:53.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.150+0000 7f8d1e000640 1 -- 172.21.15.161:0/2097901310 <== mon.1 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1626350669 0 0) 0x7f8d04003520 con 0x7f8d2015e3e0 2024-03-07T06:50:53.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.150+0000 7f8d1f400640 1 -- 172.21.15.161:0/2097901310 >> v1:172.21.15.161:6789/0 conn(0x7f8d2015e3e0 legacy=0x7f8d200d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.151+0000 7f8d1f400640 1 -- 172.21.15.161:0/2097901310 >> v1:172.21.15.133:6789/0 conn(0x7f8d200d3be0 legacy=0x7f8d20070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.151+0000 7f8d1f400640 1 -- 172.21.15.161:0/2097901310 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200d2b40 con 0x7f8d20071b20 2024-03-07T06:50:53.151 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.152+0000 7f8d1f400640 1 --2- 172.21.15.161:0/2097901310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d20071b20 0x7f8d20168e80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009d90 tx=0x7f8d1802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4138b310de067dc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:53.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.152+0000 7f8d1e000640 1 -- 172.21.15.161:0/2097901310 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d18039070 con 0x7f8d20071b20 2024-03-07T06:50:53.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.152+0000 7f8d1e000640 1 -- 172.21.15.161:0/2097901310 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1802c970 con 0x7f8d20071b20 2024-03-07T06:50:53.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.152+0000 7f8d1e000640 1 -- 172.21.15.161:0/2097901310 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10001670 con 0x7f8d20071b20 2024-03-07T06:50:53.152 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.153+0000 7f8d25800640 1 -- 172.21.15.161:0/2097901310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d20071b20 msgr2=0x7f8d20168e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.153+0000 7f8d25800640 1 --2- 172.21.15.161:0/2097901310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d20071b20 0x7f8d20168e80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009d90 tx=0x7f8d1802be40 comp rx=0 tx=0).stop 2024-03-07T06:50:53.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.153+0000 7f8d25800640 1 -- 172.21.15.161:0/2097901310 shutdown_connections 2024-03-07T06:50:53.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.153+0000 7f8d25800640 1 --2- 172.21.15.161:0/2097901310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d20071b20 0x7f8d20168e80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.153+0000 7f8d25800640 1 -- 172.21.15.161:0/2097901310 >> 172.21.15.161:0/2097901310 conn(0x7f8d200d0c30 msgr2=0x7f8d200d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:53.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.153+0000 7f8d25800640 1 -- 172.21.15.161:0/2097901310 shutdown_connections 2024-03-07T06:50:53.153 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.153+0000 7f8d25800640 1 -- 172.21.15.161:0/2097901310 wait complete. 2024-03-07T06:50:53.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.155+0000 7f8d25800640 1 Processor -- start 2024-03-07T06:50:53.154 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.155+0000 7f8d25800640 1 -- start start 2024-03-07T06:50:53.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d25800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d20071b20 0x7f8d200b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:53.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d25800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d2015e3e0 0x7f8d2015a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:53.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d25800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d200b8350 0x7f8d200b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:53.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d25800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200d33e0 con 0x7f8d2015e3e0 2024-03-07T06:50:53.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d25800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200d3100 con 0x7f8d20071b20 2024-03-07T06:50:53.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d25800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d200d36c0 con 0x7f8d200b8350 2024-03-07T06:50:53.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d1ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d20071b20 0x7f8d200b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:53.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d1ea00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d20071b20 0x7f8d200b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55426/0 (socket says 172.21.15.161:55426) 2024-03-07T06:50:53.157 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d1ea00640 1 -- 172.21.15.161:0/4170096384 learned_addr learned my addr 172.21.15.161:0/4170096384 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d1f400640 1 --2- 172.21.15.161:0/4170096384 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d200b8350 0x7f8d200b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d17e00640 1 --2- 172.21.15.161:0/4170096384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d2015e3e0 0x7f8d2015a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d1ea00640 1 -- 172.21.15.161:0/4170096384 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d200b8350 msgr2=0x7f8d200b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d1ea00640 1 --2- 172.21.15.161:0/4170096384 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d200b8350 0x7f8d200b87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d1ea00640 1 -- 172.21.15.161:0/4170096384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d2015e3e0 msgr2=0x7f8d2015a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d1ea00640 1 --2- 172.21.15.161:0/4170096384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d2015e3e0 0x7f8d2015a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d1ea00640 1 -- 172.21.15.161:0/4170096384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d200cc150 con 0x7f8d20071b20 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.156+0000 7f8d17e00640 1 --2- 172.21.15.161:0/4170096384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d2015e3e0 0x7f8d2015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.157+0000 7f8d1ea00640 1 --2- 172.21.15.161:0/4170096384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d20071b20 0x7f8d200b3a30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000c560 tx=0x7f8d1000ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.157+0000 7f8d16a00640 1 -- 172.21.15.161:0/4170096384 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10018070 con 0x7f8d20071b20 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.157+0000 7f8d16a00640 1 -- 172.21.15.161:0/4170096384 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d10007500 con 0x7f8d20071b20 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.157+0000 7f8d25800640 1 -- 172.21.15.161:0/4170096384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d200cc430 con 0x7f8d20071b20 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.157+0000 7f8d16a00640 1 -- 172.21.15.161:0/4170096384 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10004a20 con 0x7f8d20071b20 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.157+0000 7f8d25800640 1 -- 172.21.15.161:0/4170096384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d200cc9f0 con 0x7f8d20071b20 2024-03-07T06:50:53.158 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.157+0000 7f8d16a00640 1 -- 172.21.15.161:0/4170096384 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8d1001b070 con 0x7f8d20071b20 2024-03-07T06:50:53.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.157+0000 7f8d16a00640 1 -- 172.21.15.161:0/4170096384 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f8d10004bc0 con 0x7f8d20071b20 2024-03-07T06:50:53.159 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.158+0000 7f8d25800640 1 -- 172.21.15.161:0/4170096384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d20062c90 con 0x7f8d20071b20 2024-03-07T06:50:53.162 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.162+0000 7f8d16a00640 1 -- 172.21.15.161:0/4170096384 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8d1001d440 con 0x7f8d20071b20 2024-03-07T06:50:53.166 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:53.167+0000 7f396b241040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-07T06:50:53.239 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.240+0000 7f8d25800640 1 -- 172.21.15.161:0/4170096384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "972747df-dbbf-4ae5-8101-be5d39fde1ed", "id": 5} v 0) v1 -- 0x7f8d2015f940 con 0x7f8d20071b20 2024-03-07T06:50:53.246 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.246+0000 7f8d16a00640 1 -- 172.21.15.161:0/4170096384 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "972747df-dbbf-4ae5-8101-be5d39fde1ed", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8d1001d620 con 0x7f8d20071b20 2024-03-07T06:50:53.246 INFO:teuthology.orchestra.run.smithi161.stdout:5 2024-03-07T06:50:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.247+0000 7f8d25800640 1 -- 172.21.15.161:0/4170096384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d20071b20 msgr2=0x7f8d200b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.247+0000 7f8d25800640 1 --2- 172.21.15.161:0/4170096384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d20071b20 0x7f8d200b3a30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8d1000c560 tx=0x7f8d1000ca20 comp rx=0 tx=0).stop 2024-03-07T06:50:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.247+0000 7f8d25800640 1 -- 172.21.15.161:0/4170096384 shutdown_connections 2024-03-07T06:50:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.247+0000 7f8d25800640 1 --2- 172.21.15.161:0/4170096384 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d200b8350 0x7f8d200b87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.247+0000 7f8d25800640 1 --2- 172.21.15.161:0/4170096384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d2015e3e0 0x7f8d2015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.247+0000 7f8d25800640 1 --2- 172.21.15.161:0/4170096384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d20071b20 0x7f8d200b3a30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.248+0000 7f8d25800640 1 -- 172.21.15.161:0/4170096384 >> 172.21.15.161:0/4170096384 conn(0x7f8d200d0c30 msgr2=0x7f8d200d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:53.247 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.248+0000 7f8d25800640 1 -- 172.21.15.161:0/4170096384 shutdown_connections 2024-03-07T06:50:53.248 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.248+0000 7f8d25800640 1 -- 172.21.15.161:0/4170096384 wait complete. 2024-03-07T06:50:53.264 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new df89d7ba-8eab-4e1f-b03b-6e322d1a26d5 6 2024-03-07T06:50:53.344 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:53.345+0000 7f396b241040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-07T06:50:53.395 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:53.395+0000 7f396b241040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-07T06:50:53.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.449+0000 7f2953200640 1 Processor -- start 2024-03-07T06:50:53.449 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.449+0000 7f2953200640 1 -- start start 2024-03-07T06:50:53.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.451+0000 7f2953200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f294c162a10 0x7f294c070e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:53.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.451+0000 7f2953200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c0715b0 con 0x7f294c15e1d0 2024-03-07T06:50:53.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.451+0000 7f2953200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c071890 con 0x7f294c162580 2024-03-07T06:50:53.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.451+0000 7f2953200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c0d2220 con 0x7f294c162a10 2024-03-07T06:50:53.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.451+0000 7f2951200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f294c162a10 0x7f294c070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:53.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.451+0000 7f2943e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f294c162580 0x7f294c0d2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53746/0 (socket says 172.21.15.161:53746) 2024-03-07T06:50:53.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.451+0000 7f2943e00640 1 -- 172.21.15.161:0/2341346102 learned_addr learned my addr 172.21.15.161:0/2341346102 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-03-07T06:50:53.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.451+0000 7f2951200640 1 -- 172.21.15.161:0/2341346102 >> v1:172.21.15.161:6789/0 conn(0x7f294c162580 legacy=0x7f294c0d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.451+0000 7f2951200640 1 -- 172.21.15.161:0/2341346102 >> v1:172.21.15.133:6789/0 conn(0x7f294c15e1d0 legacy=0x7f294c161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.451 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.451+0000 7f2951200640 1 -- 172.21.15.161:0/2341346102 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c071b70 con 0x7f294c162a10 2024-03-07T06:50:53.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.453+0000 7f2951200640 1 --2- 172.21.15.161:0/2341346102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f294c162a10 0x7f294c070e10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2944009d90 tx=0x7f294402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9d57138d0a665d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:53.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.453+0000 7f294b400640 1 -- 172.21.15.161:0/2341346102 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944039070 con 0x7f294c162a10 2024-03-07T06:50:53.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.453+0000 7f294b400640 1 -- 172.21.15.161:0/2341346102 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294402cc60 con 0x7f294c162a10 2024-03-07T06:50:53.452 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.453+0000 7f294b400640 1 -- 172.21.15.161:0/2341346102 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944034660 con 0x7f294c162a10 2024-03-07T06:50:53.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.453+0000 7f2953200640 1 -- 172.21.15.161:0/2341346102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f294c162a10 msgr2=0x7f294c070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.453+0000 7f2953200640 1 --2- 172.21.15.161:0/2341346102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f294c162a10 0x7f294c070e10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2944009d90 tx=0x7f294402beb0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.454+0000 7f2953200640 1 -- 172.21.15.161:0/2341346102 shutdown_connections 2024-03-07T06:50:53.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.454+0000 7f2953200640 1 --2- 172.21.15.161:0/2341346102 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f294c162a10 0x7f294c070e10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.453 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.454+0000 7f2953200640 1 -- 172.21.15.161:0/2341346102 >> 172.21.15.161:0/2341346102 conn(0x7f294c0d4230 msgr2=0x7f294c0d4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:53.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.454+0000 7f2953200640 1 -- 172.21.15.161:0/2341346102 shutdown_connections 2024-03-07T06:50:53.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.454+0000 7f2953200640 1 -- 172.21.15.161:0/2341346102 wait complete. 2024-03-07T06:50:53.454 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.455+0000 7f2953200640 1 Processor -- start 2024-03-07T06:50:53.455 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.455+0000 7f2953200640 1 -- start start 2024-03-07T06:50:53.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.456+0000 7f2953200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f294c15e1d0 0x7f294c15a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:53.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.456+0000 7f2953200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f294c162580 0x7f294c0cd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:53.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.456+0000 7f2953200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f294c162a10 0x7f294c0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:53.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.456+0000 7f2953200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c071350 con 0x7f294c15e1d0 2024-03-07T06:50:53.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.456+0000 7f2953200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0d2500 con 0x7f294c162580 2024-03-07T06:50:53.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.456+0000 7f2953200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c0d30a0 con 0x7f294c162a10 2024-03-07T06:50:53.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.456+0000 7f2943e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f294c162580 0x7f294c0cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:53.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.456+0000 7f2951200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f294c162a10 0x7f294c0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:53.458 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.457+0000 7f2951200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f294c162a10 0x7f294c0cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:56002/0 (socket says 172.21.15.161:56002) 2024-03-07T06:50:53.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.457+0000 7f2951200640 1 -- 172.21.15.161:0/2707157826 learned_addr learned my addr 172.21.15.161:0/2707157826 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-07T06:50:53.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.457+0000 7f294be00640 1 --2- 172.21.15.161:0/2707157826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f294c15e1d0 0x7f294c15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:53.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.457+0000 7f2951200640 1 -- 172.21.15.161:0/2707157826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f294c162580 msgr2=0x7f294c0cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.457+0000 7f2951200640 1 --2- 172.21.15.161:0/2707157826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f294c162580 0x7f294c0cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.457+0000 7f2951200640 1 -- 172.21.15.161:0/2707157826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f294c15e1d0 msgr2=0x7f294c15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.457+0000 7f2951200640 1 --2- 172.21.15.161:0/2707157826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f294c15e1d0 0x7f294c15a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.457+0000 7f2951200640 1 -- 172.21.15.161:0/2707157826 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c1e9b40 con 0x7f294c162a10 2024-03-07T06:50:53.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.457+0000 7f2943e00640 1 --2- 172.21.15.161:0/2707157826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f294c162580 0x7f294c0cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T06:50:53.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.457+0000 7f2951200640 1 --2- 172.21.15.161:0/2707157826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f294c162a10 0x7f294c0cd670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2944009d90 tx=0x7f2944004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:53.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.457+0000 7f2948c00640 1 -- 172.21.15.161:0/2707157826 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944039050 con 0x7f294c162a10 2024-03-07T06:50:53.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.458+0000 7f2948c00640 1 -- 172.21.15.161:0/2707157826 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2944045070 con 0x7f294c162a10 2024-03-07T06:50:53.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.458+0000 7f2948c00640 1 -- 172.21.15.161:0/2707157826 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294403e830 con 0x7f294c162a10 2024-03-07T06:50:53.459 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.458+0000 7f2953200640 1 -- 172.21.15.161:0/2707157826 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c15aab0 con 0x7f294c162a10 2024-03-07T06:50:53.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.458+0000 7f2953200640 1 -- 172.21.15.161:0/2707157826 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c15b070 con 0x7f294c162a10 2024-03-07T06:50:53.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.458+0000 7f2948c00640 1 -- 172.21.15.161:0/2707157826 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f294403e9d0 con 0x7f294c162a10 2024-03-07T06:50:53.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.458+0000 7f2948c00640 1 -- 172.21.15.161:0/2707157826 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f294404ab90 con 0x7f294c162a10 2024-03-07T06:50:53.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.459+0000 7f2953200640 1 -- 172.21.15.161:0/2707157826 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294c0d4610 con 0x7f294c162a10 2024-03-07T06:50:53.460 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.461+0000 7f2948c00640 1 -- 172.21.15.161:0/2707157826 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f294c0d4610 con 0x7f294c162a10 2024-03-07T06:50:53.540 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.540+0000 7f2953200640 1 -- 172.21.15.161:0/2707157826 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd new", "uuid": "df89d7ba-8eab-4e1f-b03b-6e322d1a26d5", "id": 6} v 0) v1 -- 0x7f294c0d1dd0 con 0x7f294c162a10 2024-03-07T06:50:53.549 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.550+0000 7f2948c00640 1 -- 172.21.15.161:0/2707157826 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "df89d7ba-8eab-4e1f-b03b-6e322d1a26d5", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f294403ee00 con 0x7f294c162a10 2024-03-07T06:50:53.549 INFO:teuthology.orchestra.run.smithi161.stdout:6 2024-03-07T06:50:53.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.551+0000 7f2953200640 1 -- 172.21.15.161:0/2707157826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f294c162a10 msgr2=0x7f294c0cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.551+0000 7f2953200640 1 --2- 172.21.15.161:0/2707157826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f294c162a10 0x7f294c0cd670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2944009d90 tx=0x7f2944004a40 comp rx=0 tx=0).stop 2024-03-07T06:50:53.551 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.551+0000 7f2953200640 1 -- 172.21.15.161:0/2707157826 shutdown_connections 2024-03-07T06:50:53.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.551+0000 7f2953200640 1 --2- 172.21.15.161:0/2707157826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f294c162a10 0x7f294c0cd670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.551+0000 7f2953200640 1 --2- 172.21.15.161:0/2707157826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f294c162580 0x7f294c0cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.551+0000 7f2953200640 1 --2- 172.21.15.161:0/2707157826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f294c15e1d0 0x7f294c15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.551+0000 7f2953200640 1 -- 172.21.15.161:0/2707157826 >> 172.21.15.161:0/2707157826 conn(0x7f294c0d4230 msgr2=0x7f294c0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:53.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.551+0000 7f2953200640 1 -- 172.21.15.161:0/2707157826 shutdown_connections 2024-03-07T06:50:53.552 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.551+0000 7f2953200640 1 -- 172.21.15.161:0/2707157826 wait complete. 2024-03-07T06:50:53.566 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new a53c260c-9f9b-418c-b6d2-8aa034368372 7 2024-03-07T06:50:53.570 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:53.571+0000 7f396b241040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-07T06:50:53.744 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.745+0000 7fde46800640 1 Processor -- start 2024-03-07T06:50:53.745 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.745+0000 7fde46800640 1 -- start start 2024-03-07T06:50:53.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.746+0000 7fde46800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fde40162a10 0x7fde40070e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:53.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.746+0000 7fde46800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde400715b0 con 0x7fde4015e1d0 2024-03-07T06:50:53.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.746+0000 7fde46800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde40071890 con 0x7fde40162580 2024-03-07T06:50:53.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.746+0000 7fde46800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde400d2220 con 0x7fde40162a10 2024-03-07T06:50:53.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.746+0000 7fde3ea00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fde40162580 0x7fde400d2990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:53754/0 (socket says 172.21.15.161:53754) 2024-03-07T06:50:53.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.746+0000 7fde3ea00640 1 -- 172.21.15.161:0/442848972 learned_addr learned my addr 172.21.15.161:0/442848972 (peer_addr_for_me v1:172.21.15.161:0/0) 2024-03-07T06:50:53.746 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.747+0000 7fde3fe00640 1 --2- 172.21.15.161:0/442848972 >> v2:172.21.15.161:3300/0 conn(0x7fde40162a10 0x7fde40070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:53.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.747+0000 7fde3e000640 1 -- 172.21.15.161:0/442848972 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801192750 0 0) 0x7fde40071890 con 0x7fde40162580 2024-03-07T06:50:53.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.747+0000 7fde3e000640 1 -- 172.21.15.161:0/442848972 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde24003520 con 0x7fde40162580 2024-03-07T06:50:53.747 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.747+0000 7fde3fe00640 1 -- 172.21.15.161:0/442848972 >> v1:172.21.15.161:6789/0 conn(0x7fde40162580 legacy=0x7fde400d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.747+0000 7fde3fe00640 1 -- 172.21.15.161:0/442848972 >> v1:172.21.15.133:6789/0 conn(0x7fde4015e1d0 legacy=0x7fde40161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.747+0000 7fde3e000640 1 -- 172.21.15.161:0/442848972 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564480369 0 0) 0x7fde400715b0 con 0x7fde4015e1d0 2024-03-07T06:50:53.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.747+0000 7fde3fe00640 1 -- 172.21.15.161:0/442848972 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde40071b70 con 0x7fde40162a10 2024-03-07T06:50:53.748 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.749+0000 7fde3fe00640 1 --2- 172.21.15.161:0/442848972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde40162a10 0x7fde40070e10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fde380097a0 tx=0x7fde3802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6969bcdb945e9f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:53.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.749+0000 7fde3e000640 1 -- 172.21.15.161:0/442848972 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde38035020 con 0x7fde40162a10 2024-03-07T06:50:53.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.749+0000 7fde3e000640 1 -- 172.21.15.161:0/442848972 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde38031830 con 0x7fde40162a10 2024-03-07T06:50:53.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.749+0000 7fde3e000640 1 -- 172.21.15.161:0/442848972 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde38031c00 con 0x7fde40162a10 2024-03-07T06:50:53.749 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.750+0000 7fde46800640 1 -- 172.21.15.161:0/442848972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde40162a10 msgr2=0x7fde40070e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.750+0000 7fde46800640 1 --2- 172.21.15.161:0/442848972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde40162a10 0x7fde40070e10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fde380097a0 tx=0x7fde3802b8d0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.750+0000 7fde46800640 1 -- 172.21.15.161:0/442848972 shutdown_connections 2024-03-07T06:50:53.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.750+0000 7fde46800640 1 --2- 172.21.15.161:0/442848972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde40162a10 0x7fde40070e10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.750+0000 7fde46800640 1 -- 172.21.15.161:0/442848972 >> 172.21.15.161:0/442848972 conn(0x7fde400d4230 msgr2=0x7fde400d4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:53.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.750+0000 7fde46800640 1 -- 172.21.15.161:0/442848972 shutdown_connections 2024-03-07T06:50:53.750 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.750+0000 7fde46800640 1 -- 172.21.15.161:0/442848972 wait complete. 2024-03-07T06:50:53.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.752+0000 7fde46800640 1 Processor -- start 2024-03-07T06:50:53.751 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.752+0000 7fde46800640 1 -- start start 2024-03-07T06:50:53.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.753+0000 7fde46800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4015e1d0 0x7fde4015b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:53.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.753+0000 7fde46800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde40162580 0x7fde401e8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:53.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.753+0000 7fde46800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde40162a10 0x7fde4015fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:53.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.753+0000 7fde46800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde40071350 con 0x7fde40162580 2024-03-07T06:50:53.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.753+0000 7fde46800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde400d2500 con 0x7fde4015e1d0 2024-03-07T06:50:53.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.753+0000 7fde46800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde400d30a0 con 0x7fde40162a10 2024-03-07T06:50:53.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.753+0000 7fde3f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4015e1d0 0x7fde4015b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:53.754 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.753+0000 7fde3f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4015e1d0 0x7fde4015b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:55454/0 (socket says 172.21.15.161:55454) 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.753+0000 7fde3fe00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde40162a10 0x7fde4015fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.753+0000 7fde3f400640 1 -- 172.21.15.161:0/1322304536 learned_addr learned my addr 172.21.15.161:0/1322304536 (peer_addr_for_me v2:172.21.15.161:0/0) 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.753+0000 7fde3ea00640 1 --2- 172.21.15.161:0/1322304536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde40162580 0x7fde401e8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.754+0000 7fde3f400640 1 -- 172.21.15.161:0/1322304536 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde40162a10 msgr2=0x7fde4015fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.754+0000 7fde3f400640 1 --2- 172.21.15.161:0/1322304536 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde40162a10 0x7fde4015fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.754+0000 7fde3f400640 1 -- 172.21.15.161:0/1322304536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde40162580 msgr2=0x7fde401e8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.754+0000 7fde3f400640 1 --2- 172.21.15.161:0/1322304536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde40162580 0x7fde401e8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.754+0000 7fde3f400640 1 -- 172.21.15.161:0/1322304536 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde401e7ca0 con 0x7fde4015e1d0 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.754+0000 7fde3ea00640 1 --2- 172.21.15.161:0/1322304536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde40162580 0x7fde401e8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.754+0000 7fde3f400640 1 --2- 172.21.15.161:0/1322304536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4015e1d0 0x7fde4015b6d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fde2800c460 tx=0x7fde2800c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.754+0000 7fde37400640 1 -- 172.21.15.161:0/1322304536 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28005480 con 0x7fde4015e1d0 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.754+0000 7fde37400640 1 -- 172.21.15.161:0/1322304536 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde280035f0 con 0x7fde4015e1d0 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.754+0000 7fde46800640 1 -- 172.21.15.161:0/1322304536 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde401e7f80 con 0x7fde4015e1d0 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.754+0000 7fde37400640 1 -- 172.21.15.161:0/1322304536 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28003a00 con 0x7fde4015e1d0 2024-03-07T06:50:53.755 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.754+0000 7fde46800640 1 -- 172.21.15.161:0/1322304536 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde401f5180 con 0x7fde4015e1d0 2024-03-07T06:50:53.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.755+0000 7fde37400640 1 -- 172.21.15.161:0/1322304536 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fde28003ba0 con 0x7fde4015e1d0 2024-03-07T06:50:53.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.755+0000 7fde37400640 1 -- 172.21.15.161:0/1322304536 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fde2801ba60 con 0x7fde4015e1d0 2024-03-07T06:50:53.756 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.756+0000 7fde46800640 1 -- 172.21.15.161:0/1322304536 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde401f5380 con 0x7fde4015e1d0 2024-03-07T06:50:53.757 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.758+0000 7fde37400640 1 -- 172.21.15.161:0/1322304536 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fde401f5380 con 0x7fde4015e1d0 2024-03-07T06:50:53.831 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.832+0000 7fde46800640 1 -- 172.21.15.161:0/1322304536 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a53c260c-9f9b-418c-b6d2-8aa034368372", "id": 7} v 0) v1 -- 0x7fde40062c90 con 0x7fde4015e1d0 2024-03-07T06:50:53.842 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.842+0000 7fde37400640 1 -- 172.21.15.161:0/1322304536 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a53c260c-9f9b-418c-b6d2-8aa034368372", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fde2801b400 con 0x7fde4015e1d0 2024-03-07T06:50:53.842 INFO:teuthology.orchestra.run.smithi161.stdout:7 2024-03-07T06:50:53.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.843+0000 7fde46800640 1 -- 172.21.15.161:0/1322304536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4015e1d0 msgr2=0x7fde4015b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:53.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.843+0000 7fde46800640 1 --2- 172.21.15.161:0/1322304536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4015e1d0 0x7fde4015b6d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fde2800c460 tx=0x7fde2800c920 comp rx=0 tx=0).stop 2024-03-07T06:50:53.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.843+0000 7fde46800640 1 -- 172.21.15.161:0/1322304536 shutdown_connections 2024-03-07T06:50:53.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.843+0000 7fde46800640 1 --2- 172.21.15.161:0/1322304536 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fde40162a10 0x7fde4015fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.843+0000 7fde46800640 1 --2- 172.21.15.161:0/1322304536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fde40162580 0x7fde401e8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.843+0000 7fde46800640 1 --2- 172.21.15.161:0/1322304536 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fde4015e1d0 0x7fde4015b6d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:53.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.843+0000 7fde46800640 1 -- 172.21.15.161:0/1322304536 >> 172.21.15.161:0/1322304536 conn(0x7fde400d4230 msgr2=0x7fde400cfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:53.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.843+0000 7fde46800640 1 -- 172.21.15.161:0/1322304536 shutdown_connections 2024-03-07T06:50:53.843 INFO:teuthology.orchestra.run.smithi161.stderr:2024-03-07T06:50:53.844+0000 7fde46800640 1 -- 172.21.15.161:0/1322304536 wait complete. 2024-03-07T06:50:53.859 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-07T06:50:53.859 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-07T06:50:53.867 INFO:tasks.ceph.osd.0:Started 2024-03-07T06:50:53.867 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-07T06:50:53.867 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-07T06:50:53.874 INFO:tasks.ceph.osd.1:Started 2024-03-07T06:50:53.874 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-07T06:50:53.874 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-07T06:50:53.880 INFO:tasks.ceph.osd.2:Started 2024-03-07T06:50:53.880 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-07T06:50:53.880 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-07T06:50:53.884 INFO:tasks.ceph.osd.3:Started 2024-03-07T06:50:53.884 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-07T06:50:53.885 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-07T06:50:53.892 INFO:tasks.ceph.osd.4:Started 2024-03-07T06:50:53.892 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-07T06:50:53.892 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-07T06:50:53.899 INFO:tasks.ceph.osd.5:Started 2024-03-07T06:50:53.899 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-07T06:50:53.899 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-07T06:50:53.900 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:53.898+0000 7f63afe14040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-07T06:50:53.904 INFO:tasks.ceph.osd.6:Started 2024-03-07T06:50:53.904 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-07T06:50:53.905 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-07T06:50:53.913 INFO:tasks.ceph.osd.7:Started 2024-03-07T06:50:53.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:50:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.139+0000 7f9650200640 1 Processor -- start 2024-03-07T06:50:54.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.139+0000 7f9650200640 1 -- start start 2024-03-07T06:50:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.141+0000 7f9650200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f96480ce5f0 0x7f96480b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.141+0000 7f9650200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96480af950 0x7f9648071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.141+0000 7f9650200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964806f140 con 0x7f964815b1b0 2024-03-07T06:50:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.141+0000 7f9650200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9648071610 con 0x7f96480af950 2024-03-07T06:50:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.141+0000 7f9650200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964806f420 con 0x7f96480ce5f0 2024-03-07T06:50:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.141+0000 7f964e200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96480af950 0x7f9648071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.141+0000 7f964d800640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f964815b1b0 0x7f96480cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:40984/0 (socket says 172.21.15.133:40984) 2024-03-07T06:50:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.141+0000 7f964d800640 1 -- 172.21.15.133:0/703567314 learned_addr learned my addr 172.21.15.133:0/703567314 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:50:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.141+0000 7f964ce00640 1 --2- 172.21.15.133:0/703567314 >> v2:172.21.15.161:3300/0 conn(0x7f96480ce5f0 0x7f96480b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:54.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.141+0000 7f964e200640 1 -- 172.21.15.133:0/703567314 >> v2:172.21.15.161:3300/0 conn(0x7f96480ce5f0 msgr2=0x7f96480b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:50:54.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.141+0000 7f964e200640 1 --2- 172.21.15.133:0/703567314 >> v2:172.21.15.161:3300/0 conn(0x7f96480ce5f0 0x7f96480b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:54.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.141+0000 7f964e200640 1 -- 172.21.15.133:0/703567314 >> v1:172.21.15.133:6789/0 conn(0x7f964815b1b0 legacy=0x7f96480cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:54.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.141+0000 7f964e200640 1 -- 172.21.15.133:0/703567314 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648071860 con 0x7f96480af950 2024-03-07T06:50:54.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.141+0000 7f9647e00640 1 -- 172.21.15.133:0/703567314 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257900937 0 0) 0x7f964806f140 con 0x7f964815b1b0 2024-03-07T06:50:54.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.142+0000 7f964e200640 1 --2- 172.21.15.133:0/703567314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96480af950 0x7f9648071d80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f96480b0270 tx=0x7f964002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777cdc99eefc5553 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:54.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.143+0000 7f9647e00640 1 -- 172.21.15.133:0/703567314 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640039070 con 0x7f96480af950 2024-03-07T06:50:54.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.143+0000 7f9647e00640 1 -- 172.21.15.133:0/703567314 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964002cbf0 con 0x7f96480af950 2024-03-07T06:50:54.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.143+0000 7f9647e00640 1 -- 172.21.15.133:0/703567314 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9640034440 con 0x7f96480af950 2024-03-07T06:50:54.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.143+0000 7f9650200640 1 -- 172.21.15.133:0/703567314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96480af950 msgr2=0x7f9648071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:54.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.143+0000 7f9650200640 1 --2- 172.21.15.133:0/703567314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96480af950 0x7f9648071d80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f96480b0270 tx=0x7f964002be40 comp rx=0 tx=0).stop 2024-03-07T06:50:54.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.143+0000 7f9650200640 1 -- 172.21.15.133:0/703567314 shutdown_connections 2024-03-07T06:50:54.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.143+0000 7f9650200640 1 --2- 172.21.15.133:0/703567314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96480af950 0x7f9648071d80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:54.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.143+0000 7f9650200640 1 --2- 172.21.15.133:0/703567314 >> v2:172.21.15.161:3300/0 conn(0x7f96480ce5f0 0x7f96480b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:54.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.143+0000 7f9650200640 1 -- 172.21.15.133:0/703567314 >> 172.21.15.133:0/703567314 conn(0x7f96480b3120 msgr2=0x7f96480b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:54.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.143+0000 7f9650200640 1 -- 172.21.15.133:0/703567314 shutdown_connections 2024-03-07T06:50:54.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.143+0000 7f9650200640 1 -- 172.21.15.133:0/703567314 wait complete. 2024-03-07T06:50:54.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.145+0000 7f9650200640 1 Processor -- start 2024-03-07T06:50:54.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.145+0000 7f9650200640 1 -- start start 2024-03-07T06:50:54.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.146+0000 7f9650200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96480af950 0x7f964815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:54.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.146+0000 7f9650200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96480ce5f0 0x7f9648144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:54.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.146+0000 7f9650200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f964815b1b0 0x7f96481474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:54.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.146+0000 7f9650200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964815b590 con 0x7f96480af950 2024-03-07T06:50:54.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.146+0000 7f9650200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964806ee30 con 0x7f96480ce5f0 2024-03-07T06:50:54.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.146+0000 7f9650200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964815b710 con 0x7f964815b1b0 2024-03-07T06:50:54.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.146+0000 7f964d800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96480af950 0x7f964815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:54.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.146+0000 7f964d800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96480af950 0x7f964815bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42394/0 (socket says 172.21.15.133:42394) 2024-03-07T06:50:54.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.146+0000 7f964d800640 1 -- 172.21.15.133:0/1008573143 learned_addr learned my addr 172.21.15.133:0/1008573143 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:50:54.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.146+0000 7f964ce00640 1 --2- 172.21.15.133:0/1008573143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96480ce5f0 0x7f9648144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:54.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.146+0000 7f964e200640 1 --2- 172.21.15.133:0/1008573143 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f964815b1b0 0x7f96481474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:54.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.147+0000 7f964d800640 1 -- 172.21.15.133:0/1008573143 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f964815b1b0 msgr2=0x7f96481474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:54.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.147+0000 7f964d800640 1 --2- 172.21.15.133:0/1008573143 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f964815b1b0 0x7f96481474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:54.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.147+0000 7f964d800640 1 -- 172.21.15.133:0/1008573143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96480ce5f0 msgr2=0x7f9648144c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:54.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.147+0000 7f964d800640 1 --2- 172.21.15.133:0/1008573143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96480ce5f0 0x7f9648144c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:54.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.147+0000 7f964d800640 1 -- 172.21.15.133:0/1008573143 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9648145180 con 0x7f96480af950 2024-03-07T06:50:54.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.147+0000 7f964d800640 1 --2- 172.21.15.133:0/1008573143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96480af950 0x7f964815bd40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f963c00c430 tx=0x7f963c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:54.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.147+0000 7f9645600640 1 -- 172.21.15.133:0/1008573143 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c0074e0 con 0x7f96480af950 2024-03-07T06:50:54.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.147+0000 7f9645600640 1 -- 172.21.15.133:0/1008573143 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963c0047e0 con 0x7f96480af950 2024-03-07T06:50:54.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.147+0000 7f9645600640 1 -- 172.21.15.133:0/1008573143 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c004bd0 con 0x7f96480af950 2024-03-07T06:50:54.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.147+0000 7f9650200640 1 -- 172.21.15.133:0/1008573143 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9648147d20 con 0x7f96480af950 2024-03-07T06:50:54.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.147+0000 7f9650200640 1 -- 172.21.15.133:0/1008573143 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96481482e0 con 0x7f96480af950 2024-03-07T06:50:54.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.148+0000 7f9645600640 1 -- 172.21.15.133:0/1008573143 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f963c011070 con 0x7f96480af950 2024-03-07T06:50:54.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.148+0000 7f9645600640 1 -- 172.21.15.133:0/1008573143 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f963c013a90 con 0x7f96480af950 2024-03-07T06:50:54.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.148+0000 7f9650200640 1 -- 172.21.15.133:0/1008573143 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9648062c90 con 0x7f96480af950 2024-03-07T06:50:54.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.151+0000 7f9645600640 1 -- 172.21.15.133:0/1008573143 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9648145180 con 0x7f96480af950 2024-03-07T06:50:54.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.221+0000 7f9650200640 1 -- 172.21.15.133:0/1008573143 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96480cc660 con 0x7f96480af950 2024-03-07T06:50:54.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.222+0000 7f9645600640 1 -- 172.21.15.133:0/1008573143 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f963c004d70 con 0x7f96480af950 2024-03-07T06:50:54.222 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:50:54.222 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:50:53.834716+0000","last_up_change":"0.000000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:50:54.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.223+0000 7f9650200640 1 -- 172.21.15.133:0/1008573143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96480af950 msgr2=0x7f964815bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:54.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.223+0000 7f9650200640 1 --2- 172.21.15.133:0/1008573143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96480af950 0x7f964815bd40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f963c00c430 tx=0x7f963c00c8f0 comp rx=0 tx=0).stop 2024-03-07T06:50:54.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.223+0000 7f9650200640 1 -- 172.21.15.133:0/1008573143 shutdown_connections 2024-03-07T06:50:54.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.223+0000 7f9650200640 1 --2- 172.21.15.133:0/1008573143 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f964815b1b0 0x7f96481474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:54.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.223+0000 7f9650200640 1 --2- 172.21.15.133:0/1008573143 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96480ce5f0 0x7f9648144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:54.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.223+0000 7f9650200640 1 --2- 172.21.15.133:0/1008573143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96480af950 0x7f964815bd40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:54.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.223+0000 7f9650200640 1 -- 172.21.15.133:0/1008573143 >> 172.21.15.133:0/1008573143 conn(0x7f96480b3120 msgr2=0x7f96480cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:54.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.223+0000 7f9650200640 1 -- 172.21.15.133:0/1008573143 shutdown_connections 2024-03-07T06:50:54.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:50:54.223+0000 7f9650200640 1 -- 172.21.15.133:0/1008573143 wait complete. 2024-03-07T06:50:54.238 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-07T06:50:54.238 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-07T06:50:54.250 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:54.251+0000 7f63afe14040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-07T06:50:54.311 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:54.311+0000 7f63afe14040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-07T06:50:54.442 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:54.443+0000 7f63afe14040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-07T06:50:54.578 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-03-07T06:50:54.578+0000 7fe2266ac640 -1 Falling back to public interface 2024-03-07T06:50:54.585 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-03-07T06:50:54.585+0000 7f428a014640 -1 Falling back to public interface 2024-03-07T06:50:54.588 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-03-07T06:50:54.589+0000 7f77ee411640 -1 Falling back to public interface 2024-03-07T06:50:54.602 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-03-07T06:50:54.603+0000 7fa1c561b640 -1 Falling back to public interface 2024-03-07T06:50:54.606 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-03-07T06:50:54.607+0000 7f9897971640 -1 Falling back to public interface 2024-03-07T06:50:54.607 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-03-07T06:50:54.607+0000 7fd83e011640 -1 Falling back to public interface 2024-03-07T06:50:54.607 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-03-07T06:50:54.608+0000 7f3defe9b640 -1 Falling back to public interface 2024-03-07T06:50:54.620 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-03-07T06:50:54.621+0000 7fb2d760f640 -1 Falling back to public interface 2024-03-07T06:50:54.690 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:54.690+0000 7f396b241040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-07T06:50:55.001 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:55.002+0000 7f396b241040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-07T06:50:55.045 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-07T06:50:55.062 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:55.062+0000 7f396b241040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-07T06:50:55.200 INFO:tasks.ceph.mgr.y.smithi133.stderr:2024-03-07T06:50:55.200+0000 7f396b241040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-07T06:50:55.226 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.207+0000 7f150fc00640 1 Processor -- start 2024-03-07T06:50:55.226 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.207+0000 7f150fc00640 1 -- start start 2024-03-07T06:50:55.226 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.208+0000 7f150fc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f150815e070 0x7f150815e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:55.226 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.208+0000 7f150fc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1508061aa0 0x7f15080b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:55.226 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.208+0000 7f150fc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15080d0e50 con 0x7f1508061ee0 2024-03-07T06:50:55.226 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.208+0000 7f150fc00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15080d1130 con 0x7f1508061aa0 2024-03-07T06:50:55.226 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.208+0000 7f150fc00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15080b5370 con 0x7f150815e070 2024-03-07T06:50:55.226 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.209+0000 7f150d200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f150815e070 0x7f150815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:55.226 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.209+0000 7f1507e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1508061aa0 0x7f15080b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:55.227 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.209+0000 7f150d200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f150815e070 0x7f150815e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:52850/0 (socket says 172.21.15.133:52850) 2024-03-07T06:50:55.227 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.209+0000 7f150d200640 1 -- 172.21.15.133:0/1697983057 learned_addr learned my addr 172.21.15.133:0/1697983057 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:50:55.227 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.209+0000 7f150d200640 1 -- 172.21.15.133:0/1697983057 >> v2:172.21.15.161:3300/0 conn(0x7f1508061aa0 msgr2=0x7f15080b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:50:55.227 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.209+0000 7f150d200640 1 --2- 172.21.15.133:0/1697983057 >> v2:172.21.15.161:3300/0 conn(0x7f1508061aa0 0x7f15080b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:55.227 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.209+0000 7f150d200640 1 -- 172.21.15.133:0/1697983057 >> v1:172.21.15.161:6789/0 conn(0x7f1508061ee0 legacy=0x7f150806b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:55.227 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.209+0000 7f150d200640 1 -- 172.21.15.133:0/1697983057 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15080b5650 con 0x7f150815e070 2024-03-07T06:50:55.227 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.210+0000 7f150d200640 1 --2- 172.21.15.133:0/1697983057 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f150815e070 0x7f150815e450 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009870 tx=0x7f14fc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8716493673ea635d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:55.227 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.211+0000 7f1507400640 1 -- 172.21.15.133:0/1697983057 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc035020 con 0x7f150815e070 2024-03-07T06:50:55.227 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.211+0000 7f1507400640 1 -- 172.21.15.133:0/1697983057 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc031640 con 0x7f150815e070 2024-03-07T06:50:55.227 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.211+0000 7f1507400640 1 -- 172.21.15.133:0/1697983057 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc02d040 con 0x7f150815e070 2024-03-07T06:50:55.227 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.211+0000 7f150fc00640 1 -- 172.21.15.133:0/1697983057 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f150815e070 msgr2=0x7f150815e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:55.227 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.211+0000 7f150fc00640 1 --2- 172.21.15.133:0/1697983057 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f150815e070 0x7f150815e450 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009870 tx=0x7f14fc02b8f0 comp rx=0 tx=0).stop 2024-03-07T06:50:55.227 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.211+0000 7f150fc00640 1 -- 172.21.15.133:0/1697983057 shutdown_connections 2024-03-07T06:50:55.227 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.211+0000 7f150fc00640 1 --2- 172.21.15.133:0/1697983057 >> v2:172.21.15.161:3300/0 conn(0x7f1508061aa0 0x7f15080b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.211+0000 7f150fc00640 1 --2- 172.21.15.133:0/1697983057 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f150815e070 0x7f150815e450 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.211+0000 7f150fc00640 1 -- 172.21.15.133:0/1697983057 >> 172.21.15.133:0/1697983057 conn(0x7f1508072720 msgr2=0x7f1508072b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.211+0000 7f150fc00640 1 -- 172.21.15.133:0/1697983057 shutdown_connections 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.211+0000 7f150fc00640 1 -- 172.21.15.133:0/1697983057 wait complete. 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.213+0000 7f150fc00640 1 Processor -- start 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.213+0000 7f150fc00640 1 -- start start 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.214+0000 7f150fc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1508061aa0 0x7f15080b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.214+0000 7f150fc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1508061ee0 0x7f15080b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.214+0000 7f150fc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f150815e070 0x7f15080c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.214+0000 7f150fc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150806b8e0 con 0x7f150815e070 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.214+0000 7f150fc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150806b760 con 0x7f1508061ee0 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.214+0000 7f150fc00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15080b60b0 con 0x7f1508061aa0 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.214+0000 7f150dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f150815e070 0x7f15080c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.214+0000 7f150dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f150815e070 0x7f15080c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42404/0 (socket says 172.21.15.133:42404) 2024-03-07T06:50:55.228 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.214+0000 7f150dc00640 1 -- 172.21.15.133:0/714168323 learned_addr learned my addr 172.21.15.133:0/714168323 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:50:55.229 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.215+0000 7f1507e00640 1 --2- 172.21.15.133:0/714168323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1508061ee0 0x7f15080b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:55.229 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.215+0000 7f150d200640 1 --2- 172.21.15.133:0/714168323 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1508061aa0 0x7f15080b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:50:55.231 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.215+0000 7f150dc00640 1 -- 172.21.15.133:0/714168323 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1508061aa0 msgr2=0x7f15080b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:55.231 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.215+0000 7f150dc00640 1 --2- 172.21.15.133:0/714168323 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1508061aa0 0x7f15080b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:55.231 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.215+0000 7f150dc00640 1 -- 172.21.15.133:0/714168323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1508061ee0 msgr2=0x7f15080b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:55.231 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.215+0000 7f150dc00640 1 --2- 172.21.15.133:0/714168323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1508061ee0 0x7f15080b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:55.231 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.215+0000 7f150dc00640 1 -- 172.21.15.133:0/714168323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15081e8d30 con 0x7f150815e070 2024-03-07T06:50:55.231 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.215+0000 7f150dc00640 1 --2- 172.21.15.133:0/714168323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f150815e070 0x7f15080c5f70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f150000c7a0 tx=0x7f150000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:50:55.231 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.215+0000 7f1504c00640 1 -- 172.21.15.133:0/714168323 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500003b70 con 0x7f150815e070 2024-03-07T06:50:55.231 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.215+0000 7f1504c00640 1 -- 172.21.15.133:0/714168323 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1500003d30 con 0x7f150815e070 2024-03-07T06:50:55.232 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.215+0000 7f1504c00640 1 -- 172.21.15.133:0/714168323 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150000aa30 con 0x7f150815e070 2024-03-07T06:50:55.232 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.215+0000 7f150fc00640 1 -- 172.21.15.133:0/714168323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15081e9010 con 0x7f150815e070 2024-03-07T06:50:55.232 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.215+0000 7f150fc00640 1 -- 172.21.15.133:0/714168323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15081e95d0 con 0x7f150815e070 2024-03-07T06:50:55.232 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.216+0000 7f150fc00640 1 -- 172.21.15.133:0/714168323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1508068570 con 0x7f150815e070 2024-03-07T06:50:55.232 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.217+0000 7f1504c00640 1 -- 172.21.15.133:0/714168323 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 2) v1 ==== 44914+0+0 (secure 0 0 0) 0x7f150000abd0 con 0x7f150815e070 2024-03-07T06:50:55.232 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.217+0000 7f1504c00640 1 -- 172.21.15.133:0/714168323 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f1500049de0 con 0x7f150815e070 2024-03-07T06:50:55.232 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.219+0000 7f1504c00640 1 -- 172.21.15.133:0/714168323 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1500011540 con 0x7f150815e070 2024-03-07T06:50:55.285 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.285+0000 7f150fc00640 1 -- 172.21.15.133:0/714168323 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1508073310 con 0x7f150815e070 2024-03-07T06:50:55.286 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.286+0000 7f1504c00640 1 -- 172.21.15.133:0/714168323 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f15000117a0 con 0x7f150815e070 2024-03-07T06:50:55.286 INFO:teuthology.misc.health.smithi133.stdout: 2024-03-07T06:50:55.286 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":10,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:50:53.834716+0000","last_up_change":"0.000000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:50:55.287 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.287+0000 7f150fc00640 1 -- 172.21.15.133:0/714168323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f150815e070 msgr2=0x7f15080c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:50:55.287 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.287+0000 7f150fc00640 1 --2- 172.21.15.133:0/714168323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f150815e070 0x7f15080c5f70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f150000c7a0 tx=0x7f150000cc60 comp rx=0 tx=0).stop 2024-03-07T06:50:55.288 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.287+0000 7f150fc00640 1 -- 172.21.15.133:0/714168323 shutdown_connections 2024-03-07T06:50:55.288 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.287+0000 7f150fc00640 1 --2- 172.21.15.133:0/714168323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f150815e070 0x7f15080c5f70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:55.288 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.287+0000 7f150fc00640 1 --2- 172.21.15.133:0/714168323 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1508061ee0 0x7f15080b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:55.288 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.287+0000 7f150fc00640 1 --2- 172.21.15.133:0/714168323 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1508061aa0 0x7f15080b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:50:55.288 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.287+0000 7f150fc00640 1 -- 172.21.15.133:0/714168323 >> 172.21.15.133:0/714168323 conn(0x7f1508072720 msgr2=0x7f150806e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:50:55.288 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.287+0000 7f150fc00640 1 -- 172.21.15.133:0/714168323 shutdown_connections 2024-03-07T06:50:55.288 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:50:55.288+0000 7f150fc00640 1 -- 172.21.15.133:0/714168323 wait complete. 2024-03-07T06:50:55.300 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-07T06:50:55.479 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:55.479+0000 7f6350200640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-07T06:50:55.481 INFO:tasks.ceph.mgr.x.smithi161.stderr:2024-03-07T06:50:55.481+0000 7f6350200640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-07T06:50:58.098 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-03-07T06:50:58.098+0000 7f77ee411640 -1 osd.1 0 log_to_monitors true 2024-03-07T06:50:58.120 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-03-07T06:50:58.121+0000 7fe2266ac640 -1 osd.6 0 log_to_monitors true 2024-03-07T06:50:58.149 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-03-07T06:50:58.149+0000 7f428a014640 -1 osd.7 0 log_to_monitors true 2024-03-07T06:50:58.177 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-03-07T06:50:58.178+0000 7f9897971640 -1 osd.5 0 log_to_monitors true 2024-03-07T06:50:58.190 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-03-07T06:50:58.191+0000 7fb2d760f640 -1 osd.4 0 log_to_monitors true 2024-03-07T06:50:58.191 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-03-07T06:50:58.192+0000 7fa1c561b640 -1 osd.2 0 log_to_monitors true 2024-03-07T06:50:58.228 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-03-07T06:50:58.228+0000 7f3defe9b640 -1 osd.3 0 log_to_monitors true 2024-03-07T06:50:58.412 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-03-07T06:50:58.412+0000 7fd83e011640 -1 osd.0 0 log_to_monitors true 2024-03-07T06:50:58.854 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:50:58.862 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:51:01.750 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-03-07T06:51:01.751+0000 7fd837c00640 -1 osd.0 0 waiting for initial osdmap 2024-03-07T06:51:01.759 INFO:tasks.ceph.osd.0.smithi133.stderr:2024-03-07T06:51:01.760+0000 7fd82e600640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T06:51:01.781 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-03-07T06:51:01.781+0000 7f4285800640 -1 osd.7 0 waiting for initial osdmap 2024-03-07T06:51:01.791 INFO:tasks.ceph.osd.7.smithi161.stderr:2024-03-07T06:51:01.792+0000 7f427a400640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T06:51:01.825 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-03-07T06:51:01.826+0000 7f77e9e00640 -1 osd.1 0 waiting for initial osdmap 2024-03-07T06:51:01.832 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-03-07T06:51:01.833+0000 7fa1bf000640 -1 osd.2 0 waiting for initial osdmap 2024-03-07T06:51:01.833 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-03-07T06:51:01.834+0000 7fe220800640 -1 osd.6 0 waiting for initial osdmap 2024-03-07T06:51:01.838 INFO:tasks.ceph.osd.1.smithi133.stderr:2024-03-07T06:51:01.839+0000 7f77dea00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T06:51:01.840 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-03-07T06:51:01.840+0000 7f9893800640 -1 osd.5 0 waiting for initial osdmap 2024-03-07T06:51:01.840 INFO:tasks.ceph.osd.6.smithi161.stderr:2024-03-07T06:51:01.840+0000 7fe217200640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T06:51:01.841 INFO:tasks.ceph.osd.2.smithi133.stderr:2024-03-07T06:51:01.842+0000 7fa1b5a00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T06:51:01.849 INFO:tasks.ceph.osd.5.smithi161.stderr:2024-03-07T06:51:01.850+0000 7f9888400640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T06:51:01.887 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-03-07T06:51:01.887+0000 7f3dea000640 -1 osd.3 0 waiting for initial osdmap 2024-03-07T06:51:01.895 INFO:tasks.ceph.osd.3.smithi133.stderr:2024-03-07T06:51:01.895+0000 7f3de0a00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T06:51:01.906 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-03-07T06:51:01.907+0000 7fb2d3000640 -1 osd.4 0 waiting for initial osdmap 2024-03-07T06:51:01.915 INFO:tasks.ceph.osd.4.smithi161.stderr:2024-03-07T06:51:01.915+0000 7fb2c7c00640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-07T06:51:02.107 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:02.286 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.286+0000 7f409d200640 1 Processor -- start 2024-03-07T06:51:02.286 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.286+0000 7f409d200640 1 -- start start 2024-03-07T06:51:02.288 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f409d200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4098160510 0x7f40980cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:02.288 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f409d200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f40980d0460 0x7f40980b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:02.288 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f409d200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409815fa50 con 0x7f40981599d0 2024-03-07T06:51:02.288 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f409d200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40980d08b0 con 0x7f40980d0460 2024-03-07T06:51:02.288 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f409d200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4098159db0 con 0x7f4098160510 2024-03-07T06:51:02.289 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f4095e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f40981599d0 0x7f409815c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55892/0 (socket says 172.21.15.133:55892) 2024-03-07T06:51:02.289 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f4095e00640 1 -- 172.21.15.133:0/3197852521 learned_addr learned my addr 172.21.15.133:0/3197852521 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:02.289 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f4096800640 1 --2- 172.21.15.133:0/3197852521 >> v2:172.21.15.133:3300/0 conn(0x7f40980d0460 0x7f40980b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:02.289 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f408d400640 1 --2- 172.21.15.133:0/3197852521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4098160510 0x7f40980cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:02.289 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f4095400640 1 -- 172.21.15.133:0/3197852521 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770467323 0 0) 0x7f409815fa50 con 0x7f40981599d0 2024-03-07T06:51:02.289 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f4095400640 1 -- 172.21.15.133:0/3197852521 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4078003520 con 0x7f40981599d0 2024-03-07T06:51:02.289 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f4096800640 1 -- 172.21.15.133:0/3197852521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4098160510 msgr2=0x7f40980cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:02.289 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f4096800640 1 --2- 172.21.15.133:0/3197852521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4098160510 0x7f40980cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:02.289 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f4096800640 1 -- 172.21.15.133:0/3197852521 >> v1:172.21.15.133:6789/0 conn(0x7f40981599d0 legacy=0x7f409815c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:02.289 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.288+0000 7f4096800640 1 -- 172.21.15.133:0/3197852521 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409815cf90 con 0x7f40980d0460 2024-03-07T06:51:02.290 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.289+0000 7f4096800640 1 --2- 172.21.15.133:0/3197852521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40980d0460 0x7f40980b3a10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4090009d90 tx=0x7f409002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e7c198f512a0d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:02.290 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.290+0000 7f4095400640 1 -- 172.21.15.133:0/3197852521 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090039070 con 0x7f40980d0460 2024-03-07T06:51:02.290 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.290+0000 7f4095400640 1 -- 172.21.15.133:0/3197852521 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409002cbc0 con 0x7f40980d0460 2024-03-07T06:51:02.291 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.290+0000 7f4095400640 1 -- 172.21.15.133:0/3197852521 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090034440 con 0x7f40980d0460 2024-03-07T06:51:02.291 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.290+0000 7f409d200640 1 -- 172.21.15.133:0/3197852521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40980d0460 msgr2=0x7f40980b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:02.291 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.290+0000 7f409d200640 1 --2- 172.21.15.133:0/3197852521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40980d0460 0x7f40980b3a10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4090009d90 tx=0x7f409002be10 comp rx=0 tx=0).stop 2024-03-07T06:51:02.291 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.290+0000 7f409d200640 1 -- 172.21.15.133:0/3197852521 shutdown_connections 2024-03-07T06:51:02.291 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.290+0000 7f409d200640 1 --2- 172.21.15.133:0/3197852521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40980d0460 0x7f40980b3a10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:02.291 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.290+0000 7f409d200640 1 --2- 172.21.15.133:0/3197852521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4098160510 0x7f40980cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:02.291 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.290+0000 7f409d200640 1 -- 172.21.15.133:0/3197852521 >> 172.21.15.133:0/3197852521 conn(0x7f409806b440 msgr2=0x7f409806b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:02.291 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.290+0000 7f409d200640 1 -- 172.21.15.133:0/3197852521 shutdown_connections 2024-03-07T06:51:02.291 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.290+0000 7f409d200640 1 -- 172.21.15.133:0/3197852521 wait complete. 2024-03-07T06:51:02.292 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.292+0000 7f409d200640 1 Processor -- start 2024-03-07T06:51:02.292 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.292+0000 7f409d200640 1 -- start start 2024-03-07T06:51:02.294 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.293+0000 7f409d200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40980d0460 0x7f409806c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:02.294 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.293+0000 7f409d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40981599d0 0x7f40980c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:02.294 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.293+0000 7f409d200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4098160510 0x7f40980c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:02.294 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.293+0000 7f409d200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40980cead0 con 0x7f40981599d0 2024-03-07T06:51:02.294 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.293+0000 7f409d200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409815fd10 con 0x7f40980d0460 2024-03-07T06:51:02.294 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.293+0000 7f409d200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409815cb90 con 0x7f4098160510 2024-03-07T06:51:02.294 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.293+0000 7f408d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40981599d0 0x7f40980c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:02.294 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.293+0000 7f408d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40981599d0 0x7f40980c8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42522/0 (socket says 172.21.15.133:42522) 2024-03-07T06:51:02.294 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.293+0000 7f408d400640 1 -- 172.21.15.133:0/1946374038 learned_addr learned my addr 172.21.15.133:0/1946374038 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:02.294 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.294+0000 7f4095e00640 1 --2- 172.21.15.133:0/1946374038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40980d0460 0x7f409806c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:02.294 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.294+0000 7f408d400640 1 -- 172.21.15.133:0/1946374038 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4098160510 msgr2=0x7f40980c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:02.294 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.294+0000 7f408d400640 1 --2- 172.21.15.133:0/1946374038 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4098160510 0x7f40980c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:02.295 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.294+0000 7f408d400640 1 -- 172.21.15.133:0/1946374038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40980d0460 msgr2=0x7f409806c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:02.295 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.294+0000 7f408d400640 1 --2- 172.21.15.133:0/1946374038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40980d0460 0x7f409806c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:02.295 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.294+0000 7f408d400640 1 -- 172.21.15.133:0/1946374038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409806cfb0 con 0x7f40981599d0 2024-03-07T06:51:02.295 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.294+0000 7f408d400640 1 --2- 172.21.15.133:0/1946374038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40981599d0 0x7f40980c8f30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f408000d5f0 tx=0x7f408000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:02.295 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.294+0000 7f408ea00640 1 -- 172.21.15.133:0/1946374038 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40800125f0 con 0x7f40981599d0 2024-03-07T06:51:02.295 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.294+0000 7f408ea00640 1 -- 172.21.15.133:0/1946374038 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4080019070 con 0x7f40981599d0 2024-03-07T06:51:02.295 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.294+0000 7f408ea00640 1 -- 172.21.15.133:0/1946374038 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40800129d0 con 0x7f40981599d0 2024-03-07T06:51:02.295 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.294+0000 7f409d200640 1 -- 172.21.15.133:0/1946374038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409806d290 con 0x7f40981599d0 2024-03-07T06:51:02.295 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.294+0000 7f409d200640 1 -- 172.21.15.133:0/1946374038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409806d850 con 0x7f40981599d0 2024-03-07T06:51:02.296 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.295+0000 7f409d200640 1 -- 172.21.15.133:0/1946374038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409804f3c0 con 0x7f40981599d0 2024-03-07T06:51:02.296 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.296+0000 7f408ea00640 1 -- 172.21.15.133:0/1946374038 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) v1 ==== 89722+0+0 (secure 0 0 0) 0x7f4080012b70 con 0x7f40981599d0 2024-03-07T06:51:02.296 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.297+0000 7f408ea00640 1 --2- 172.21.15.133:0/1946374038 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f407406b440 0x7f407406d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:02.296 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.297+0000 7f408ea00640 1 -- 172.21.15.133:0/1946374038 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f4080089750 con 0x7f40981599d0 2024-03-07T06:51:02.297 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.297+0000 7f4095e00640 1 --2- 172.21.15.133:0/1946374038 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f407406b440 0x7f407406d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:02.298 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.298+0000 7f4095e00640 1 --2- 172.21.15.133:0/1946374038 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f407406b440 0x7f407406d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f40880011e0 tx=0x7f4088009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:02.301 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.302+0000 7f408ea00640 1 -- 172.21.15.133:0/1946374038 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4080059430 con 0x7f40981599d0 2024-03-07T06:51:02.481 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.482+0000 7f409d200640 1 -- 172.21.15.133:0/1946374038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40980ce430 con 0x7f40981599d0 2024-03-07T06:51:02.482 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.483+0000 7f408ea00640 1 -- 172.21.15.133:0/1946374038 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f408005cc40 con 0x7f40981599d0 2024-03-07T06:51:02.483 INFO:teuthology.misc.health.smithi133.stdout: 2024-03-07T06:51:02.483 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":12,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:50:59.492289+0000","last_up_change":"0.000000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:02.486 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.486+0000 7f409d200640 1 -- 172.21.15.133:0/1946374038 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f407406b440 msgr2=0x7f407406d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:02.486 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.486+0000 7f409d200640 1 --2- 172.21.15.133:0/1946374038 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f407406b440 0x7f407406d900 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f40880011e0 tx=0x7f4088009040 comp rx=0 tx=0).stop 2024-03-07T06:51:02.486 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.486+0000 7f409d200640 1 -- 172.21.15.133:0/1946374038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40981599d0 msgr2=0x7f40980c8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:02.486 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.486+0000 7f409d200640 1 --2- 172.21.15.133:0/1946374038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40981599d0 0x7f40980c8f30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f408000d5f0 tx=0x7f408000dab0 comp rx=0 tx=0).stop 2024-03-07T06:51:02.486 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.486+0000 7f409d200640 1 -- 172.21.15.133:0/1946374038 shutdown_connections 2024-03-07T06:51:02.486 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.486+0000 7f409d200640 1 --2- 172.21.15.133:0/1946374038 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4098160510 0x7f40980c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:02.487 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.486+0000 7f409d200640 1 --2- 172.21.15.133:0/1946374038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40981599d0 0x7f40980c8f30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:02.487 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.486+0000 7f409d200640 1 --2- 172.21.15.133:0/1946374038 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f407406b440 0x7f407406d900 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:02.487 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.486+0000 7f409d200640 1 --2- 172.21.15.133:0/1946374038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40980d0460 0x7f409806c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:02.487 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.486+0000 7f409d200640 1 -- 172.21.15.133:0/1946374038 >> 172.21.15.133:0/1946374038 conn(0x7f409806b440 msgr2=0x7f409815c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:02.487 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.486+0000 7f409d200640 1 -- 172.21.15.133:0/1946374038 shutdown_connections 2024-03-07T06:51:02.487 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:02.486+0000 7f409d200640 1 -- 172.21.15.133:0/1946374038 wait complete. 2024-03-07T06:51:02.498 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-07T06:51:09.306 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:09.492 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.492+0000 7fab37c00640 1 Processor -- start 2024-03-07T06:51:09.492 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.492+0000 7fab37c00640 1 -- start start 2024-03-07T06:51:09.494 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.493+0000 7fab37c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3013de90 0x7fab3013e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:09.494 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.494+0000 7fab37c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab30071210 0x7fab300cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:09.494 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.494+0000 7fab37c00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab3015edd0 con 0x7fab300d0460 2024-03-07T06:51:09.494 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.494+0000 7fab37c00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab3013ea20 con 0x7fab30071210 2024-03-07T06:51:09.494 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.494+0000 7fab37c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab300d08a0 con 0x7fab3013de90 2024-03-07T06:51:09.494 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.494+0000 7fab2fe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fab30071210 0x7fab300cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:09.495 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.494+0000 7fab35c00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fab300d0460 0x7fab301408a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55902/0 (socket says 172.21.15.133:55902) 2024-03-07T06:51:09.495 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.494+0000 7fab35c00640 1 -- 172.21.15.133:0/3841732913 learned_addr learned my addr 172.21.15.133:0/3841732913 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:09.495 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.494+0000 7fab35200640 1 --2- 172.21.15.133:0/3841732913 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3013de90 0x7fab3013e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:09.495 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.494+0000 7fab2fe00640 1 -- 172.21.15.133:0/3841732913 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3013de90 msgr2=0x7fab3013e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:09.495 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.494+0000 7fab2fe00640 1 --2- 172.21.15.133:0/3841732913 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3013de90 0x7fab3013e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.495 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.494+0000 7fab2fe00640 1 -- 172.21.15.133:0/3841732913 >> v1:172.21.15.133:6789/0 conn(0x7fab300d0460 legacy=0x7fab301408a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:09.495 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.494+0000 7fab2fe00640 1 -- 172.21.15.133:0/3841732913 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3015f0b0 con 0x7fab30071210 2024-03-07T06:51:09.495 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.494+0000 7fab2f400640 1 -- 172.21.15.133:0/3841732913 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731306291 0 0) 0x7fab3015edd0 con 0x7fab300d0460 2024-03-07T06:51:09.496 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.496+0000 7fab2fe00640 1 --2- 172.21.15.133:0/3841732913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab30071210 0x7fab300cff20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0097a0 tx=0x7fab1c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffcfdfa3518fc60 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:09.496 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.496+0000 7fab2f400640 1 -- 172.21.15.133:0/3841732913 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c031780 con 0x7fab30071210 2024-03-07T06:51:09.496 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.496+0000 7fab2f400640 1 -- 172.21.15.133:0/3841732913 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c037070 con 0x7fab30071210 2024-03-07T06:51:09.497 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.496+0000 7fab2f400640 1 -- 172.21.15.133:0/3841732913 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c031b40 con 0x7fab30071210 2024-03-07T06:51:09.497 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.496+0000 7fab37c00640 1 -- 172.21.15.133:0/3841732913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab30071210 msgr2=0x7fab300cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:09.497 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.496+0000 7fab37c00640 1 --2- 172.21.15.133:0/3841732913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab30071210 0x7fab300cff20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0097a0 tx=0x7fab1c02b820 comp rx=0 tx=0).stop 2024-03-07T06:51:09.497 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.497+0000 7fab37c00640 1 -- 172.21.15.133:0/3841732913 shutdown_connections 2024-03-07T06:51:09.497 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.497+0000 7fab37c00640 1 --2- 172.21.15.133:0/3841732913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab30071210 0x7fab300cff20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.497 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.497+0000 7fab37c00640 1 --2- 172.21.15.133:0/3841732913 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3013de90 0x7fab3013e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.497 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.497+0000 7fab37c00640 1 -- 172.21.15.133:0/3841732913 >> 172.21.15.133:0/3841732913 conn(0x7fab3006b440 msgr2=0x7fab3006b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:09.497 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.497+0000 7fab37c00640 1 -- 172.21.15.133:0/3841732913 shutdown_connections 2024-03-07T06:51:09.497 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.497+0000 7fab37c00640 1 -- 172.21.15.133:0/3841732913 wait complete. 2024-03-07T06:51:09.498 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.498+0000 7fab37c00640 1 Processor -- start 2024-03-07T06:51:09.498 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.498+0000 7fab37c00640 1 -- start start 2024-03-07T06:51:09.500 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.499+0000 7fab37c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab30071210 0x7fab30070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:09.500 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.499+0000 7fab37c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab300d0460 0x7fab3014f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:09.500 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.499+0000 7fab37c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3013de90 0x7fab300b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:09.500 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.499+0000 7fab37c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab30071650 con 0x7fab300d0460 2024-03-07T06:51:09.501 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.499+0000 7fab37c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3015f670 con 0x7fab30071210 2024-03-07T06:51:09.501 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.499+0000 7fab37c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3015ea80 con 0x7fab3013de90 2024-03-07T06:51:09.501 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.500+0000 7fab2fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab300d0460 0x7fab3014f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:09.501 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.500+0000 7fab2fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab300d0460 0x7fab3014f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42534/0 (socket says 172.21.15.133:42534) 2024-03-07T06:51:09.501 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.500+0000 7fab2fe00640 1 -- 172.21.15.133:0/532173493 learned_addr learned my addr 172.21.15.133:0/532173493 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:09.501 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.500+0000 7fab35c00640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3013de90 0x7fab300b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:09.501 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.500+0000 7fab35200640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab30071210 0x7fab30070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:09.501 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.500+0000 7fab2fe00640 1 -- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3013de90 msgr2=0x7fab300b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:09.501 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.500+0000 7fab2fe00640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3013de90 0x7fab300b3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.501 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.500+0000 7fab2fe00640 1 -- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab30071210 msgr2=0x7fab30070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:09.501 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.500+0000 7fab2fe00640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab30071210 0x7fab30070cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.501 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.500+0000 7fab2fe00640 1 -- 172.21.15.133:0/532173493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3014fb40 con 0x7fab300d0460 2024-03-07T06:51:09.501 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.500+0000 7fab35c00640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3013de90 0x7fab300b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:51:09.501 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.500+0000 7fab2fe00640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab300d0460 0x7fab3014f600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fab1c034040 tx=0x7fab1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:09.502 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.501+0000 7fab2cc00640 1 -- 172.21.15.133:0/532173493 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c004230 con 0x7fab300d0460 2024-03-07T06:51:09.502 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.501+0000 7fab2cc00640 1 -- 172.21.15.133:0/532173493 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c0043d0 con 0x7fab300d0460 2024-03-07T06:51:09.502 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.501+0000 7fab2cc00640 1 -- 172.21.15.133:0/532173493 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c003850 con 0x7fab300d0460 2024-03-07T06:51:09.502 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.501+0000 7fab37c00640 1 -- 172.21.15.133:0/532173493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab300b4740 con 0x7fab300d0460 2024-03-07T06:51:09.502 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.501+0000 7fab37c00640 1 -- 172.21.15.133:0/532173493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3015d3e0 con 0x7fab300d0460 2024-03-07T06:51:09.502 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.502+0000 7fab37c00640 1 -- 172.21.15.133:0/532173493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3004f3c0 con 0x7fab300d0460 2024-03-07T06:51:09.503 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.503+0000 7fab2cc00640 1 -- 172.21.15.133:0/532173493 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fab1c0039f0 con 0x7fab300d0460 2024-03-07T06:51:09.503 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.504+0000 7fab2cc00640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fab1006b4d0 0x7fab1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:09.503 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.504+0000 7fab2cc00640 1 -- 172.21.15.133:0/532173493 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fab1c0abdc0 con 0x7fab300d0460 2024-03-07T06:51:09.503 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.504+0000 7fab35200640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fab1006b4d0 0x7fab1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:09.504 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.505+0000 7fab35200640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fab1006b4d0 0x7fab1006d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fab24006fd0 tx=0x7fab24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:09.509 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.509+0000 7fab2cc00640 1 -- 172.21.15.133:0/532173493 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fab1c041070 con 0x7fab300d0460 2024-03-07T06:51:09.709 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.709+0000 7fab37c00640 1 -- 172.21.15.133:0/532173493 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab3006ba70 con 0x7fab300d0460 2024-03-07T06:51:09.710 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.710+0000 7fab2cc00640 1 -- 172.21.15.133:0/532173493 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12932 (secure 0 0 0) 0x7fab1c02f050 con 0x7fab300d0460 2024-03-07T06:51:09.710 INFO:teuthology.misc.health.smithi133.stdout: 2024-03-07T06:51:09.710 INFO:teuthology.misc.health.smithi133.stdout:{"epoch":17,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:06.829749+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:09.713 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.713+0000 7fab37c00640 1 -- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fab1006b4d0 msgr2=0x7fab1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:09.713 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.713+0000 7fab37c00640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fab1006b4d0 0x7fab1006d990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fab24006fd0 tx=0x7fab24008040 comp rx=0 tx=0).stop 2024-03-07T06:51:09.713 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.713+0000 7fab37c00640 1 -- 172.21.15.133:0/532173493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab300d0460 msgr2=0x7fab3014f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:09.713 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.713+0000 7fab37c00640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab300d0460 0x7fab3014f600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fab1c034040 tx=0x7fab1c0046c0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.713 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.713+0000 7fab37c00640 1 -- 172.21.15.133:0/532173493 shutdown_connections 2024-03-07T06:51:09.713 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.713+0000 7fab37c00640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3013de90 0x7fab300b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.714 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.713+0000 7fab37c00640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab300d0460 0x7fab3014f600 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.714 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.713+0000 7fab37c00640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fab1006b4d0 0x7fab1006d990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.714 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.713+0000 7fab37c00640 1 --2- 172.21.15.133:0/532173493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab30071210 0x7fab30070cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.714 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.713+0000 7fab37c00640 1 -- 172.21.15.133:0/532173493 >> 172.21.15.133:0/532173493 conn(0x7fab3006b440 msgr2=0x7fab3013f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:09.714 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.713+0000 7fab37c00640 1 -- 172.21.15.133:0/532173493 shutdown_connections 2024-03-07T06:51:09.714 INFO:teuthology.misc.health.smithi133.stderr:2024-03-07T06:51:09.713+0000 7fab37c00640 1 -- 172.21.15.133:0/532173493 wait complete. 2024-03-07T06:51:09.727 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-07T06:51:09.727 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-07T06:51:09.728 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-07T06:51:09.728 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-07T06:51:09.735 INFO:tasks.ceph.mds.a:Started 2024-03-07T06:51:09.735 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-07T06:51:09.735 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-07T06:51:09.742 INFO:tasks.ceph.mds.c:Started 2024-03-07T06:51:09.742 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-07T06:51:09.742 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-07T06:51:09.750 INFO:tasks.ceph.mds.b:Started 2024-03-07T06:51:09.750 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-07T06:51:09.750 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-07T06:51:09.758 INFO:tasks.ceph.mds.d:Started 2024-03-07T06:51:09.758 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-07T06:51:09.758 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:09.882 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-03-07T06:51:09.891 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-03-07T06:51:09.911 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2024-03-07T06:51:09.916 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2024-03-07T06:51:09.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.938+0000 7f7d79800640 1 Processor -- start 2024-03-07T06:51:09.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.938+0000 7f7d79800640 1 -- start start 2024-03-07T06:51:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.939+0000 7f7d79800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7d74071d20 0x7f7d740724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.939+0000 7f7d79800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7d74073050 0x7f7d740ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.939+0000 7f7d79800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d740cdf80 con 0x7f7d7415e390 2024-03-07T06:51:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.939+0000 7f7d79800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d740ce4d0 con 0x7f7d74071d20 2024-03-07T06:51:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.939+0000 7f7d79800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7415e770 con 0x7f7d74073050 2024-03-07T06:51:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.940+0000 7f7d72000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7d74071d20 0x7f7d740724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.940+0000 7f7d72000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7d74071d20 0x7f7d740724b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42556/0 (socket says 172.21.15.133:42556) 2024-03-07T06:51:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.940+0000 7f7d72000640 1 -- 172.21.15.133:0/1632978280 learned_addr learned my addr 172.21.15.133:0/1632978280 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.940+0000 7f7d72a00640 1 --1- 172.21.15.133:0/1632978280 >> v1:172.21.15.133:6789/0 conn(0x7f7d7415e390 0x7f7d74071610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55920/0 (socket says 172.21.15.133:55920) 2024-03-07T06:51:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.940+0000 7f7d73400640 1 --2- 172.21.15.133:0/1632978280 >> v2:172.21.15.161:3300/0 conn(0x7f7d74073050 0x7f7d740ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.940+0000 7f7d72000640 1 -- 172.21.15.133:0/1632978280 >> v2:172.21.15.161:3300/0 conn(0x7f7d74073050 msgr2=0x7f7d740ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.940+0000 7f7d72000640 1 --2- 172.21.15.133:0/1632978280 >> v2:172.21.15.161:3300/0 conn(0x7f7d74073050 0x7f7d740ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.940+0000 7f7d72000640 1 -- 172.21.15.133:0/1632978280 >> v1:172.21.15.133:6789/0 conn(0x7f7d7415e390 legacy=0x7f7d74071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:09.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.940+0000 7f7d72000640 1 -- 172.21.15.133:0/1632978280 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d740b3d90 con 0x7f7d74071d20 2024-03-07T06:51:09.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.940+0000 7f7d71600640 1 -- 172.21.15.133:0/1632978280 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624429168 0 0) 0x7f7d740cdf80 con 0x7f7d7415e390 2024-03-07T06:51:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.941+0000 7f7d72000640 1 --2- 172.21.15.133:0/1632978280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d74071d20 0x7f7d740724b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7d74068cd0 tx=0x7f7d6002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c458b07723f3414a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.941+0000 7f7d71600640 1 -- 172.21.15.133:0/1632978280 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60034070 con 0x7f7d74071d20 2024-03-07T06:51:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.941+0000 7f7d71600640 1 -- 172.21.15.133:0/1632978280 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d60030860 con 0x7f7d74071d20 2024-03-07T06:51:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.941+0000 7f7d71600640 1 -- 172.21.15.133:0/1632978280 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d60030c30 con 0x7f7d74071d20 2024-03-07T06:51:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.942+0000 7f7d79800640 1 -- 172.21.15.133:0/1632978280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d74071d20 msgr2=0x7f7d740724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.942+0000 7f7d79800640 1 --2- 172.21.15.133:0/1632978280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d74071d20 0x7f7d740724b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7d74068cd0 tx=0x7f7d6002b9a0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.942+0000 7f7d79800640 1 -- 172.21.15.133:0/1632978280 shutdown_connections 2024-03-07T06:51:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.942+0000 7f7d79800640 1 --2- 172.21.15.133:0/1632978280 >> v2:172.21.15.161:3300/0 conn(0x7f7d74073050 0x7f7d740ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.942+0000 7f7d79800640 1 --2- 172.21.15.133:0/1632978280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d74071d20 0x7f7d740724b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.942+0000 7f7d79800640 1 -- 172.21.15.133:0/1632978280 >> 172.21.15.133:0/1632978280 conn(0x7f7d740b3120 msgr2=0x7f7d740b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.942+0000 7f7d79800640 1 -- 172.21.15.133:0/1632978280 shutdown_connections 2024-03-07T06:51:09.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.942+0000 7f7d79800640 1 -- 172.21.15.133:0/1632978280 wait complete. 2024-03-07T06:51:09.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.944+0000 7f7d79800640 1 Processor -- start 2024-03-07T06:51:09.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.944+0000 7f7d79800640 1 -- start start 2024-03-07T06:51:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d79800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d74071d20 0x7f7d74140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d79800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d74073050 0x7f7d740b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d79800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d7415e390 0x7f7d74160f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d79800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7406d2a0 con 0x7f7d74071d20 2024-03-07T06:51:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d79800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d740b4070 con 0x7f7d74073050 2024-03-07T06:51:09.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d79800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7406d420 con 0x7f7d7415e390 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d72a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d74071d20 0x7f7d74140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d72a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d74071d20 0x7f7d74140c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42564/0 (socket says 172.21.15.133:42564) 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d72a00640 1 -- 172.21.15.133:0/1853835126 learned_addr learned my addr 172.21.15.133:0/1853835126 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d73400640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d7415e390 0x7f7d74160f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d72000640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d74073050 0x7f7d740b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d72a00640 1 -- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d7415e390 msgr2=0x7f7d74160f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d72a00640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d7415e390 0x7f7d74160f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d72a00640 1 -- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d74073050 msgr2=0x7f7d740b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d72a00640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d74073050 0x7f7d740b43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.945+0000 7f7d72a00640 1 -- 172.21.15.133:0/1853835126 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7415fed0 con 0x7f7d74071d20 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.946+0000 7f7d72000640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d74073050 0x7f7d740b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.946+0000 7f7d72a00640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d74071d20 0x7f7d74140c70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400c460 tx=0x7f7d6400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.946+0000 7f7d6aa00640 1 -- 172.21.15.133:0/1853835126 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64018070 con 0x7f7d74071d20 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.946+0000 7f7d6aa00640 1 -- 172.21.15.133:0/1853835126 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d64007500 con 0x7f7d74071d20 2024-03-07T06:51:09.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.946+0000 7f7d79800640 1 -- 172.21.15.133:0/1853835126 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d741471a0 con 0x7f7d74071d20 2024-03-07T06:51:09.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.946+0000 7f7d6aa00640 1 -- 172.21.15.133:0/1853835126 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64004a20 con 0x7f7d74071d20 2024-03-07T06:51:09.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.946+0000 7f7d79800640 1 -- 172.21.15.133:0/1853835126 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d74160450 con 0x7f7d74071d20 2024-03-07T06:51:09.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.947+0000 7f7d79800640 1 -- 172.21.15.133:0/1853835126 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d74062c90 con 0x7f7d74071d20 2024-03-07T06:51:09.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.948+0000 7f7d6aa00640 1 -- 172.21.15.133:0/1853835126 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7d6401b070 con 0x7f7d74071d20 2024-03-07T06:51:09.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.949+0000 7f7d6aa00640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7d5406b4d0 0x7f7d5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:09.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.949+0000 7f7d6aa00640 1 -- 172.21.15.133:0/1853835126 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7d64089170 con 0x7f7d74071d20 2024-03-07T06:51:09.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.949+0000 7f7d72000640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7d5406b4d0 0x7f7d5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:09.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.950+0000 7f7d72000640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7d5406b4d0 0x7f7d5406d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7d60036000 tx=0x7f7d60008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:09.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:09.954+0000 7f7d6aa00640 1 -- 172.21.15.133:0/1853835126 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7d64057c50 con 0x7f7d74071d20 2024-03-07T06:51:10.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.149+0000 7f7d79800640 1 -- 172.21.15.133:0/1853835126 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d74074260 con 0x7f7d74071d20 2024-03-07T06:51:10.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.151+0000 7f7d6aa00640 1 -- 172.21.15.133:0/1853835126 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12932 (secure 0 0 0) 0x7f7d6405bcf0 con 0x7f7d74071d20 2024-03-07T06:51:10.151 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:10.151 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:06.829749+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:10.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.154+0000 7f7d79800640 1 -- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7d5406b4d0 msgr2=0x7f7d5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:10.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.154+0000 7f7d79800640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7d5406b4d0 0x7f7d5406d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7d60036000 tx=0x7f7d60008030 comp rx=0 tx=0).stop 2024-03-07T06:51:10.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.154+0000 7f7d79800640 1 -- 172.21.15.133:0/1853835126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d74071d20 msgr2=0x7f7d74140c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:10.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.154+0000 7f7d79800640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d74071d20 0x7f7d74140c70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400c460 tx=0x7f7d6400c920 comp rx=0 tx=0).stop 2024-03-07T06:51:10.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.154+0000 7f7d79800640 1 -- 172.21.15.133:0/1853835126 shutdown_connections 2024-03-07T06:51:10.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.154+0000 7f7d79800640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d7415e390 0x7f7d74160f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.154+0000 7f7d79800640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7d5406b4d0 0x7f7d5406d990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.154+0000 7f7d79800640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d74073050 0x7f7d740b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.154+0000 7f7d79800640 1 --2- 172.21.15.133:0/1853835126 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d74071d20 0x7f7d74140c70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.154+0000 7f7d79800640 1 -- 172.21.15.133:0/1853835126 >> 172.21.15.133:0/1853835126 conn(0x7f7d740b3120 msgr2=0x7f7d74071610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:10.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.154+0000 7f7d79800640 1 -- 172.21.15.133:0/1853835126 shutdown_connections 2024-03-07T06:51:10.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.154+0000 7f7d79800640 1 -- 172.21.15.133:0/1853835126 wait complete. 2024-03-07T06:51:10.172 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:51:10.172 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:51:10.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.377+0000 7f638e200640 1 Processor -- start 2024-03-07T06:51:10.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.377+0000 7f638e200640 1 -- start start 2024-03-07T06:51:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.379+0000 7f638e200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f638815e0e0 0x7f638806e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.379+0000 7f638e200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6388072c40 0x7f638806ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.379+0000 7f638e200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63880b3560 con 0x7f6388157f20 2024-03-07T06:51:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.379+0000 7f638e200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638815e5e0 con 0x7f6388072c40 2024-03-07T06:51:10.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.379+0000 7f638e200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f638815e8c0 con 0x7f638815e0e0 2024-03-07T06:51:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.379+0000 7f6387e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6388072c40 0x7f638806ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.379+0000 7f6387e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6388072c40 0x7f638806ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42572/0 (socket says 172.21.15.133:42572) 2024-03-07T06:51:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.379+0000 7f6387e00640 1 -- 172.21.15.133:0/2617323940 learned_addr learned my addr 172.21.15.133:0/2617323940 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.380+0000 7f637ea00640 1 --2- 172.21.15.133:0/2617323940 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f638815e0e0 0x7f638806e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.380+0000 7f6387e00640 1 -- 172.21.15.133:0/2617323940 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f638815e0e0 msgr2=0x7f638806e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:10.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.380+0000 7f6387e00640 1 --2- 172.21.15.133:0/2617323940 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f638815e0e0 0x7f638806e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.380+0000 7f6387e00640 1 -- 172.21.15.133:0/2617323940 >> v1:172.21.15.161:6789/0 conn(0x7f6388157f20 legacy=0x7f63880b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:10.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.380+0000 7f6387e00640 1 -- 172.21.15.133:0/2617323940 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638815eba0 con 0x7f6388072c40 2024-03-07T06:51:10.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.381+0000 7f6387e00640 1 --2- 172.21.15.133:0/2617323940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6388072c40 0x7f638806ed60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6380009870 tx=0x7f638002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888fabdb7e7b5792 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:10.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.382+0000 7f6386a00640 1 -- 172.21.15.133:0/2617323940 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380034070 con 0x7f6388072c40 2024-03-07T06:51:10.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.382+0000 7f6386a00640 1 -- 172.21.15.133:0/2617323940 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6380030840 con 0x7f6388072c40 2024-03-07T06:51:10.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.382+0000 7f6386a00640 1 -- 172.21.15.133:0/2617323940 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380030c10 con 0x7f6388072c40 2024-03-07T06:51:10.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.382+0000 7f638e200640 1 -- 172.21.15.133:0/2617323940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6388072c40 msgr2=0x7f638806ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:10.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.382+0000 7f638e200640 1 --2- 172.21.15.133:0/2617323940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6388072c40 0x7f638806ed60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6380009870 tx=0x7f638002b980 comp rx=0 tx=0).stop 2024-03-07T06:51:10.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.383+0000 7f638e200640 1 -- 172.21.15.133:0/2617323940 shutdown_connections 2024-03-07T06:51:10.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.383+0000 7f638e200640 1 --2- 172.21.15.133:0/2617323940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6388072c40 0x7f638806ed60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.383+0000 7f638e200640 1 --2- 172.21.15.133:0/2617323940 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f638815e0e0 0x7f638806e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.383+0000 7f638e200640 1 -- 172.21.15.133:0/2617323940 >> 172.21.15.133:0/2617323940 conn(0x7f63880b1d30 msgr2=0x7f63880b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:10.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.383+0000 7f638e200640 1 -- 172.21.15.133:0/2617323940 shutdown_connections 2024-03-07T06:51:10.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.383+0000 7f638e200640 1 -- 172.21.15.133:0/2617323940 wait complete. 2024-03-07T06:51:10.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.385+0000 7f638e200640 1 Processor -- start 2024-03-07T06:51:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.385+0000 7f638e200640 1 -- start start 2024-03-07T06:51:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.386+0000 7f638e200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6388072c40 0x7f638814f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.386+0000 7f638e200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6388157f20 0x7f638814f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.386+0000 7f638e200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638815e0e0 0x7f6388150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.386+0000 7f638e200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6388158300 con 0x7f638815e0e0 2024-03-07T06:51:10.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.386+0000 7f638e200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638815aaf0 con 0x7f6388157f20 2024-03-07T06:51:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.386+0000 7f638e200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6388158480 con 0x7f6388072c40 2024-03-07T06:51:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.387+0000 7f6387e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638815e0e0 0x7f6388150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.387+0000 7f6387e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638815e0e0 0x7f6388150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42574/0 (socket says 172.21.15.133:42574) 2024-03-07T06:51:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.387+0000 7f6387e00640 1 -- 172.21.15.133:0/3662872340 learned_addr learned my addr 172.21.15.133:0/3662872340 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.387+0000 7f637ea00640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6388157f20 0x7f638814f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:10.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.387+0000 7f6387400640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6388072c40 0x7f638814f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:10.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.387+0000 7f6387e00640 1 -- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6388072c40 msgr2=0x7f638814f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:10.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.387+0000 7f6387e00640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6388072c40 0x7f638814f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.387+0000 7f6387e00640 1 -- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6388157f20 msgr2=0x7f638814f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:10.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.387+0000 7f6387e00640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6388157f20 0x7f638814f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.387+0000 7f6387e00640 1 -- 172.21.15.133:0/3662872340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388151ea0 con 0x7f638815e0e0 2024-03-07T06:51:10.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.387+0000 7f6387400640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6388072c40 0x7f638814f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.387+0000 7f6387e00640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638815e0e0 0x7f6388150030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6380009870 tx=0x7f6380004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.388+0000 7f637fe00640 1 -- 172.21.15.133:0/3662872340 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380048070 con 0x7f638815e0e0 2024-03-07T06:51:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.388+0000 7f637fe00640 1 -- 172.21.15.133:0/3662872340 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6380030640 con 0x7f638815e0e0 2024-03-07T06:51:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.388+0000 7f637fe00640 1 -- 172.21.15.133:0/3662872340 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6380034040 con 0x7f638815e0e0 2024-03-07T06:51:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.388+0000 7f638e200640 1 -- 172.21.15.133:0/3662872340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6388152180 con 0x7f638815e0e0 2024-03-07T06:51:10.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.388+0000 7f638e200640 1 -- 172.21.15.133:0/3662872340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63880abfe0 con 0x7f638815e0e0 2024-03-07T06:51:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.389+0000 7f638e200640 1 -- 172.21.15.133:0/3662872340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6388062c90 con 0x7f638815e0e0 2024-03-07T06:51:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.390+0000 7f637fe00640 1 -- 172.21.15.133:0/3662872340 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6380037070 con 0x7f638815e0e0 2024-03-07T06:51:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.391+0000 7f637fe00640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f636406b500 0x7f636406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.392+0000 7f637fe00640 1 -- 172.21.15.133:0/3662872340 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f63800abb30 con 0x7f638815e0e0 2024-03-07T06:51:10.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.392+0000 7f6387400640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f636406b500 0x7f636406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:10.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.393+0000 7f6387400640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f636406b500 0x7f636406d9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6378001d70 tx=0x7f6378009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:10.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.396+0000 7f637fe00640 1 -- 172.21.15.133:0/3662872340 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f638007ac60 con 0x7f638815e0e0 2024-03-07T06:51:10.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.590+0000 7f638e200640 1 -- 172.21.15.133:0/3662872340 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63880b2d80 con 0x7f638815e0e0 2024-03-07T06:51:10.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.591+0000 7f637fe00640 1 -- 172.21.15.133:0/3662872340 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f638007a560 con 0x7f638815e0e0 2024-03-07T06:51:10.591 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:51:10.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.593+0000 7f638e200640 1 -- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f636406b500 msgr2=0x7f636406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:10.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.593+0000 7f638e200640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f636406b500 0x7f636406d9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6378001d70 tx=0x7f6378009040 comp rx=0 tx=0).stop 2024-03-07T06:51:10.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.593+0000 7f638e200640 1 -- 172.21.15.133:0/3662872340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638815e0e0 msgr2=0x7f6388150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:10.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.593+0000 7f638e200640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638815e0e0 0x7f6388150030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6380009870 tx=0x7f6380004730 comp rx=0 tx=0).stop 2024-03-07T06:51:10.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.594+0000 7f638e200640 1 -- 172.21.15.133:0/3662872340 shutdown_connections 2024-03-07T06:51:10.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.594+0000 7f638e200640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f638815e0e0 0x7f6388150030 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.594+0000 7f638e200640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f636406b500 0x7f636406d9c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.594+0000 7f638e200640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6388157f20 0x7f638814f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.594+0000 7f638e200640 1 --2- 172.21.15.133:0/3662872340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6388072c40 0x7f638814f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.594+0000 7f638e200640 1 -- 172.21.15.133:0/3662872340 >> 172.21.15.133:0/3662872340 conn(0x7f63880b1d30 msgr2=0x7f63880b3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:10.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.594+0000 7f638e200640 1 -- 172.21.15.133:0/3662872340 shutdown_connections 2024-03-07T06:51:10.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.594+0000 7f638e200640 1 -- 172.21.15.133:0/3662872340 wait complete. 2024-03-07T06:51:10.612 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:51:10.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.807+0000 7fd3e9800640 1 Processor -- start 2024-03-07T06:51:10.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.807+0000 7fd3e9800640 1 -- start start 2024-03-07T06:51:10.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.809+0000 7fd3e9800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd3e4162030 0x7fd3e415e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:10.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.809+0000 7fd3e9800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd3e40d3490 0x7fd3e40d38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:10.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.809+0000 7fd3e9800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e4141c60 con 0x7fd3e41416d0 2024-03-07T06:51:10.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.809+0000 7fd3e9800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e40b44b0 con 0x7fd3e40d3490 2024-03-07T06:51:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.809+0000 7fd3e9800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3e40d3e00 con 0x7fd3e4162030 2024-03-07T06:51:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.809+0000 7fd3e3400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd3e40d3490 0x7fd3e40d38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.809+0000 7fd3e2000640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd3e41416d0 0x7fd3e4162410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55924/0 (socket says 172.21.15.133:55924) 2024-03-07T06:51:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.809+0000 7fd3e2000640 1 -- 172.21.15.133:0/4168927757 learned_addr learned my addr 172.21.15.133:0/4168927757 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.809+0000 7fd3e2a00640 1 --2- 172.21.15.133:0/4168927757 >> v2:172.21.15.161:3300/0 conn(0x7fd3e4162030 0x7fd3e415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.810+0000 7fd3e3400640 1 -- 172.21.15.133:0/4168927757 >> v2:172.21.15.161:3300/0 conn(0x7fd3e4162030 msgr2=0x7fd3e415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.810+0000 7fd3e3400640 1 --2- 172.21.15.133:0/4168927757 >> v2:172.21.15.161:3300/0 conn(0x7fd3e4162030 0x7fd3e415e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.810+0000 7fd3e3400640 1 -- 172.21.15.133:0/4168927757 >> v1:172.21.15.133:6789/0 conn(0x7fd3e41416d0 legacy=0x7fd3e4162410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.810+0000 7fd3e3400640 1 -- 172.21.15.133:0/4168927757 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e415ecd0 con 0x7fd3e40d3490 2024-03-07T06:51:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.810+0000 7fd3e1600640 1 -- 172.21.15.133:0/4168927757 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3183666187 0 0) 0x7fd3e4141c60 con 0x7fd3e41416d0 2024-03-07T06:51:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.811+0000 7fd3e3400640 1 --2- 172.21.15.133:0/4168927757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e40d3490 0x7fd3e40d38c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40b4630 tx=0x7fd3dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94faec66f7836f22 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.811+0000 7fd3e1600640 1 -- 172.21.15.133:0/4168927757 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc035020 con 0x7fd3e40d3490 2024-03-07T06:51:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.811+0000 7fd3e1600640 1 -- 172.21.15.133:0/4168927757 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc031830 con 0x7fd3e40d3490 2024-03-07T06:51:10.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.811+0000 7fd3e1600640 1 -- 172.21.15.133:0/4168927757 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc031c00 con 0x7fd3e40d3490 2024-03-07T06:51:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.811+0000 7fd3e9800640 1 -- 172.21.15.133:0/4168927757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e40d3490 msgr2=0x7fd3e40d38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.811+0000 7fd3e9800640 1 --2- 172.21.15.133:0/4168927757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e40d3490 0x7fd3e40d38c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd3e40b4630 tx=0x7fd3dc02b8d0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.812+0000 7fd3e9800640 1 -- 172.21.15.133:0/4168927757 shutdown_connections 2024-03-07T06:51:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.812+0000 7fd3e9800640 1 --2- 172.21.15.133:0/4168927757 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e40d3490 0x7fd3e40d38c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.812+0000 7fd3e9800640 1 --2- 172.21.15.133:0/4168927757 >> v2:172.21.15.161:3300/0 conn(0x7fd3e4162030 0x7fd3e415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.812+0000 7fd3e9800640 1 -- 172.21.15.133:0/4168927757 >> 172.21.15.133:0/4168927757 conn(0x7fd3e4070400 msgr2=0x7fd3e40d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.812+0000 7fd3e9800640 1 -- 172.21.15.133:0/4168927757 shutdown_connections 2024-03-07T06:51:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.812+0000 7fd3e9800640 1 -- 172.21.15.133:0/4168927757 wait complete. 2024-03-07T06:51:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.813+0000 7fd3e9800640 1 Processor -- start 2024-03-07T06:51:10.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.813+0000 7fd3e9800640 1 -- start start 2024-03-07T06:51:10.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.814+0000 7fd3e9800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3e40d3490 0x7fd3e4154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.814+0000 7fd3e9800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3e41416d0 0x7fd3e41557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.814+0000 7fd3e9800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e4162030 0x7fd3e415a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.814+0000 7fd3e9800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e4162b20 con 0x7fd3e4162030 2024-03-07T06:51:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.814+0000 7fd3e9800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e415f290 con 0x7fd3e40d3490 2024-03-07T06:51:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.814+0000 7fd3e9800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e415f570 con 0x7fd3e41416d0 2024-03-07T06:51:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.814+0000 7fd3e3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e4162030 0x7fd3e415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.815+0000 7fd3e3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e4162030 0x7fd3e415a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42592/0 (socket says 172.21.15.133:42592) 2024-03-07T06:51:10.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.815+0000 7fd3e3400640 1 -- 172.21.15.133:0/4227403870 learned_addr learned my addr 172.21.15.133:0/4227403870 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.815+0000 7fd3e2a00640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3e40d3490 0x7fd3e4154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.815+0000 7fd3e2000640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3e41416d0 0x7fd3e41557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.815+0000 7fd3e3400640 1 -- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3e41416d0 msgr2=0x7fd3e41557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.815+0000 7fd3e3400640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3e41416d0 0x7fd3e41557a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.815+0000 7fd3e3400640 1 -- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3e40d3490 msgr2=0x7fd3e4154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.815+0000 7fd3e3400640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3e40d3490 0x7fd3e4154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.815+0000 7fd3e3400640 1 -- 172.21.15.133:0/4227403870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e4154ab0 con 0x7fd3e4162030 2024-03-07T06:51:10.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.815+0000 7fd3e2a00640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3e40d3490 0x7fd3e4154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.815+0000 7fd3e3400640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e4162030 0x7fd3e415a830 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc034040 tx=0x7fd3dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.815+0000 7fd3daa00640 1 -- 172.21.15.133:0/4227403870 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc041070 con 0x7fd3e4162030 2024-03-07T06:51:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.816+0000 7fd3daa00640 1 -- 172.21.15.133:0/4227403870 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc004230 con 0x7fd3e4162030 2024-03-07T06:51:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.816+0000 7fd3daa00640 1 -- 172.21.15.133:0/4227403870 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc035040 con 0x7fd3e4162030 2024-03-07T06:51:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.816+0000 7fd3e9800640 1 -- 172.21.15.133:0/4227403870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e415ca80 con 0x7fd3e4162030 2024-03-07T06:51:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.816+0000 7fd3e9800640 1 -- 172.21.15.133:0/4227403870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e4159d10 con 0x7fd3e4162030 2024-03-07T06:51:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.817+0000 7fd3e9800640 1 -- 172.21.15.133:0/4227403870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e4062c90 con 0x7fd3e4162030 2024-03-07T06:51:10.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.818+0000 7fd3daa00640 1 -- 172.21.15.133:0/4227403870 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd3dc038020 con 0x7fd3e4162030 2024-03-07T06:51:10.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.818+0000 7fd3daa00640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd3c406b4d0 0x7fd3c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:10.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.818+0000 7fd3daa00640 1 -- 172.21.15.133:0/4227403870 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd3dc0ab900 con 0x7fd3e4162030 2024-03-07T06:51:10.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.819+0000 7fd3e2a00640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd3c406b4d0 0x7fd3c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:10.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.820+0000 7fd3e2a00640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd3c406b4d0 0x7fd3c406d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4008890 tx=0x7fd3d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:10.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:10.823+0000 7fd3daa00640 1 -- 172.21.15.133:0/4227403870 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd3dc07aad0 con 0x7fd3e4162030 2024-03-07T06:51:11.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.041+0000 7fd3e9800640 1 -- 172.21.15.133:0/4227403870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3e40b6f70 con 0x7fd3e4162030 2024-03-07T06:51:11.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.042+0000 7fd3daa00640 1 -- 172.21.15.133:0/4227403870 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3706 (secure 0 0 0) 0x7fd3dc07a3d0 con 0x7fd3e4162030 2024-03-07T06:51:11.042 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:11.042 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1116649196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1116649196},{"type":"v1","addr":"172.21.15.161:6835","nonce":1116649196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3485625105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3485625105},{"type":"v1","addr":"172.21.15.133:6835","nonce":3485625105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3866978630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3866978630},{"type":"v1","addr":"172.21.15.133:6833","nonce":3866978630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/2885273912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2885273912},{"type":"v1","addr":"172.21.15.161:6837","nonce":2885273912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-07T06:51:11.042 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 3 2024-03-07T06:51:11.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.045+0000 7fd3e9800640 1 -- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd3c406b4d0 msgr2=0x7fd3c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:11.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.045+0000 7fd3e9800640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd3c406b4d0 0x7fd3c406d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4008890 tx=0x7fd3d4008040 comp rx=0 tx=0).stop 2024-03-07T06:51:11.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.045+0000 7fd3e9800640 1 -- 172.21.15.133:0/4227403870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e4162030 msgr2=0x7fd3e415a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:11.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.045+0000 7fd3e9800640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e4162030 0x7fd3e415a830 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc034040 tx=0x7fd3dc0046c0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.045+0000 7fd3e9800640 1 -- 172.21.15.133:0/4227403870 shutdown_connections 2024-03-07T06:51:11.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.045+0000 7fd3e9800640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3e4162030 0x7fd3e415a830 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.045+0000 7fd3e9800640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3e41416d0 0x7fd3e41557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.045+0000 7fd3e9800640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd3c406b4d0 0x7fd3c406d990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.045+0000 7fd3e9800640 1 --2- 172.21.15.133:0/4227403870 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3e40d3490 0x7fd3e4154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.045+0000 7fd3e9800640 1 -- 172.21.15.133:0/4227403870 >> 172.21.15.133:0/4227403870 conn(0x7fd3e4070400 msgr2=0x7fd3e415dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:11.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.045+0000 7fd3e9800640 1 -- 172.21.15.133:0/4227403870 shutdown_connections 2024-03-07T06:51:11.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.045+0000 7fd3e9800640 1 -- 172.21.15.133:0/4227403870 wait complete. 2024-03-07T06:51:11.063 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:11.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.258+0000 7f8699200640 1 Processor -- start 2024-03-07T06:51:11.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.258+0000 7f8699200640 1 -- start start 2024-03-07T06:51:11.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.260+0000 7f8699200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86940b5dd0 0x7f8694144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:11.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.260+0000 7f8699200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8694071410 0x7f8694071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:11.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.260+0000 7f8699200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8694162c80 con 0x7f86940b43f0 2024-03-07T06:51:11.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.260+0000 7f8699200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8694141700 con 0x7f8694071410 2024-03-07T06:51:11.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.260+0000 7f8699200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8694161e00 con 0x7f86940b5dd0 2024-03-07T06:51:11.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.260+0000 7f8692800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8694071410 0x7f8694071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:11.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.260+0000 7f8692800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8694071410 0x7f8694071850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46180/0 (socket says 172.21.15.133:46180) 2024-03-07T06:51:11.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.260+0000 7f8692800640 1 -- 172.21.15.133:0/1566523402 learned_addr learned my addr 172.21.15.133:0/1566523402 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:11.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.261+0000 7f8691e00640 1 --2- 172.21.15.133:0/1566523402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86940b5dd0 0x7f8694144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:11.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.261+0000 7f8692800640 1 -- 172.21.15.133:0/1566523402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86940b5dd0 msgr2=0x7f8694144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:11.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.261+0000 7f8692800640 1 --2- 172.21.15.133:0/1566523402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86940b5dd0 0x7f8694144540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.261+0000 7f8692800640 1 -- 172.21.15.133:0/1566523402 >> v1:172.21.15.161:6789/0 conn(0x7f86940b43f0 legacy=0x7f8694162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:11.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.261+0000 7f8692800640 1 -- 172.21.15.133:0/1566523402 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940b4bd0 con 0x7f8694071410 2024-03-07T06:51:11.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.262+0000 7f8692800640 1 --2- 172.21.15.133:0/1566523402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8694071410 0x7f8694071850 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f868c009870 tx=0x7f868c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2853936a6c98c8f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:11.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.262+0000 7f8690a00640 1 -- 172.21.15.133:0/1566523402 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c0317c0 con 0x7f8694071410 2024-03-07T06:51:11.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.262+0000 7f8690a00640 1 -- 172.21.15.133:0/1566523402 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c037070 con 0x7f8694071410 2024-03-07T06:51:11.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.262+0000 7f8690a00640 1 -- 172.21.15.133:0/1566523402 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c031b80 con 0x7f8694071410 2024-03-07T06:51:11.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.263+0000 7f8699200640 1 -- 172.21.15.133:0/1566523402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8694071410 msgr2=0x7f8694071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:11.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.263+0000 7f8699200640 1 --2- 172.21.15.133:0/1566523402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8694071410 0x7f8694071850 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f868c009870 tx=0x7f868c02b860 comp rx=0 tx=0).stop 2024-03-07T06:51:11.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.263+0000 7f8699200640 1 -- 172.21.15.133:0/1566523402 shutdown_connections 2024-03-07T06:51:11.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.263+0000 7f8699200640 1 --2- 172.21.15.133:0/1566523402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8694071410 0x7f8694071850 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.263+0000 7f8699200640 1 --2- 172.21.15.133:0/1566523402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86940b5dd0 0x7f8694144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.263+0000 7f8699200640 1 -- 172.21.15.133:0/1566523402 >> 172.21.15.133:0/1566523402 conn(0x7f86940d0e70 msgr2=0x7f86940d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:11.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.263+0000 7f8699200640 1 -- 172.21.15.133:0/1566523402 shutdown_connections 2024-03-07T06:51:11.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.263+0000 7f8699200640 1 -- 172.21.15.133:0/1566523402 wait complete. 2024-03-07T06:51:11.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.265+0000 7f8699200640 1 Processor -- start 2024-03-07T06:51:11.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.265+0000 7f8699200640 1 -- start start 2024-03-07T06:51:11.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8699200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8694071410 0x7f869415abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:11.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8699200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86940b43f0 0x7f8694158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:11.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8699200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86940b5dd0 0x7f869415d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:11.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8699200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86940b5190 con 0x7f86940b43f0 2024-03-07T06:51:11.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8699200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8694071d90 con 0x7f8694071410 2024-03-07T06:51:11.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8699200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8694162050 con 0x7f86940b5dd0 2024-03-07T06:51:11.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8691400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86940b43f0 0x7f8694158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8691400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86940b43f0 0x7f8694158eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46184/0 (socket says 172.21.15.133:46184) 2024-03-07T06:51:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8691400640 1 -- 172.21.15.133:0/726314061 learned_addr learned my addr 172.21.15.133:0/726314061 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8691400640 1 -- 172.21.15.133:0/726314061 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86940b5dd0 msgr2=0x7f869415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8691400640 1 --2- 172.21.15.133:0/726314061 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86940b5dd0 0x7f869415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8691e00640 1 --2- 172.21.15.133:0/726314061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8694071410 0x7f869415abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8691400640 1 -- 172.21.15.133:0/726314061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8694071410 msgr2=0x7f869415abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8691400640 1 --2- 172.21.15.133:0/726314061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8694071410 0x7f869415abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.266+0000 7f8691400640 1 -- 172.21.15.133:0/726314061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869415b120 con 0x7f86940b43f0 2024-03-07T06:51:11.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.267+0000 7f8691400640 1 --2- 172.21.15.133:0/726314061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86940b43f0 0x7f8694158eb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f868400c7c0 tx=0x7f868400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:11.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.267+0000 7f868a000640 1 -- 172.21.15.133:0/726314061 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684016020 con 0x7f86940b43f0 2024-03-07T06:51:11.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.267+0000 7f868a000640 1 -- 172.21.15.133:0/726314061 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868400aa50 con 0x7f86940b43f0 2024-03-07T06:51:11.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.267+0000 7f868a000640 1 -- 172.21.15.133:0/726314061 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684012420 con 0x7f86940b43f0 2024-03-07T06:51:11.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.267+0000 7f8699200640 1 -- 172.21.15.133:0/726314061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869415ec80 con 0x7f86940b43f0 2024-03-07T06:51:11.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.267+0000 7f8699200640 1 -- 172.21.15.133:0/726314061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86941599c0 con 0x7f86940b43f0 2024-03-07T06:51:11.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.268+0000 7f8699200640 1 -- 172.21.15.133:0/726314061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8694062c90 con 0x7f86940b43f0 2024-03-07T06:51:11.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.269+0000 7f868a000640 1 -- 172.21.15.133:0/726314061 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8684019020 con 0x7f86940b43f0 2024-03-07T06:51:11.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.270+0000 7f868a000640 1 --2- 172.21.15.133:0/726314061 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f867006b4d0 0x7f867006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:11.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.271+0000 7f868a000640 1 -- 172.21.15.133:0/726314061 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8684089c20 con 0x7f86940b43f0 2024-03-07T06:51:11.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.271+0000 7f8691e00640 1 --2- 172.21.15.133:0/726314061 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f867006b4d0 0x7f867006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:11.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.272+0000 7f8691e00640 1 --2- 172.21.15.133:0/726314061 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f867006b4d0 0x7f867006d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f867c00b7c0 tx=0x7f867c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:11.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.275+0000 7f868a000640 1 -- 172.21.15.133:0/726314061 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8684020070 con 0x7f86940b43f0 2024-03-07T06:51:11.470 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.470+0000 7f8699200640 1 -- 172.21.15.133:0/726314061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86940b8a10 con 0x7f86940b43f0 2024-03-07T06:51:11.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.471+0000 7f868a000640 1 -- 172.21.15.133:0/726314061 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12932 (secure 0 0 0) 0x7f869415b120 con 0x7f86940b43f0 2024-03-07T06:51:11.471 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:11.471 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:06.829749+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:11.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.474+0000 7f8699200640 1 -- 172.21.15.133:0/726314061 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f867006b4d0 msgr2=0x7f867006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:11.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.474+0000 7f8699200640 1 --2- 172.21.15.133:0/726314061 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f867006b4d0 0x7f867006d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f867c00b7c0 tx=0x7f867c008040 comp rx=0 tx=0).stop 2024-03-07T06:51:11.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.474+0000 7f8699200640 1 -- 172.21.15.133:0/726314061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86940b43f0 msgr2=0x7f8694158eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:11.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.474+0000 7f8699200640 1 --2- 172.21.15.133:0/726314061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86940b43f0 0x7f8694158eb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f868400c7c0 tx=0x7f868400cc80 comp rx=0 tx=0).stop 2024-03-07T06:51:11.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.474+0000 7f8699200640 1 -- 172.21.15.133:0/726314061 shutdown_connections 2024-03-07T06:51:11.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.474+0000 7f8699200640 1 --2- 172.21.15.133:0/726314061 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86940b5dd0 0x7f869415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.474+0000 7f8699200640 1 --2- 172.21.15.133:0/726314061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86940b43f0 0x7f8694158eb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.474+0000 7f8699200640 1 --2- 172.21.15.133:0/726314061 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f867006b4d0 0x7f867006d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.474+0000 7f8699200640 1 --2- 172.21.15.133:0/726314061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8694071410 0x7f869415abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.474+0000 7f8699200640 1 -- 172.21.15.133:0/726314061 >> 172.21.15.133:0/726314061 conn(0x7f86940d0e70 msgr2=0x7f8694144210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:11.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.474+0000 7f8699200640 1 -- 172.21.15.133:0/726314061 shutdown_connections 2024-03-07T06:51:11.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.474+0000 7f8699200640 1 -- 172.21.15.133:0/726314061 wait complete. 2024-03-07T06:51:11.492 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:51:11.492 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:51:11.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.706+0000 7efcf4000640 1 Processor -- start 2024-03-07T06:51:11.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.706+0000 7efcf4000640 1 -- start start 2024-03-07T06:51:11.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.708+0000 7efcf4000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efcec0aec80 0x7efcec0b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:11.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.708+0000 7efcf4000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efcec06e3f0 0x7efcec06e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:11.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.708+0000 7efcf4000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcec06f450 con 0x7efcec0cf620 2024-03-07T06:51:11.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.708+0000 7efcf4000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcec06f170 con 0x7efcec0aec80 2024-03-07T06:51:11.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.708+0000 7efcf4000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcec071ad0 con 0x7efcec06e3f0 2024-03-07T06:51:11.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.708+0000 7efcf0c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efcec0aec80 0x7efcec0b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:11.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.708+0000 7efcf0c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7efcec0aec80 0x7efcec0b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46188/0 (socket says 172.21.15.133:46188) 2024-03-07T06:51:11.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.708+0000 7efcf0c00640 1 -- 172.21.15.133:0/1488069519 learned_addr learned my addr 172.21.15.133:0/1488069519 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:11.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.708+0000 7efcf2000640 1 --2- 172.21.15.133:0/1488069519 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efcec06e3f0 0x7efcec06e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:11.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.708+0000 7efcf0c00640 1 -- 172.21.15.133:0/1488069519 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efcec06e3f0 msgr2=0x7efcec06e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:11.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.708+0000 7efcf0c00640 1 --2- 172.21.15.133:0/1488069519 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efcec06e3f0 0x7efcec06e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.708+0000 7efcf0c00640 1 -- 172.21.15.133:0/1488069519 >> v1:172.21.15.161:6789/0 conn(0x7efcec0cf620 legacy=0x7efcec0ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:11.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.708+0000 7efcf0c00640 1 -- 172.21.15.133:0/1488069519 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcec071db0 con 0x7efcec0aec80 2024-03-07T06:51:11.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.710+0000 7efcf0c00640 1 --2- 172.21.15.133:0/1488069519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcec0aec80 0x7efcec0b2e50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009870 tx=0x7efcd802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472121b003d21c6f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:11.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.710+0000 7efcebe00640 1 -- 172.21.15.133:0/1488069519 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd80317c0 con 0x7efcec0aec80 2024-03-07T06:51:11.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.710+0000 7efcebe00640 1 -- 172.21.15.133:0/1488069519 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd8037070 con 0x7efcec0aec80 2024-03-07T06:51:11.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.710+0000 7efcebe00640 1 -- 172.21.15.133:0/1488069519 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8031b80 con 0x7efcec0aec80 2024-03-07T06:51:11.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.711+0000 7efcf4000640 1 -- 172.21.15.133:0/1488069519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcec0aec80 msgr2=0x7efcec0b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:11.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.711+0000 7efcf4000640 1 --2- 172.21.15.133:0/1488069519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcec0aec80 0x7efcec0b2e50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009870 tx=0x7efcd802b860 comp rx=0 tx=0).stop 2024-03-07T06:51:11.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.711+0000 7efcf4000640 1 -- 172.21.15.133:0/1488069519 shutdown_connections 2024-03-07T06:51:11.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.711+0000 7efcf4000640 1 --2- 172.21.15.133:0/1488069519 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efcec06e3f0 0x7efcec06e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.711+0000 7efcf4000640 1 --2- 172.21.15.133:0/1488069519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcec0aec80 0x7efcec0b2e50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.711+0000 7efcf4000640 1 -- 172.21.15.133:0/1488069519 >> 172.21.15.133:0/1488069519 conn(0x7efcec0b1d30 msgr2=0x7efcec0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:11.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.711+0000 7efcf4000640 1 -- 172.21.15.133:0/1488069519 shutdown_connections 2024-03-07T06:51:11.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.711+0000 7efcf4000640 1 -- 172.21.15.133:0/1488069519 wait complete. 2024-03-07T06:51:11.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.713+0000 7efcf4000640 1 Processor -- start 2024-03-07T06:51:11.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.713+0000 7efcf4000640 1 -- start start 2024-03-07T06:51:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf4000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efcec06e3f0 0x7efcec13f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf4000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcec0aec80 0x7efcec154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf4000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcec0cf620 0x7efcec152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf4000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec06eee0 con 0x7efcec0aec80 2024-03-07T06:51:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf4000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec06ed60 con 0x7efcec0cf620 2024-03-07T06:51:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf4000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcec0725b0 con 0x7efcec06e3f0 2024-03-07T06:51:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf0c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcec0aec80 0x7efcec154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf0c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcec0aec80 0x7efcec154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46194/0 (socket says 172.21.15.133:46194) 2024-03-07T06:51:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf0c00640 1 -- 172.21.15.133:0/1710577891 learned_addr learned my addr 172.21.15.133:0/1710577891 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:11.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf1600640 1 --2- 172.21.15.133:0/1710577891 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efcec06e3f0 0x7efcec13f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:11.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf2000640 1 --2- 172.21.15.133:0/1710577891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcec0cf620 0x7efcec152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:11.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf0c00640 1 -- 172.21.15.133:0/1710577891 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efcec06e3f0 msgr2=0x7efcec13f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:11.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf0c00640 1 --2- 172.21.15.133:0/1710577891 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efcec06e3f0 0x7efcec13f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf0c00640 1 -- 172.21.15.133:0/1710577891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcec0cf620 msgr2=0x7efcec152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:11.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf0c00640 1 --2- 172.21.15.133:0/1710577891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcec0cf620 0x7efcec152960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.714+0000 7efcf0c00640 1 -- 172.21.15.133:0/1710577891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcec155a50 con 0x7efcec0aec80 2024-03-07T06:51:11.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.715+0000 7efcf0c00640 1 --2- 172.21.15.133:0/1710577891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcec0aec80 0x7efcec154290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efcd8000c00 tx=0x7efcd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:11.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.715+0000 7efce9600640 1 -- 172.21.15.133:0/1710577891 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8032860 con 0x7efcec0aec80 2024-03-07T06:51:11.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.715+0000 7efce9600640 1 -- 172.21.15.133:0/1710577891 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd8032a20 con 0x7efcec0aec80 2024-03-07T06:51:11.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.715+0000 7efce9600640 1 -- 172.21.15.133:0/1710577891 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8032e10 con 0x7efcec0aec80 2024-03-07T06:51:11.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.715+0000 7efcf4000640 1 -- 172.21.15.133:0/1710577891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcec140ca0 con 0x7efcec0aec80 2024-03-07T06:51:11.716 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.715+0000 7efcf4000640 1 -- 172.21.15.133:0/1710577891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcec141260 con 0x7efcec0aec80 2024-03-07T06:51:11.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.716+0000 7efcf4000640 1 -- 172.21.15.133:0/1710577891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcec062c90 con 0x7efcec0aec80 2024-03-07T06:51:11.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.717+0000 7efce9600640 1 -- 172.21.15.133:0/1710577891 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7efcd804c060 con 0x7efcec0aec80 2024-03-07T06:51:11.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.718+0000 7efce9600640 1 --2- 172.21.15.133:0/1710577891 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7efccc06b4d0 0x7efccc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:11.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.719+0000 7efce9600640 1 -- 172.21.15.133:0/1710577891 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efcd80ab7b0 con 0x7efcec0aec80 2024-03-07T06:51:11.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.719+0000 7efcf1600640 1 --2- 172.21.15.133:0/1710577891 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7efccc06b4d0 0x7efccc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:11.720 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.720+0000 7efcf1600640 1 --2- 172.21.15.133:0/1710577891 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7efccc06b4d0 0x7efccc06d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efce0006fd0 tx=0x7efce0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:11.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.723+0000 7efce9600640 1 -- 172.21.15.133:0/1710577891 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7efcd8040070 con 0x7efcec0aec80 2024-03-07T06:51:11.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.921+0000 7efcf4000640 1 -- 172.21.15.133:0/1710577891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efcec15deb0 con 0x7efcec0aec80 2024-03-07T06:51:11.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.922+0000 7efce9600640 1 -- 172.21.15.133:0/1710577891 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7efcd804c350 con 0x7efcec0aec80 2024-03-07T06:51:11.922 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:51:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.924+0000 7efcf4000640 1 -- 172.21.15.133:0/1710577891 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7efccc06b4d0 msgr2=0x7efccc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.924+0000 7efcf4000640 1 --2- 172.21.15.133:0/1710577891 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7efccc06b4d0 0x7efccc06d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efce0006fd0 tx=0x7efce0008040 comp rx=0 tx=0).stop 2024-03-07T06:51:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.924+0000 7efcf4000640 1 -- 172.21.15.133:0/1710577891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcec0aec80 msgr2=0x7efcec154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.924+0000 7efcf4000640 1 --2- 172.21.15.133:0/1710577891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcec0aec80 0x7efcec154290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efcd8000c00 tx=0x7efcd80046c0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.925+0000 7efcf4000640 1 -- 172.21.15.133:0/1710577891 shutdown_connections 2024-03-07T06:51:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.925+0000 7efcf4000640 1 --2- 172.21.15.133:0/1710577891 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7efccc06b4d0 0x7efccc06d990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.925+0000 7efcf4000640 1 --2- 172.21.15.133:0/1710577891 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efcec0cf620 0x7efcec152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.925+0000 7efcf4000640 1 --2- 172.21.15.133:0/1710577891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efcec0aec80 0x7efcec154290 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.925+0000 7efcf4000640 1 --2- 172.21.15.133:0/1710577891 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efcec06e3f0 0x7efcec13f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.925+0000 7efcf4000640 1 -- 172.21.15.133:0/1710577891 >> 172.21.15.133:0/1710577891 conn(0x7efcec0b1d30 msgr2=0x7efcec15f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:11.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.925+0000 7efcf4000640 1 -- 172.21.15.133:0/1710577891 shutdown_connections 2024-03-07T06:51:11.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:11.925+0000 7efcf4000640 1 -- 172.21.15.133:0/1710577891 wait complete. 2024-03-07T06:51:11.944 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-07T06:51:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.157+0000 7fa982400640 1 Processor -- start 2024-03-07T06:51:12.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.158+0000 7fa982400640 1 -- start start 2024-03-07T06:51:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.159+0000 7fa982400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa97c15fbb0 0x7fa97c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.159+0000 7fa982400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa97c072950 0x7fa97c0cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.159+0000 7fa982400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa97c15ff90 0x7fa97c0acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.159+0000 7fa982400640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97c072f60 con 0x7fa97c15fbb0 2024-03-07T06:51:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.159+0000 7fa982400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa97c073490 con 0x7fa97c072950 2024-03-07T06:51:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.159+0000 7fa982400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c0731b0 con 0x7fa97c15ff90 2024-03-07T06:51:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.159+0000 7fa97b400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa97c15fbb0 0x7fa97c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.159+0000 7fa97b400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa97c15fbb0 0x7fa97c0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46198/0 (socket says 172.21.15.133:46198) 2024-03-07T06:51:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.159+0000 7fa97b400640 1 -- 172.21.15.133:0/118589686 learned_addr learned my addr 172.21.15.133:0/118589686 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.159+0000 7fa973e00640 1 --2- 172.21.15.133:0/118589686 >> v2:172.21.15.161:3300/0 conn(0x7fa97c072950 0x7fa97c0cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.159+0000 7fa97be00640 1 --2- 172.21.15.133:0/118589686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa97c15ff90 0x7fa97c0acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.160+0000 7fa97b400640 1 -- 172.21.15.133:0/118589686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa97c15ff90 msgr2=0x7fa97c0acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.160+0000 7fa97b400640 1 --2- 172.21.15.133:0/118589686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa97c15ff90 0x7fa97c0acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.160+0000 7fa97b400640 1 -- 172.21.15.133:0/118589686 >> v2:172.21.15.161:3300/0 conn(0x7fa97c072950 msgr2=0x7fa97c0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.160+0000 7fa97b400640 1 --2- 172.21.15.133:0/118589686 >> v2:172.21.15.161:3300/0 conn(0x7fa97c072950 0x7fa97c0cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.160+0000 7fa97b400640 1 -- 172.21.15.133:0/118589686 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c072150 con 0x7fa97c15fbb0 2024-03-07T06:51:12.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.161+0000 7fa97b400640 1 --2- 172.21.15.133:0/118589686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa97c15fbb0 0x7fa97c0cf7d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0cdd10 tx=0x7fa96c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00b6d893c5d865e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:12.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.161+0000 7fa97aa00640 1 -- 172.21.15.133:0/118589686 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c02e070 con 0x7fa97c15fbb0 2024-03-07T06:51:12.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.161+0000 7fa97aa00640 1 -- 172.21.15.133:0/118589686 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96c037070 con 0x7fa97c15fbb0 2024-03-07T06:51:12.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.161+0000 7fa97aa00640 1 -- 172.21.15.133:0/118589686 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c031a00 con 0x7fa97c15fbb0 2024-03-07T06:51:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.162+0000 7fa982400640 1 -- 172.21.15.133:0/118589686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa97c15fbb0 msgr2=0x7fa97c0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.162+0000 7fa982400640 1 --2- 172.21.15.133:0/118589686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa97c15fbb0 0x7fa97c0cf7d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0cdd10 tx=0x7fa96c02b880 comp rx=0 tx=0).stop 2024-03-07T06:51:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.162+0000 7fa982400640 1 -- 172.21.15.133:0/118589686 shutdown_connections 2024-03-07T06:51:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.162+0000 7fa982400640 1 --2- 172.21.15.133:0/118589686 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa97c15ff90 0x7fa97c0acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.162+0000 7fa982400640 1 --2- 172.21.15.133:0/118589686 >> v2:172.21.15.161:3300/0 conn(0x7fa97c072950 0x7fa97c0cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.162+0000 7fa982400640 1 --2- 172.21.15.133:0/118589686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa97c15fbb0 0x7fa97c0cf7d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.162+0000 7fa982400640 1 -- 172.21.15.133:0/118589686 >> 172.21.15.133:0/118589686 conn(0x7fa97c0b3120 msgr2=0x7fa97c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:12.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.162+0000 7fa982400640 1 -- 172.21.15.133:0/118589686 shutdown_connections 2024-03-07T06:51:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.162+0000 7fa982400640 1 -- 172.21.15.133:0/118589686 wait complete. 2024-03-07T06:51:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.164+0000 7fa982400640 1 Processor -- start 2024-03-07T06:51:12.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.164+0000 7fa982400640 1 -- start start 2024-03-07T06:51:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.165+0000 7fa982400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa97c15fbb0 0x7fa97c145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.165+0000 7fa982400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa97c15ff90 0x7fa97c15e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.165+0000 7fa982400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa97c15e5f0 0x7fa97c1f74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.165+0000 7fa982400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c0b1a00 con 0x7fa97c15fbb0 2024-03-07T06:51:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.165+0000 7fa982400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c160980 con 0x7fa97c15ff90 2024-03-07T06:51:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.165+0000 7fa982400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97c1603b0 con 0x7fa97c15e5f0 2024-03-07T06:51:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.165+0000 7fa97b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa97c15fbb0 0x7fa97c145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:12.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.165+0000 7fa97b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa97c15fbb0 0x7fa97c145020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46202/0 (socket says 172.21.15.133:46202) 2024-03-07T06:51:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.165+0000 7fa97b400640 1 -- 172.21.15.133:0/3437888249 learned_addr learned my addr 172.21.15.133:0/3437888249 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.166+0000 7fa973e00640 1 --2- 172.21.15.133:0/3437888249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa97c15ff90 0x7fa97c15e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.166+0000 7fa97be00640 1 --2- 172.21.15.133:0/3437888249 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa97c15e5f0 0x7fa97c1f74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.166+0000 7fa97b400640 1 -- 172.21.15.133:0/3437888249 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa97c15e5f0 msgr2=0x7fa97c1f74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.166+0000 7fa97b400640 1 --2- 172.21.15.133:0/3437888249 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa97c15e5f0 0x7fa97c1f74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.166+0000 7fa97b400640 1 -- 172.21.15.133:0/3437888249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa97c15ff90 msgr2=0x7fa97c15e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.166+0000 7fa97b400640 1 --2- 172.21.15.133:0/3437888249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa97c15ff90 0x7fa97c15e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.166+0000 7fa97b400640 1 -- 172.21.15.133:0/3437888249 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97c15eaa0 con 0x7fa97c15fbb0 2024-03-07T06:51:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.166+0000 7fa97b400640 1 --2- 172.21.15.133:0/3437888249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa97c15fbb0 0x7fa97c145020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa96c02b8c0 tx=0x7fa96c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.166+0000 7fa973400640 1 -- 172.21.15.133:0/3437888249 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c032880 con 0x7fa97c15fbb0 2024-03-07T06:51:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.166+0000 7fa973400640 1 -- 172.21.15.133:0/3437888249 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96c032a40 con 0x7fa97c15fbb0 2024-03-07T06:51:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.166+0000 7fa973400640 1 -- 172.21.15.133:0/3437888249 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c032e30 con 0x7fa97c15fbb0 2024-03-07T06:51:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.166+0000 7fa982400640 1 -- 172.21.15.133:0/3437888249 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa97c153d50 con 0x7fa97c15fbb0 2024-03-07T06:51:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.167+0000 7fa982400640 1 -- 172.21.15.133:0/3437888249 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97c15f000 con 0x7fa97c15fbb0 2024-03-07T06:51:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.168+0000 7fa982400640 1 -- 172.21.15.133:0/3437888249 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97c062c90 con 0x7fa97c15fbb0 2024-03-07T06:51:12.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.168+0000 7fa973400640 1 -- 172.21.15.133:0/3437888249 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa96c04c060 con 0x7fa97c15fbb0 2024-03-07T06:51:12.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.169+0000 7fa973400640 1 --2- 172.21.15.133:0/3437888249 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa95806b520 0x7fa95806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.169+0000 7fa973400640 1 -- 172.21.15.133:0/3437888249 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa96c0abaf0 con 0x7fa97c15fbb0 2024-03-07T06:51:12.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.170+0000 7fa973e00640 1 --2- 172.21.15.133:0/3437888249 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa95806b520 0x7fa95806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:12.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.171+0000 7fa973e00640 1 --2- 172.21.15.133:0/3437888249 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa95806b520 0x7fa95806d9e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa964006fd0 tx=0x7fa964008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:12.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.174+0000 7fa973400640 1 -- 172.21.15.133:0/3437888249 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa97c15eaa0 con 0x7fa97c15fbb0 2024-03-07T06:51:12.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.371+0000 7fa982400640 1 -- 172.21.15.133:0/3437888249 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa97c0cf1c0 con 0x7fa97c15fbb0 2024-03-07T06:51:12.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.372+0000 7fa973400640 1 -- 172.21.15.133:0/3437888249 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa96c040070 con 0x7fa97c15fbb0 2024-03-07T06:51:12.372 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:12.372 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-03-07T06:51:12.372 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:51:12.372 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-03-07T06:51:12.372 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-03-07T06:51:12.372 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:51:12.373 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-03-07T06:51:12.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.375+0000 7fa982400640 1 -- 172.21.15.133:0/3437888249 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa95806b520 msgr2=0x7fa95806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:12.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.375+0000 7fa982400640 1 --2- 172.21.15.133:0/3437888249 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa95806b520 0x7fa95806d9e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa964006fd0 tx=0x7fa964008040 comp rx=0 tx=0).stop 2024-03-07T06:51:12.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.375+0000 7fa982400640 1 -- 172.21.15.133:0/3437888249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa97c15fbb0 msgr2=0x7fa97c145020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:12.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.375+0000 7fa982400640 1 --2- 172.21.15.133:0/3437888249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa97c15fbb0 0x7fa97c145020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa96c02b8c0 tx=0x7fa96c0046c0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.375+0000 7fa982400640 1 -- 172.21.15.133:0/3437888249 shutdown_connections 2024-03-07T06:51:12.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.375+0000 7fa982400640 1 --2- 172.21.15.133:0/3437888249 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa97c15e5f0 0x7fa97c1f74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.375+0000 7fa982400640 1 --2- 172.21.15.133:0/3437888249 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa95806b520 0x7fa95806d9e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.375+0000 7fa982400640 1 --2- 172.21.15.133:0/3437888249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa97c15ff90 0x7fa97c15e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.375+0000 7fa982400640 1 --2- 172.21.15.133:0/3437888249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa97c15fbb0 0x7fa97c145020 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.375+0000 7fa982400640 1 -- 172.21.15.133:0/3437888249 >> 172.21.15.133:0/3437888249 conn(0x7fa97c0b3120 msgr2=0x7fa97c0b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:12.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.375+0000 7fa982400640 1 -- 172.21.15.133:0/3437888249 shutdown_connections 2024-03-07T06:51:12.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.376+0000 7fa982400640 1 -- 172.21.15.133:0/3437888249 wait complete. 2024-03-07T06:51:12.394 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-07T06:51:12.394 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-07T06:51:12.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.612+0000 7f76c8200640 1 Processor -- start 2024-03-07T06:51:12.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.612+0000 7f76c8200640 1 -- start start 2024-03-07T06:51:12.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.613+0000 7f76c8200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f76c00d0310 0x7f76c015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.613+0000 7f76c8200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c0160560 con 0x7f76c015faa0 2024-03-07T06:51:12.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.613+0000 7f76c8200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c0160840 con 0x7f76c0159b90 2024-03-07T06:51:12.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.613+0000 7f76c8200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c00ce3c0 con 0x7f76c00d0310 2024-03-07T06:51:12.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.614+0000 7f76c4e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f76c015faa0 0x7f76c00cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53138/0 (socket says 172.21.15.133:53138) 2024-03-07T06:51:12.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.614+0000 7f76c4e00640 1 -- 172.21.15.133:0/3209383008 learned_addr learned my addr 172.21.15.133:0/3209383008 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:12.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.614+0000 7f76c6200640 1 --2- 172.21.15.133:0/3209383008 >> v2:172.21.15.161:3300/0 conn(0x7f76c00d0310 0x7f76c015ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:12.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.614+0000 7f76bfe00640 1 -- 172.21.15.133:0/3209383008 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684645670 0 0) 0x7f76c0160560 con 0x7f76c015faa0 2024-03-07T06:51:12.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.614+0000 7f76bfe00640 1 -- 172.21.15.133:0/3209383008 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76a4003520 con 0x7f76c015faa0 2024-03-07T06:51:12.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.614+0000 7f76bfe00640 1 -- 172.21.15.133:0/3209383008 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 385409477 0 0) 0x7f76a4003520 con 0x7f76c015faa0 2024-03-07T06:51:12.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.614+0000 7f76bfe00640 1 -- 172.21.15.133:0/3209383008 >> v2:172.21.15.161:3300/0 conn(0x7f76c00d0310 msgr2=0x7f76c015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:12.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.614+0000 7f76bfe00640 1 --2- 172.21.15.133:0/3209383008 >> v2:172.21.15.161:3300/0 conn(0x7f76c00d0310 0x7f76c015ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.614+0000 7f76bfe00640 1 -- 172.21.15.133:0/3209383008 >> v1:172.21.15.161:6789/0 conn(0x7f76c0159b90 legacy=0x7f76c00cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:12.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.614+0000 7f76bfe00640 1 -- 172.21.15.133:0/3209383008 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c00ce5f0 con 0x7f76c015faa0 2024-03-07T06:51:12.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.614+0000 7f76bfe00640 1 -- 172.21.15.133:0/3209383008 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f76ac0019f0 con 0x7f76c015faa0 2024-03-07T06:51:12.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76bfe00640 1 -- 172.21.15.133:0/3209383008 >> v1:172.21.15.133:6789/0 conn(0x7f76c015faa0 legacy=0x7f76c00cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:12.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76bfe00640 1 --2- 172.21.15.133:0/3209383008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76a4003a00 0x7f76a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76bfe00640 1 --2- 172.21.15.133:0/3209383008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a4004800 0x7f76a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76bfe00640 1 --2- 172.21.15.133:0/3209383008 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76a40051c0 0x7f76a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76bfe00640 1 -- 172.21.15.133:0/3209383008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a400c090 con 0x7f76a4004800 2024-03-07T06:51:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76bfe00640 1 -- 172.21.15.133:0/3209383008 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a400c240 con 0x7f76a4003a00 2024-03-07T06:51:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76bfe00640 1 -- 172.21.15.133:0/3209383008 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a400c440 con 0x7f76a40051c0 2024-03-07T06:51:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76c4e00640 1 --2- 172.21.15.133:0/3209383008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a4004800 0x7f76a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76c4e00640 1 -- 172.21.15.133:0/3209383008 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76a40051c0 msgr2=0x7f76a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76c4e00640 1 --2- 172.21.15.133:0/3209383008 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76a40051c0 0x7f76a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76c4e00640 1 -- 172.21.15.133:0/3209383008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76a4003a00 msgr2=0x7f76a4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76c5800640 1 --2- 172.21.15.133:0/3209383008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76a4003a00 0x7f76a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76c4e00640 1 --2- 172.21.15.133:0/3209383008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76a4003a00 0x7f76a4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76c4e00640 1 -- 172.21.15.133:0/3209383008 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a4003520 con 0x7f76a4004800 2024-03-07T06:51:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76c6200640 1 -- 172.21.15.133:0/3209383008 reap_dead start 2024-03-07T06:51:12.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.615+0000 7f76c5800640 1 --2- 172.21.15.133:0/3209383008 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76a4003a00 0x7f76a4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.616+0000 7f76c4e00640 1 --2- 172.21.15.133:0/3209383008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a4004800 0x7f76a4004c80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00a770 tx=0x7f76a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8275cc27fd8a1bd server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.616+0000 7f76bfe00640 1 -- 172.21.15.133:0/3209383008 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00a390 con 0x7f76a4004800 2024-03-07T06:51:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.616+0000 7f76bfe00640 1 -- 172.21.15.133:0/3209383008 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ac034650 con 0x7f76a4004800 2024-03-07T06:51:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.616+0000 7f76bfe00640 1 -- 172.21.15.133:0/3209383008 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac034a80 con 0x7f76a4004800 2024-03-07T06:51:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.617+0000 7f76c8200640 1 -- 172.21.15.133:0/3209383008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a4004800 msgr2=0x7f76a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.617+0000 7f76c8200640 1 --2- 172.21.15.133:0/3209383008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a4004800 0x7f76a4004c80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00a770 tx=0x7f76a4005830 comp rx=0 tx=0).stop 2024-03-07T06:51:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.617+0000 7f76c8200640 1 -- 172.21.15.133:0/3209383008 shutdown_connections 2024-03-07T06:51:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.617+0000 7f76c8200640 1 --2- 172.21.15.133:0/3209383008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a4004800 0x7f76a4004c80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.617+0000 7f76c8200640 1 -- 172.21.15.133:0/3209383008 >> 172.21.15.133:0/3209383008 conn(0x7f76c00b39b0 msgr2=0x7f76c00b3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.617+0000 7f76c8200640 1 -- 172.21.15.133:0/3209383008 shutdown_connections 2024-03-07T06:51:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.617+0000 7f76c8200640 1 -- 172.21.15.133:0/3209383008 wait complete. 2024-03-07T06:51:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.619+0000 7f76c8200640 1 Processor -- start 2024-03-07T06:51:12.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.619+0000 7f76c8200640 1 -- start start 2024-03-07T06:51:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.620+0000 7f76c8200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a4004800 0x7f76c01550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.620+0000 7f76c8200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76c0072470 0x7f76c00728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.620+0000 7f76c8200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76c00cb380 0x7f76c00cb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.620+0000 7f76c8200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c015ff50 con 0x7f76a4004800 2024-03-07T06:51:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.620+0000 7f76c8200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c00ce960 con 0x7f76c0072470 2024-03-07T06:51:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.620+0000 7f76c8200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c01600d0 con 0x7f76c00cb380 2024-03-07T06:51:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.620+0000 7f76c5800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a4004800 0x7f76c01550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.620+0000 7f76c5800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a4004800 0x7f76c01550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46220/0 (socket says 172.21.15.133:46220) 2024-03-07T06:51:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.620+0000 7f76c5800640 1 -- 172.21.15.133:0/1015813613 learned_addr learned my addr 172.21.15.133:0/1015813613 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.621+0000 7f76c6200640 1 --2- 172.21.15.133:0/1015813613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76c00cb380 0x7f76c00cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.621+0000 7f76c4e00640 1 --2- 172.21.15.133:0/1015813613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76c0072470 0x7f76c00728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.621+0000 7f76c5800640 1 -- 172.21.15.133:0/1015813613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76c00cb380 msgr2=0x7f76c00cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.621+0000 7f76c5800640 1 --2- 172.21.15.133:0/1015813613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76c00cb380 0x7f76c00cb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.621+0000 7f76c5800640 1 -- 172.21.15.133:0/1015813613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76c0072470 msgr2=0x7f76c00728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.621+0000 7f76c5800640 1 --2- 172.21.15.133:0/1015813613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76c0072470 0x7f76c00728f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.621+0000 7f76c5800640 1 -- 172.21.15.133:0/1015813613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c00b4b50 con 0x7f76a4004800 2024-03-07T06:51:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.621+0000 7f76c5800640 1 --2- 172.21.15.133:0/1015813613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a4004800 0x7f76c01550b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76b000d480 tx=0x7f76b000d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.621+0000 7f76bd600640 1 -- 172.21.15.133:0/1015813613 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0005480 con 0x7f76a4004800 2024-03-07T06:51:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.621+0000 7f76bd600640 1 -- 172.21.15.133:0/1015813613 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b0017070 con 0x7f76a4004800 2024-03-07T06:51:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.621+0000 7f76bd600640 1 -- 172.21.15.133:0/1015813613 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0003850 con 0x7f76a4004800 2024-03-07T06:51:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.621+0000 7f76c8200640 1 -- 172.21.15.133:0/1015813613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c00b4e30 con 0x7f76a4004800 2024-03-07T06:51:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.621+0000 7f76c8200640 1 -- 172.21.15.133:0/1015813613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c0071810 con 0x7f76a4004800 2024-03-07T06:51:12.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.623+0000 7f76c8200640 1 -- 172.21.15.133:0/1015813613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c0062c90 con 0x7f76a4004800 2024-03-07T06:51:12.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.623+0000 7f76bd600640 1 -- 172.21.15.133:0/1015813613 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f76b00039f0 con 0x7f76a4004800 2024-03-07T06:51:12.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.624+0000 7f76bd600640 1 --2- 172.21.15.133:0/1015813613 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f76a006b500 0x7f76a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:12.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.624+0000 7f76bd600640 1 -- 172.21.15.133:0/1015813613 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f76b0089ca0 con 0x7f76a4004800 2024-03-07T06:51:12.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.625+0000 7f76c4e00640 1 --2- 172.21.15.133:0/1015813613 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f76a006b500 0x7f76a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:12.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.626+0000 7f76c4e00640 1 --2- 172.21.15.133:0/1015813613 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f76a006b500 0x7f76a006d9c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f76ac034000 tx=0x7f76ac009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:12.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.628+0000 7f76bd600640 1 -- 172.21.15.133:0/1015813613 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f76b000f2f0 con 0x7f76a4004800 2024-03-07T06:51:12.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:12.808+0000 7f76c8200640 1 -- 172.21.15.133:0/1015813613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f76c00cf580 con 0x7f76a4004800 2024-03-07T06:51:13.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.122+0000 7f76bd600640 1 -- 172.21.15.133:0/1015813613 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f76b0058f20 con 0x7f76a4004800 2024-03-07T06:51:13.122 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-03-07T06:51:13.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.126+0000 7f76c8200640 1 -- 172.21.15.133:0/1015813613 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f76a006b500 msgr2=0x7f76a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:13.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.126+0000 7f76c8200640 1 --2- 172.21.15.133:0/1015813613 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f76a006b500 0x7f76a006d9c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f76ac034000 tx=0x7f76ac009030 comp rx=0 tx=0).stop 2024-03-07T06:51:13.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.126+0000 7f76c8200640 1 -- 172.21.15.133:0/1015813613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a4004800 msgr2=0x7f76c01550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:13.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.126+0000 7f76c8200640 1 --2- 172.21.15.133:0/1015813613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a4004800 0x7f76c01550b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76b000d480 tx=0x7f76b000d940 comp rx=0 tx=0).stop 2024-03-07T06:51:13.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.126+0000 7f76c8200640 1 -- 172.21.15.133:0/1015813613 shutdown_connections 2024-03-07T06:51:13.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.126+0000 7f76c8200640 1 --2- 172.21.15.133:0/1015813613 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f76c00cb380 0x7f76c00cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:13.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.126+0000 7f76c8200640 1 --2- 172.21.15.133:0/1015813613 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f76a006b500 0x7f76a006d9c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:13.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.126+0000 7f76c8200640 1 --2- 172.21.15.133:0/1015813613 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f76c0072470 0x7f76c00728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:13.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.126+0000 7f76c8200640 1 --2- 172.21.15.133:0/1015813613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76a4004800 0x7f76c01550b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:13.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.126+0000 7f76c8200640 1 -- 172.21.15.133:0/1015813613 >> 172.21.15.133:0/1015813613 conn(0x7f76c00b39b0 msgr2=0x7f76c006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:13.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.126+0000 7f76c8200640 1 -- 172.21.15.133:0/1015813613 shutdown_connections 2024-03-07T06:51:13.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.126+0000 7f76c8200640 1 -- 172.21.15.133:0/1015813613 wait complete. 2024-03-07T06:51:13.149 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-07T06:51:13.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.353+0000 7ff65dc00640 1 Processor -- start 2024-03-07T06:51:13.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.354+0000 7ff65dc00640 1 -- start start 2024-03-07T06:51:13.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.355+0000 7ff65dc00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff6580cfa40 0x7ff658160470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:13.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.355+0000 7ff65dc00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff658071a00 con 0x7ff65815be20 2024-03-07T06:51:13.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.355+0000 7ff65dc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65815f990 con 0x7ff6580c5850 2024-03-07T06:51:13.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.355+0000 7ff65dc00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6580b6080 con 0x7ff6580cfa40 2024-03-07T06:51:13.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.356+0000 7ff656000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff6580cfa40 0x7ff658160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:13.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.356+0000 7ff656a00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff65815be20 0x7ff6580cf330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53152/0 (socket says 172.21.15.133:53152) 2024-03-07T06:51:13.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.356+0000 7ff656a00640 1 -- 172.21.15.133:0/114514062 learned_addr learned my addr 172.21.15.133:0/114514062 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:13.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.356+0000 7ff656000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff6580cfa40 0x7ff658160470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46236/0 (socket says 172.21.15.133:46236) 2024-03-07T06:51:13.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.356+0000 7ff656000640 1 -- 172.21.15.133:0/114514062 >> v1:172.21.15.161:6789/0 conn(0x7ff6580c5850 legacy=0x7ff65815c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:13.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.356+0000 7ff656000640 1 -- 172.21.15.133:0/114514062 >> v1:172.21.15.133:6789/0 conn(0x7ff65815be20 legacy=0x7ff6580cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:13.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.356+0000 7ff656000640 1 -- 172.21.15.133:0/114514062 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65815fc70 con 0x7ff6580cfa40 2024-03-07T06:51:13.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.356+0000 7ff655600640 1 -- 172.21.15.133:0/114514062 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682052191 0 0) 0x7ff658071a00 con 0x7ff65815be20 2024-03-07T06:51:13.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.357+0000 7ff656000640 1 --2- 172.21.15.133:0/114514062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6580cfa40 0x7ff658160470 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff6400097a0 tx=0x7ff64002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8c0faeec837485 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:13.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.358+0000 7ff655600640 1 -- 172.21.15.133:0/114514062 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6400315a0 con 0x7ff6580cfa40 2024-03-07T06:51:13.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.358+0000 7ff655600640 1 -- 172.21.15.133:0/114514062 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff640037070 con 0x7ff6580cfa40 2024-03-07T06:51:13.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.358+0000 7ff655600640 1 -- 172.21.15.133:0/114514062 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640031960 con 0x7ff6580cfa40 2024-03-07T06:51:13.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.358+0000 7ff65dc00640 1 -- 172.21.15.133:0/114514062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6580cfa40 msgr2=0x7ff658160470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:13.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.358+0000 7ff65dc00640 1 --2- 172.21.15.133:0/114514062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6580cfa40 0x7ff658160470 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff6400097a0 tx=0x7ff64002b840 comp rx=0 tx=0).stop 2024-03-07T06:51:13.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.358+0000 7ff65dc00640 1 -- 172.21.15.133:0/114514062 shutdown_connections 2024-03-07T06:51:13.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.358+0000 7ff65dc00640 1 --2- 172.21.15.133:0/114514062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6580cfa40 0x7ff658160470 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:13.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.358+0000 7ff65dc00640 1 -- 172.21.15.133:0/114514062 >> 172.21.15.133:0/114514062 conn(0x7ff65806dab0 msgr2=0x7ff6580b57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:13.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.358+0000 7ff65dc00640 1 -- 172.21.15.133:0/114514062 shutdown_connections 2024-03-07T06:51:13.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.358+0000 7ff65dc00640 1 -- 172.21.15.133:0/114514062 wait complete. 2024-03-07T06:51:13.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.360+0000 7ff65dc00640 1 Processor -- start 2024-03-07T06:51:13.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.360+0000 7ff65dc00640 1 -- start start 2024-03-07T06:51:13.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.361+0000 7ff65dc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6580c5850 0x7ff6581f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:13.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.361+0000 7ff65dc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6580cfa40 0x7ff6580ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:13.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.361+0000 7ff65dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff65815be20 0x7ff6580cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:13.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.361+0000 7ff65dc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6580716b0 con 0x7ff65815be20 2024-03-07T06:51:13.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.361+0000 7ff65dc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff658071ce0 con 0x7ff6580c5850 2024-03-07T06:51:13.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.361+0000 7ff65dc00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65806ef30 con 0x7ff6580cfa40 2024-03-07T06:51:13.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.361+0000 7ff657400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff65815be20 0x7ff6580cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:13.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.361+0000 7ff657400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff65815be20 0x7ff6580cac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46240/0 (socket says 172.21.15.133:46240) 2024-03-07T06:51:13.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.361+0000 7ff657400640 1 -- 172.21.15.133:0/1960489615 learned_addr learned my addr 172.21.15.133:0/1960489615 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:13.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.362+0000 7ff657400640 1 -- 172.21.15.133:0/1960489615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6580cfa40 msgr2=0x7ff6580ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:13.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.362+0000 7ff657400640 1 --2- 172.21.15.133:0/1960489615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6580cfa40 0x7ff6580ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:13.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.362+0000 7ff657400640 1 -- 172.21.15.133:0/1960489615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6580c5850 msgr2=0x7ff6581f4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:13.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.362+0000 7ff657400640 1 --2- 172.21.15.133:0/1960489615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6580c5850 0x7ff6581f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:13.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.362+0000 7ff657400640 1 -- 172.21.15.133:0/1960489615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6580c8330 con 0x7ff65815be20 2024-03-07T06:51:13.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.362+0000 7ff657400640 1 --2- 172.21.15.133:0/1960489615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff65815be20 0x7ff6580cac40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff65000c410 tx=0x7ff65000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:13.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.362+0000 7ff64ea00640 1 -- 172.21.15.133:0/1960489615 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650011790 con 0x7ff65815be20 2024-03-07T06:51:13.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.362+0000 7ff64ea00640 1 -- 172.21.15.133:0/1960489615 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff650017070 con 0x7ff65815be20 2024-03-07T06:51:13.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.362+0000 7ff64ea00640 1 -- 172.21.15.133:0/1960489615 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff650011bb0 con 0x7ff65815be20 2024-03-07T06:51:13.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.362+0000 7ff65dc00640 1 -- 172.21.15.133:0/1960489615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6580c8610 con 0x7ff65815be20 2024-03-07T06:51:13.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.362+0000 7ff65dc00640 1 -- 172.21.15.133:0/1960489615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6580c8bd0 con 0x7ff65815be20 2024-03-07T06:51:13.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.363+0000 7ff65dc00640 1 -- 172.21.15.133:0/1960489615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff658062c90 con 0x7ff65815be20 2024-03-07T06:51:13.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.365+0000 7ff64ea00640 1 -- 172.21.15.133:0/1960489615 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff65001c460 con 0x7ff65815be20 2024-03-07T06:51:13.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.366+0000 7ff64ea00640 1 --2- 172.21.15.133:0/1960489615 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff63406b4d0 0x7ff63406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:13.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.366+0000 7ff64ea00640 1 -- 172.21.15.133:0/1960489615 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff650089d20 con 0x7ff65815be20 2024-03-07T06:51:13.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.366+0000 7ff656a00640 1 --2- 172.21.15.133:0/1960489615 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff63406b4d0 0x7ff63406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:13.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.367+0000 7ff656a00640 1 --2- 172.21.15.133:0/1960489615 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff63406b4d0 0x7ff63406d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff648007fd0 tx=0x7ff648007ce0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:13.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.371+0000 7ff64ea00640 1 -- 172.21.15.133:0/1960489615 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff650021020 con 0x7ff65815be20 2024-03-07T06:51:13.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:13.565+0000 7ff65dc00640 1 -- 172.21.15.133:0/1960489615 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff658066f20 con 0x7ff65815be20 2024-03-07T06:51:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.129+0000 7ff64ea00640 1 -- 172.21.15.133:0/1960489615 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff650058ce0 con 0x7ff65815be20 2024-03-07T06:51:14.129 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-03-07T06:51:14.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.133+0000 7ff65dc00640 1 -- 172.21.15.133:0/1960489615 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff63406b4d0 msgr2=0x7ff63406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.133+0000 7ff65dc00640 1 --2- 172.21.15.133:0/1960489615 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff63406b4d0 0x7ff63406d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff648007fd0 tx=0x7ff648007ce0 comp rx=0 tx=0).stop 2024-03-07T06:51:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.133+0000 7ff65dc00640 1 -- 172.21.15.133:0/1960489615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff65815be20 msgr2=0x7ff6580cac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.133+0000 7ff65dc00640 1 --2- 172.21.15.133:0/1960489615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff65815be20 0x7ff6580cac40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff65000c410 tx=0x7ff65000c8d0 comp rx=0 tx=0).stop 2024-03-07T06:51:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.133+0000 7ff65dc00640 1 -- 172.21.15.133:0/1960489615 shutdown_connections 2024-03-07T06:51:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.133+0000 7ff65dc00640 1 --2- 172.21.15.133:0/1960489615 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff65815be20 0x7ff6580cac40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.133+0000 7ff65dc00640 1 --2- 172.21.15.133:0/1960489615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6580cfa40 0x7ff6580ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.133+0000 7ff65dc00640 1 --2- 172.21.15.133:0/1960489615 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff63406b4d0 0x7ff63406d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:14.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.133+0000 7ff65dc00640 1 --2- 172.21.15.133:0/1960489615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6580c5850 0x7ff6581f4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.133+0000 7ff65dc00640 1 -- 172.21.15.133:0/1960489615 >> 172.21.15.133:0/1960489615 conn(0x7ff65806dab0 msgr2=0x7ff6580b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.133+0000 7ff65dc00640 1 -- 172.21.15.133:0/1960489615 shutdown_connections 2024-03-07T06:51:14.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.133+0000 7ff65dc00640 1 -- 172.21.15.133:0/1960489615 wait complete. 2024-03-07T06:51:14.162 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-07T06:51:14.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.349+0000 7f1b4d800640 1 Processor -- start 2024-03-07T06:51:14.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.349+0000 7f1b4d800640 1 -- start start 2024-03-07T06:51:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.350+0000 7f1b4d800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1b48143950 0x7f1b480cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.350+0000 7f1b4d800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1b48144040 0x7f1b4806d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.350+0000 7f1b4d800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b480ced30 con 0x7f1b4806ea40 2024-03-07T06:51:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.350+0000 7f1b4d800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b480cce40 con 0x7f1b48144040 2024-03-07T06:51:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.350+0000 7f1b4d800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480cc5e0 con 0x7f1b48143950 2024-03-07T06:51:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.351+0000 7f1b4ce00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1b4806ea40 0x7f1b480b2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53164/0 (socket says 172.21.15.133:53164) 2024-03-07T06:51:14.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.351+0000 7f1b4ce00640 1 -- 172.21.15.133:0/1301517134 learned_addr learned my addr 172.21.15.133:0/1301517134 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:14.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.351+0000 7f1b47e00640 1 --2- 172.21.15.133:0/1301517134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1b48143950 0x7f1b480cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:14.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.351+0000 7f1b3fe00640 1 --2- 172.21.15.133:0/1301517134 >> v2:172.21.15.161:3300/0 conn(0x7f1b48144040 0x7f1b4806d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:14.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.351+0000 7f1b47400640 1 -- 172.21.15.133:0/1301517134 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941747110 0 0) 0x7f1b40001490 con 0x7f1b4806ea40 2024-03-07T06:51:14.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.351+0000 7f1b47400640 1 -- 172.21.15.133:0/1301517134 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b2c003520 con 0x7f1b4806ea40 2024-03-07T06:51:14.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.351+0000 7f1b47400640 1 -- 172.21.15.133:0/1301517134 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 474701680 0 0) 0x7f1b2c003520 con 0x7f1b4806ea40 2024-03-07T06:51:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.351+0000 7f1b47400640 1 -- 172.21.15.133:0/1301517134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1b48143950 msgr2=0x7f1b480cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.351+0000 7f1b47400640 1 --2- 172.21.15.133:0/1301517134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1b48143950 0x7f1b480cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.351+0000 7f1b47400640 1 -- 172.21.15.133:0/1301517134 >> v2:172.21.15.161:3300/0 conn(0x7f1b48144040 msgr2=0x7f1b4806d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.351+0000 7f1b47400640 1 --2- 172.21.15.133:0/1301517134 >> v2:172.21.15.161:3300/0 conn(0x7f1b48144040 0x7f1b4806d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.351+0000 7f1b47400640 1 -- 172.21.15.133:0/1301517134 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b480cc880 con 0x7f1b4806ea40 2024-03-07T06:51:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.351+0000 7f1b47400640 1 -- 172.21.15.133:0/1301517134 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f1b40001bc0 con 0x7f1b4806ea40 2024-03-07T06:51:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.351+0000 7f1b47400640 1 -- 172.21.15.133:0/1301517134 >> v1:172.21.15.133:6789/0 conn(0x7f1b4806ea40 legacy=0x7f1b480b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.352+0000 7f1b47400640 1 --2- 172.21.15.133:0/1301517134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1b48143950 0x7f1b2c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.352+0000 7f1b47400640 1 --2- 172.21.15.133:0/1301517134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c003f30 0x7f1b2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.352+0000 7f1b47400640 1 --2- 172.21.15.133:0/1301517134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b2c004d40 0x7f1b2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.352+0000 7f1b47400640 1 -- 172.21.15.133:0/1301517134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480cc5e0 con 0x7f1b2c003f30 2024-03-07T06:51:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.352+0000 7f1b47400640 1 -- 172.21.15.133:0/1301517134 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c00dbd0 con 0x7f1b2c004d40 2024-03-07T06:51:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.352+0000 7f1b47400640 1 -- 172.21.15.133:0/1301517134 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c00dd50 con 0x7f1b48143950 2024-03-07T06:51:14.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.352+0000 7f1b3fe00640 1 --2- 172.21.15.133:0/1301517134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c003f30 0x7f1b2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.352+0000 7f1b47e00640 1 --2- 172.21.15.133:0/1301517134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1b48143950 0x7f1b2c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.352+0000 7f1b4ce00640 1 --2- 172.21.15.133:0/1301517134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b2c004d40 0x7f1b2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.352+0000 7f1b3fe00640 1 -- 172.21.15.133:0/1301517134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1b48143950 msgr2=0x7f1b2c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.352+0000 7f1b3fe00640 1 --2- 172.21.15.133:0/1301517134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1b48143950 0x7f1b2c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.352+0000 7f1b3fe00640 1 -- 172.21.15.133:0/1301517134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b2c004d40 msgr2=0x7f1b2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.352+0000 7f1b3fe00640 1 --2- 172.21.15.133:0/1301517134 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b2c004d40 0x7f1b2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.352+0000 7f1b3fe00640 1 -- 172.21.15.133:0/1301517134 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2c003520 con 0x7f1b2c003f30 2024-03-07T06:51:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.353+0000 7f1b3fe00640 1 --2- 172.21.15.133:0/1301517134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c003f30 0x7f1b2c004800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009ac0 tx=0x7f1b2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f09c9641afa6c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.354+0000 7f1b47400640 1 -- 172.21.15.133:0/1301517134 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b30039070 con 0x7f1b2c003f30 2024-03-07T06:51:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.354+0000 7f1b47400640 1 -- 172.21.15.133:0/1301517134 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3002c6d0 con 0x7f1b2c003f30 2024-03-07T06:51:14.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.354+0000 7f1b47400640 1 -- 172.21.15.133:0/1301517134 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b30009690 con 0x7f1b2c003f30 2024-03-07T06:51:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.354+0000 7f1b4d800640 1 -- 172.21.15.133:0/1301517134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c003f30 msgr2=0x7f1b2c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.354+0000 7f1b4d800640 1 --2- 172.21.15.133:0/1301517134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c003f30 0x7f1b2c004800 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009ac0 tx=0x7f1b2c005830 comp rx=0 tx=0).stop 2024-03-07T06:51:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.354+0000 7f1b4ce00640 1 -- 172.21.15.133:0/1301517134 reap_dead start 2024-03-07T06:51:14.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.354+0000 7f1b4d800640 1 -- 172.21.15.133:0/1301517134 shutdown_connections 2024-03-07T06:51:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.354+0000 7f1b4d800640 1 -- 172.21.15.133:0/1301517134 >> 172.21.15.133:0/1301517134 conn(0x7f1b48071860 msgr2=0x7f1b48071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.354+0000 7f1b4d800640 1 -- 172.21.15.133:0/1301517134 shutdown_connections 2024-03-07T06:51:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.355+0000 7f1b4d800640 1 -- 172.21.15.133:0/1301517134 wait complete. 2024-03-07T06:51:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.356+0000 7f1b4d800640 1 Processor -- start 2024-03-07T06:51:14.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.356+0000 7f1b4d800640 1 -- start start 2024-03-07T06:51:14.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.357+0000 7f1b4d800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c003f30 0x7f1b480cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:14.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.357+0000 7f1b4d800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1b4815ab20 0x7f1b4813b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:14.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.357+0000 7f1b4d800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b4813e620 0x7f1b4813ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:14.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.357+0000 7f1b4d800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4806d5e0 con 0x7f1b2c003f30 2024-03-07T06:51:14.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.357+0000 7f1b4d800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480cefb0 con 0x7f1b4813e620 2024-03-07T06:51:14.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.357+0000 7f1b4d800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48143d30 con 0x7f1b4815ab20 2024-03-07T06:51:14.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.358+0000 7f1b47e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c003f30 0x7f1b480cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:14.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.358+0000 7f1b47e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c003f30 0x7f1b480cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46264/0 (socket says 172.21.15.133:46264) 2024-03-07T06:51:14.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.358+0000 7f1b47e00640 1 -- 172.21.15.133:0/3431918717 learned_addr learned my addr 172.21.15.133:0/3431918717 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:14.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.358+0000 7f1b3fe00640 1 --2- 172.21.15.133:0/3431918717 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1b4815ab20 0x7f1b4813b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:14.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.358+0000 7f1b47e00640 1 -- 172.21.15.133:0/3431918717 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1b4815ab20 msgr2=0x7f1b4813b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:14.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.358+0000 7f1b47e00640 1 --2- 172.21.15.133:0/3431918717 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1b4815ab20 0x7f1b4813b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:14.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.358+0000 7f1b47e00640 1 -- 172.21.15.133:0/3431918717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b4813e620 msgr2=0x7f1b4813ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:14.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.358+0000 7f1b47e00640 1 --2- 172.21.15.133:0/3431918717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b4813e620 0x7f1b4813ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:14.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.358+0000 7f1b47e00640 1 -- 172.21.15.133:0/3431918717 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b480cbc30 con 0x7f1b2c003f30 2024-03-07T06:51:14.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.358+0000 7f1b47e00640 1 --2- 172.21.15.133:0/3431918717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c003f30 0x7f1b480cb6f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1b38011600 tx=0x7f1b38011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:14.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.358+0000 7f1b44c00640 1 -- 172.21.15.133:0/3431918717 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800f920 con 0x7f1b2c003f30 2024-03-07T06:51:14.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.359+0000 7f1b44c00640 1 -- 172.21.15.133:0/3431918717 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3800fac0 con 0x7f1b2c003f30 2024-03-07T06:51:14.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.359+0000 7f1b44c00640 1 -- 172.21.15.133:0/3431918717 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38015460 con 0x7f1b2c003f30 2024-03-07T06:51:14.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.359+0000 7f1b4d800640 1 -- 172.21.15.133:0/3431918717 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4813f7a0 con 0x7f1b2c003f30 2024-03-07T06:51:14.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.359+0000 7f1b4d800640 1 -- 172.21.15.133:0/3431918717 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4813fd60 con 0x7f1b2c003f30 2024-03-07T06:51:14.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.360+0000 7f1b4d800640 1 -- 172.21.15.133:0/3431918717 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b48067c80 con 0x7f1b2c003f30 2024-03-07T06:51:14.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.361+0000 7f1b44c00640 1 -- 172.21.15.133:0/3431918717 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1b38015600 con 0x7f1b2c003f30 2024-03-07T06:51:14.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.362+0000 7f1b44c00640 1 --2- 172.21.15.133:0/3431918717 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1b2406b4d0 0x7f1b2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:14.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.362+0000 7f1b44c00640 1 -- 172.21.15.133:0/3431918717 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f1b3808a750 con 0x7f1b2c003f30 2024-03-07T06:51:14.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.363+0000 7f1b3fe00640 1 --2- 172.21.15.133:0/3431918717 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1b2406b4d0 0x7f1b2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:14.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.364+0000 7f1b3fe00640 1 --2- 172.21.15.133:0/3431918717 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1b2406b4d0 0x7f1b2406d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1b30036000 tx=0x7f1b30030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:14.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.367+0000 7f1b44c00640 1 -- 172.21.15.133:0/3431918717 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1b38019030 con 0x7f1b2c003f30 2024-03-07T06:51:14.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:14.592+0000 7f1b4d800640 1 -- 172.21.15.133:0/3431918717 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1b4815f490 con 0x7f1b2c003f30 2024-03-07T06:51:15.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.178+0000 7f1b44c00640 1 -- 172.21.15.133:0/3431918717 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1b38058f40 con 0x7f1b2c003f30 2024-03-07T06:51:15.179 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 2 and data pool 3 2024-03-07T06:51:15.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.185+0000 7f1b4d800640 1 -- 172.21.15.133:0/3431918717 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1b2406b4d0 msgr2=0x7f1b2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:15.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.185+0000 7f1b4d800640 1 --2- 172.21.15.133:0/3431918717 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1b2406b4d0 0x7f1b2406d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1b30036000 tx=0x7f1b30030000 comp rx=0 tx=0).stop 2024-03-07T06:51:15.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.185+0000 7f1b4d800640 1 -- 172.21.15.133:0/3431918717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c003f30 msgr2=0x7f1b480cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:15.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.185+0000 7f1b4d800640 1 --2- 172.21.15.133:0/3431918717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c003f30 0x7f1b480cb6f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1b38011600 tx=0x7f1b38011ac0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.185+0000 7f1b4d800640 1 -- 172.21.15.133:0/3431918717 shutdown_connections 2024-03-07T06:51:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.185+0000 7f1b4d800640 1 --2- 172.21.15.133:0/3431918717 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1b2406b4d0 0x7f1b2406d990 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1b30036000 tx=0x7f1b30030000 comp rx=0 tx=0).stop 2024-03-07T06:51:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.185+0000 7f1b4d800640 1 --2- 172.21.15.133:0/3431918717 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1b4813e620 0x7f1b4813ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.185+0000 7f1b4d800640 1 --2- 172.21.15.133:0/3431918717 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1b4815ab20 0x7f1b4813b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.185+0000 7f1b4d800640 1 --2- 172.21.15.133:0/3431918717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b2c003f30 0x7f1b480cb6f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.185+0000 7f1b4d800640 1 -- 172.21.15.133:0/3431918717 >> 172.21.15.133:0/3431918717 conn(0x7f1b48071860 msgr2=0x7f1b4815d640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.186+0000 7f1b4d800640 1 -- 172.21.15.133:0/3431918717 shutdown_connections 2024-03-07T06:51:15.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.186+0000 7f1b4d800640 1 -- 172.21.15.133:0/3431918717 wait complete. 2024-03-07T06:51:15.215 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:15.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.389+0000 7f1fde600640 1 Processor -- start 2024-03-07T06:51:15.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.389+0000 7f1fde600640 1 -- start start 2024-03-07T06:51:15.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.391+0000 7f1fde600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1fd8071d20 0x7f1fd816d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:15.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.391+0000 7f1fde600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd80ce540 con 0x7f1fd80b3c10 2024-03-07T06:51:15.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.391+0000 7f1fde600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd80cdf80 con 0x7f1fd8165ed0 2024-03-07T06:51:15.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.391+0000 7f1fde600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fd80734a0 con 0x7f1fd8071d20 2024-03-07T06:51:15.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.391+0000 7f1fd6a00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1fd80b3c10 0x7f1fd8071610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53168/0 (socket says 172.21.15.133:53168) 2024-03-07T06:51:15.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.391+0000 7f1fd6a00640 1 -- 172.21.15.133:0/2927859875 learned_addr learned my addr 172.21.15.133:0/2927859875 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:15.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.391+0000 7f1fd7e00640 1 --2- 172.21.15.133:0/2927859875 >> v2:172.21.15.133:3300/0 conn(0x7f1fd8071d20 0x7f1fd816d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:15.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.392+0000 7f1fd6000640 1 -- 172.21.15.133:0/2927859875 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005499830 0 0) 0x7f1fd80ce540 con 0x7f1fd80b3c10 2024-03-07T06:51:15.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.392+0000 7f1fd6000640 1 -- 172.21.15.133:0/2927859875 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fb8003520 con 0x7f1fd80b3c10 2024-03-07T06:51:15.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.392+0000 7f1fd7e00640 1 -- 172.21.15.133:0/2927859875 >> v1:172.21.15.161:6789/0 conn(0x7f1fd8165ed0 legacy=0x7f1fd806ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:15.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.392+0000 7f1fd7e00640 1 -- 172.21.15.133:0/2927859875 >> v1:172.21.15.133:6789/0 conn(0x7f1fd80b3c10 legacy=0x7f1fd8071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:15.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.392+0000 7f1fd7e00640 1 -- 172.21.15.133:0/2927859875 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd80ce260 con 0x7f1fd8071d20 2024-03-07T06:51:15.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.393+0000 7f1fd7e00640 1 --2- 172.21.15.133:0/2927859875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd8071d20 0x7f1fd816d3d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00097a0 tx=0x7f1fd002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726b285ce8a2ff7e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:15.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.393+0000 7f1fd6000640 1 -- 172.21.15.133:0/2927859875 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0034070 con 0x7f1fd8071d20 2024-03-07T06:51:15.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.393+0000 7f1fd6000640 1 -- 172.21.15.133:0/2927859875 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd0030610 con 0x7f1fd8071d20 2024-03-07T06:51:15.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.393+0000 7f1fd6000640 1 -- 172.21.15.133:0/2927859875 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd00309c0 con 0x7f1fd8071d20 2024-03-07T06:51:15.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.394+0000 7f1fde600640 1 -- 172.21.15.133:0/2927859875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd8071d20 msgr2=0x7f1fd816d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:15.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.394+0000 7f1fde600640 1 --2- 172.21.15.133:0/2927859875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd8071d20 0x7f1fd816d3d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00097a0 tx=0x7f1fd002b960 comp rx=0 tx=0).stop 2024-03-07T06:51:15.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.394+0000 7f1fde600640 1 -- 172.21.15.133:0/2927859875 shutdown_connections 2024-03-07T06:51:15.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.394+0000 7f1fde600640 1 --2- 172.21.15.133:0/2927859875 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd8071d20 0x7f1fd816d3d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.394+0000 7f1fde600640 1 -- 172.21.15.133:0/2927859875 >> 172.21.15.133:0/2927859875 conn(0x7f1fd80b3120 msgr2=0x7f1fd80b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:15.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.394+0000 7f1fde600640 1 -- 172.21.15.133:0/2927859875 shutdown_connections 2024-03-07T06:51:15.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.394+0000 7f1fde600640 1 -- 172.21.15.133:0/2927859875 wait complete. 2024-03-07T06:51:15.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.396+0000 7f1fde600640 1 Processor -- start 2024-03-07T06:51:15.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.396+0000 7f1fde600640 1 -- start start 2024-03-07T06:51:15.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fde600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd8071d20 0x7f1fd809d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:15.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fde600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fd80b3c10 0x7f1fd814d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:15.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fde600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1fd8165ed0 0x7f1fd815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:15.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fde600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd8163fa0 con 0x7f1fd8071d20 2024-03-07T06:51:15.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fde600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd8072550 con 0x7f1fd80b3c10 2024-03-07T06:51:15.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fde600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd8164120 con 0x7f1fd8165ed0 2024-03-07T06:51:15.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fd7400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd8071d20 0x7f1fd809d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:15.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fd7400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd8071d20 0x7f1fd809d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46274/0 (socket says 172.21.15.133:46274) 2024-03-07T06:51:15.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fd7400640 1 -- 172.21.15.133:0/1937132165 learned_addr learned my addr 172.21.15.133:0/1937132165 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:15.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fd6a00640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fd80b3c10 0x7f1fd814d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:15.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fd7e00640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1fd8165ed0 0x7f1fd815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:15.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fd7400640 1 -- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1fd8165ed0 msgr2=0x7f1fd815ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:15.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fd7400640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1fd8165ed0 0x7f1fd815ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fd7400640 1 -- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fd80b3c10 msgr2=0x7f1fd814d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:15.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fd7400640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fd80b3c10 0x7f1fd814d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fd7400640 1 -- 172.21.15.133:0/1937132165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd809d7a0 con 0x7f1fd8071d20 2024-03-07T06:51:15.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.397+0000 7f1fd6a00640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fd80b3c10 0x7f1fd814d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:15.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.398+0000 7f1fd7400640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd8071d20 0x7f1fd809d260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800c400 tx=0x7f1fc800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:15.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.398+0000 7f1fcf400640 1 -- 172.21.15.133:0/1937132165 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8017070 con 0x7f1fd8071d20 2024-03-07T06:51:15.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.398+0000 7f1fcf400640 1 -- 172.21.15.133:0/1937132165 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc80052d0 con 0x7f1fd8071d20 2024-03-07T06:51:15.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.398+0000 7f1fcf400640 1 -- 172.21.15.133:0/1937132165 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8003830 con 0x7f1fd8071d20 2024-03-07T06:51:15.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.398+0000 7f1fde600640 1 -- 172.21.15.133:0/1937132165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd815a300 con 0x7f1fd8071d20 2024-03-07T06:51:15.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.398+0000 7f1fde600640 1 -- 172.21.15.133:0/1937132165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd8159d40 con 0x7f1fd8071d20 2024-03-07T06:51:15.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.399+0000 7f1fde600640 1 -- 172.21.15.133:0/1937132165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd8062c90 con 0x7f1fd8071d20 2024-03-07T06:51:15.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.400+0000 7f1fcf400640 1 -- 172.21.15.133:0/1937132165 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1fc801a070 con 0x7f1fd8071d20 2024-03-07T06:51:15.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.401+0000 7f1fcf400640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1fb406b4d0 0x7f1fb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:15.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.401+0000 7f1fcf400640 1 -- 172.21.15.133:0/1937132165 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fc8089620 con 0x7f1fd8071d20 2024-03-07T06:51:15.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.401+0000 7f1fd6a00640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1fb406b4d0 0x7f1fb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:15.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.402+0000 7f1fd6a00640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1fb406b4d0 0x7f1fb406d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1fd815c880 tx=0x7f1fc0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:15.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.406+0000 7f1fcf400640 1 -- 172.21.15.133:0/1937132165 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1fc8058420 con 0x7f1fd8071d20 2024-03-07T06:51:15.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.587+0000 7f1fde600640 1 -- 172.21.15.133:0/1937132165 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fd80cc650 con 0x7f1fd8071d20 2024-03-07T06:51:15.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.590+0000 7f1fcf400640 1 -- 172.21.15.133:0/1937132165 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16666 (secure 0 0 0) 0x7f1fc8057d20 con 0x7f1fd8071d20 2024-03-07T06:51:15.590 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:15.590 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":20,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:14.593670+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:15.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.592+0000 7f1fde600640 1 -- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1fb406b4d0 msgr2=0x7f1fb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:15.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.592+0000 7f1fde600640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1fb406b4d0 0x7f1fb406d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1fd815c880 tx=0x7f1fc0009040 comp rx=0 tx=0).stop 2024-03-07T06:51:15.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.592+0000 7f1fde600640 1 -- 172.21.15.133:0/1937132165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd8071d20 msgr2=0x7f1fd809d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:15.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.592+0000 7f1fde600640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd8071d20 0x7f1fd809d260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800c400 tx=0x7f1fc800c8c0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.593+0000 7f1fde600640 1 -- 172.21.15.133:0/1937132165 shutdown_connections 2024-03-07T06:51:15.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.593+0000 7f1fde600640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1fd8165ed0 0x7f1fd815ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.593+0000 7f1fde600640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1fb406b4d0 0x7f1fb406d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.593+0000 7f1fde600640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1fd80b3c10 0x7f1fd814d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.593+0000 7f1fde600640 1 --2- 172.21.15.133:0/1937132165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1fd8071d20 0x7f1fd809d260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.593+0000 7f1fde600640 1 -- 172.21.15.133:0/1937132165 >> 172.21.15.133:0/1937132165 conn(0x7f1fd80b3120 msgr2=0x7f1fd806ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.593+0000 7f1fde600640 1 -- 172.21.15.133:0/1937132165 shutdown_connections 2024-03-07T06:51:15.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.593+0000 7f1fde600640 1 -- 172.21.15.133:0/1937132165 wait complete. 2024-03-07T06:51:15.608 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:15.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.819+0000 7f9b9e800640 1 Processor -- start 2024-03-07T06:51:15.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.819+0000 7f9b9e800640 1 -- start start 2024-03-07T06:51:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.821+0000 7f9b9e800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b9815fbb0 0x7f9b980cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.821+0000 7f9b9e800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9b980cdee0 0x7f9b9815c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.821+0000 7f9b9e800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b980731b0 con 0x7f9b98072950 2024-03-07T06:51:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.821+0000 7f9b9e800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b98072f60 con 0x7f9b980cdee0 2024-03-07T06:51:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.821+0000 7f9b9e800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b98073490 con 0x7f9b9815fbb0 2024-03-07T06:51:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.821+0000 7f9b97e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9b980cdee0 0x7f9b9815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.821+0000 7f9b97e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9b980cdee0 0x7f9b9815c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46276/0 (socket says 172.21.15.133:46276) 2024-03-07T06:51:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.821+0000 7f9b97e00640 1 -- 172.21.15.133:0/3567764857 learned_addr learned my addr 172.21.15.133:0/3567764857 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:15.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.821+0000 7f9b96a00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9b98072950 0x7f9b9815ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53180/0 (socket says 172.21.15.133:53180) 2024-03-07T06:51:15.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.821+0000 7f9b97400640 1 --2- 172.21.15.133:0/3567764857 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b9815fbb0 0x7f9b980cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:15.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.821+0000 7f9b97e00640 1 -- 172.21.15.133:0/3567764857 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b9815fbb0 msgr2=0x7f9b980cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:15.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.821+0000 7f9b97e00640 1 --2- 172.21.15.133:0/3567764857 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b9815fbb0 0x7f9b980cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.821+0000 7f9b97e00640 1 -- 172.21.15.133:0/3567764857 >> v1:172.21.15.133:6789/0 conn(0x7f9b98072950 legacy=0x7f9b9815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:15.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.821+0000 7f9b97e00640 1 -- 172.21.15.133:0/3567764857 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b98072150 con 0x7f9b980cdee0 2024-03-07T06:51:15.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.823+0000 7f9b97e00640 1 --2- 172.21.15.133:0/3567764857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b980cdee0 0x7f9b9815c550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b980730e0 tx=0x7f9b9002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8b2fc917b2be6a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:15.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.823+0000 7f9b96000640 1 -- 172.21.15.133:0/3567764857 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90035020 con 0x7f9b980cdee0 2024-03-07T06:51:15.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.823+0000 7f9b96000640 1 -- 172.21.15.133:0/3567764857 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b90031640 con 0x7f9b980cdee0 2024-03-07T06:51:15.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.823+0000 7f9b96000640 1 -- 172.21.15.133:0/3567764857 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9002d040 con 0x7f9b980cdee0 2024-03-07T06:51:15.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.824+0000 7f9b9e800640 1 -- 172.21.15.133:0/3567764857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b980cdee0 msgr2=0x7f9b9815c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:15.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.824+0000 7f9b9e800640 1 --2- 172.21.15.133:0/3567764857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b980cdee0 0x7f9b9815c550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b980730e0 tx=0x7f9b9002b8f0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.824+0000 7f9b9e800640 1 -- 172.21.15.133:0/3567764857 shutdown_connections 2024-03-07T06:51:15.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.824+0000 7f9b9e800640 1 --2- 172.21.15.133:0/3567764857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b980cdee0 0x7f9b9815c550 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.824+0000 7f9b9e800640 1 --2- 172.21.15.133:0/3567764857 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b9815fbb0 0x7f9b980cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.824+0000 7f9b9e800640 1 -- 172.21.15.133:0/3567764857 >> 172.21.15.133:0/3567764857 conn(0x7f9b980b3120 msgr2=0x7f9b980b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:15.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.824+0000 7f9b9e800640 1 -- 172.21.15.133:0/3567764857 shutdown_connections 2024-03-07T06:51:15.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.824+0000 7f9b9e800640 1 -- 172.21.15.133:0/3567764857 wait complete. 2024-03-07T06:51:15.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.826+0000 7f9b9e800640 1 Processor -- start 2024-03-07T06:51:15.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.826+0000 7f9b9e800640 1 -- start start 2024-03-07T06:51:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b9e800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b980cdee0 0x7f9b98159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b9e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b9815fbb0 0x7f9b980aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b9e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b98159090 0x7f9b980b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b9e800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b98160980 con 0x7f9b98159090 2024-03-07T06:51:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b9e800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980ce2f0 con 0x7f9b9815fbb0 2024-03-07T06:51:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b9e800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b980b1a00 con 0x7f9b980cdee0 2024-03-07T06:51:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b97e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b98159090 0x7f9b980b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b97e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b98159090 0x7f9b980b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46286/0 (socket says 172.21.15.133:46286) 2024-03-07T06:51:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b97e00640 1 -- 172.21.15.133:0/2196415532 learned_addr learned my addr 172.21.15.133:0/2196415532 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b97e00640 1 -- 172.21.15.133:0/2196415532 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b980cdee0 msgr2=0x7f9b98159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b97e00640 1 --2- 172.21.15.133:0/2196415532 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b980cdee0 0x7f9b98159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b97e00640 1 -- 172.21.15.133:0/2196415532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b9815fbb0 msgr2=0x7f9b980aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b97e00640 1 --2- 172.21.15.133:0/2196415532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b9815fbb0 0x7f9b980aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:15.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b97e00640 1 -- 172.21.15.133:0/2196415532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b980af330 con 0x7f9b98159090 2024-03-07T06:51:15.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.827+0000 7f9b97e00640 1 --2- 172.21.15.133:0/2196415532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b98159090 0x7f9b980b5450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9b9002b930 tx=0x7f9b900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.828+0000 7f9b8f400640 1 -- 172.21.15.133:0/2196415532 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90040070 con 0x7f9b98159090 2024-03-07T06:51:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.828+0000 7f9b8f400640 1 -- 172.21.15.133:0/2196415532 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b900399f0 con 0x7f9b98159090 2024-03-07T06:51:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.828+0000 7f9b9e800640 1 -- 172.21.15.133:0/2196415532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b980af610 con 0x7f9b98159090 2024-03-07T06:51:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.828+0000 7f9b8f400640 1 -- 172.21.15.133:0/2196415532 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90035040 con 0x7f9b98159090 2024-03-07T06:51:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.828+0000 7f9b9e800640 1 -- 172.21.15.133:0/2196415532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b980afbd0 con 0x7f9b98159090 2024-03-07T06:51:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.829+0000 7f9b9e800640 1 -- 172.21.15.133:0/2196415532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b98062c90 con 0x7f9b98159090 2024-03-07T06:51:15.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.830+0000 7f9b8f400640 1 -- 172.21.15.133:0/2196415532 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9b90038020 con 0x7f9b98159090 2024-03-07T06:51:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.831+0000 7f9b8f400640 1 --2- 172.21.15.133:0/2196415532 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9b7406b4d0 0x7f9b7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.831+0000 7f9b8f400640 1 -- 172.21.15.133:0/2196415532 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b900abc60 con 0x7f9b98159090 2024-03-07T06:51:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.832+0000 7f9b97400640 1 --2- 172.21.15.133:0/2196415532 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9b7406b4d0 0x7f9b7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:15.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.833+0000 7f9b97400640 1 --2- 172.21.15.133:0/2196415532 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9b7406b4d0 0x7f9b7406d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9b880070b0 tx=0x7f9b88007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:15.836+0000 7f9b8f400640 1 -- 172.21.15.133:0/2196415532 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9b9007ab00 con 0x7f9b98159090 2024-03-07T06:51:16.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.033+0000 7f9b9e800640 1 -- 172.21.15.133:0/2196415532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b980cb5a0 con 0x7f9b98159090 2024-03-07T06:51:16.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.035+0000 7f9b8f400640 1 -- 172.21.15.133:0/2196415532 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16666 (secure 0 0 0) 0x7f9b9007a400 con 0x7f9b98159090 2024-03-07T06:51:16.034 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:16.034 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":20,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:14.593670+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:16.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.037+0000 7f9b9e800640 1 -- 172.21.15.133:0/2196415532 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9b7406b4d0 msgr2=0x7f9b7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:16.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.037+0000 7f9b9e800640 1 --2- 172.21.15.133:0/2196415532 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9b7406b4d0 0x7f9b7406d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9b880070b0 tx=0x7f9b88007040 comp rx=0 tx=0).stop 2024-03-07T06:51:16.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.037+0000 7f9b9e800640 1 -- 172.21.15.133:0/2196415532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b98159090 msgr2=0x7f9b980b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:16.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.037+0000 7f9b9e800640 1 --2- 172.21.15.133:0/2196415532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b98159090 0x7f9b980b5450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9b9002b930 tx=0x7f9b900046c0 comp rx=0 tx=0).stop 2024-03-07T06:51:16.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.037+0000 7f9b9e800640 1 -- 172.21.15.133:0/2196415532 shutdown_connections 2024-03-07T06:51:16.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.037+0000 7f9b9e800640 1 --2- 172.21.15.133:0/2196415532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9b98159090 0x7f9b980b5450 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:16.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.037+0000 7f9b9e800640 1 --2- 172.21.15.133:0/2196415532 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9b7406b4d0 0x7f9b7406d990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:16.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.037+0000 7f9b9e800640 1 --2- 172.21.15.133:0/2196415532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9b9815fbb0 0x7f9b980aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:16.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.037+0000 7f9b9e800640 1 --2- 172.21.15.133:0/2196415532 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9b980cdee0 0x7f9b98159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:16.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.037+0000 7f9b9e800640 1 -- 172.21.15.133:0/2196415532 >> 172.21.15.133:0/2196415532 conn(0x7f9b980b3120 msgr2=0x7f9b9815ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:16.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.038+0000 7f9b9e800640 1 -- 172.21.15.133:0/2196415532 shutdown_connections 2024-03-07T06:51:16.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.038+0000 7f9b9e800640 1 -- 172.21.15.133:0/2196415532 wait complete. 2024-03-07T06:51:16.057 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-07T06:51:16.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.268+0000 7f89fe000640 1 Processor -- start 2024-03-07T06:51:16.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.268+0000 7f89fe000640 1 -- start start 2024-03-07T06:51:16.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.270+0000 7f89fe000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89f815be60 0x7f89f8165930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:16.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.270+0000 7f89fe000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f806d740 con 0x7f89f815b370 2024-03-07T06:51:16.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.270+0000 7f89fe000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f806da20 con 0x7f89f80b3ed0 2024-03-07T06:51:16.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.270+0000 7f89fe000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f806dd00 con 0x7f89f815be60 2024-03-07T06:51:16.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.270+0000 7f89f7400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f89f815b370 0x7f89f80cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53190/0 (socket says 172.21.15.133:53190) 2024-03-07T06:51:16.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.270+0000 7f89f7400640 1 -- 172.21.15.133:0/2708101506 learned_addr learned my addr 172.21.15.133:0/2708101506 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:16.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f7e00640 1 --2- 172.21.15.133:0/2708101506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89f815be60 0x7f89f8165930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 -- 172.21.15.133:0/2708101506 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2547055837 0 0) 0x7f89f806d740 con 0x7f89f815b370 2024-03-07T06:51:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 -- 172.21.15.133:0/2708101506 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89dc003520 con 0x7f89f815b370 2024-03-07T06:51:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 -- 172.21.15.133:0/2708101506 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2530000495 0 0) 0x7f89dc003520 con 0x7f89f815b370 2024-03-07T06:51:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 -- 172.21.15.133:0/2708101506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89f815be60 msgr2=0x7f89f8165930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 --2- 172.21.15.133:0/2708101506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89f815be60 0x7f89f8165930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 -- 172.21.15.133:0/2708101506 >> v1:172.21.15.161:6789/0 conn(0x7f89f80b3ed0 legacy=0x7f89f815b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 -- 172.21.15.133:0/2708101506 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f806dfe0 con 0x7f89f815b370 2024-03-07T06:51:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 -- 172.21.15.133:0/2708101506 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f89e80019f0 con 0x7f89f815b370 2024-03-07T06:51:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 -- 172.21.15.133:0/2708101506 >> v1:172.21.15.133:6789/0 conn(0x7f89f815b370 legacy=0x7f89f80cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:16.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 --2- 172.21.15.133:0/2708101506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89dc0039e0 0x7f89dc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 --2- 172.21.15.133:0/2708101506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89dc004390 0x7f89dc004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 --2- 172.21.15.133:0/2708101506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89f815be60 0x7f89dc0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 -- 172.21.15.133:0/2708101506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f806dd00 con 0x7f89dc0039e0 2024-03-07T06:51:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 -- 172.21.15.133:0/2708101506 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc00b9c0 con 0x7f89dc004390 2024-03-07T06:51:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.271+0000 7f89f6a00640 1 -- 172.21.15.133:0/2708101506 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc00ddd0 con 0x7f89f815be60 2024-03-07T06:51:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.272+0000 7f89f7400640 1 --2- 172.21.15.133:0/2708101506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89dc0039e0 0x7f89dc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.272+0000 7f89eea00640 1 --2- 172.21.15.133:0/2708101506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89dc004390 0x7f89dc004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.272+0000 7f89f7e00640 1 --2- 172.21.15.133:0/2708101506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89f815be60 0x7f89dc0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.272+0000 7f89f7400640 1 -- 172.21.15.133:0/2708101506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89f815be60 msgr2=0x7f89dc0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.272+0000 7f89f7400640 1 --2- 172.21.15.133:0/2708101506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89f815be60 0x7f89dc0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.272+0000 7f89f7400640 1 -- 172.21.15.133:0/2708101506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89dc004390 msgr2=0x7f89dc004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.272+0000 7f89f7400640 1 --2- 172.21.15.133:0/2708101506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89dc004390 0x7f89dc004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:16.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.272+0000 7f89f7400640 1 -- 172.21.15.133:0/2708101506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc003520 con 0x7f89dc0039e0 2024-03-07T06:51:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.273+0000 7f89f7400640 1 --2- 172.21.15.133:0/2708101506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89dc0039e0 0x7f89dc003dc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f89e800a860 tx=0x7f89dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50249ca1e6bd85a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.274+0000 7f89f6a00640 1 -- 172.21.15.133:0/2708101506 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e800a480 con 0x7f89dc0039e0 2024-03-07T06:51:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.274+0000 7f89f6a00640 1 -- 172.21.15.133:0/2708101506 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e8034630 con 0x7f89dc0039e0 2024-03-07T06:51:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.274+0000 7f89f6a00640 1 -- 172.21.15.133:0/2708101506 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e8034a40 con 0x7f89dc0039e0 2024-03-07T06:51:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.274+0000 7f89fe000640 1 -- 172.21.15.133:0/2708101506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89dc0039e0 msgr2=0x7f89dc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.274+0000 7f89fe000640 1 --2- 172.21.15.133:0/2708101506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89dc0039e0 0x7f89dc003dc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f89e800a860 tx=0x7f89dc005830 comp rx=0 tx=0).stop 2024-03-07T06:51:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.274+0000 7f89f7e00640 1 -- 172.21.15.133:0/2708101506 reap_dead start 2024-03-07T06:51:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.274+0000 7f89fe000640 1 -- 172.21.15.133:0/2708101506 shutdown_connections 2024-03-07T06:51:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.274+0000 7f89fe000640 1 -- 172.21.15.133:0/2708101506 >> 172.21.15.133:0/2708101506 conn(0x7f89f80b1d30 msgr2=0x7f89f80b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.274+0000 7f89fe000640 1 -- 172.21.15.133:0/2708101506 shutdown_connections 2024-03-07T06:51:16.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.274+0000 7f89fe000640 1 -- 172.21.15.133:0/2708101506 wait complete. 2024-03-07T06:51:16.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.276+0000 7f89fe000640 1 Processor -- start 2024-03-07T06:51:16.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.276+0000 7f89fe000640 1 -- start start 2024-03-07T06:51:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.277+0000 7f89fe000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89dc0039e0 0x7f89f813c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.277+0000 7f89fe000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89f814b5b0 0x7f89f814ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.277+0000 7f89fe000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89f814e4c0 0x7f89f814e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.277+0000 7f89fe000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f80cf7e0 con 0x7f89f814e4c0 2024-03-07T06:51:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.277+0000 7f89fe000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f80cf660 con 0x7f89dc0039e0 2024-03-07T06:51:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.277+0000 7f89fe000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f806d180 con 0x7f89f814b5b0 2024-03-07T06:51:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.277+0000 7f89f7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89f814e4c0 0x7f89f814e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.277+0000 7f89f7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89f814e4c0 0x7f89f814e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46304/0 (socket says 172.21.15.133:46304) 2024-03-07T06:51:16.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.277+0000 7f89f7e00640 1 -- 172.21.15.133:0/1430924547 learned_addr learned my addr 172.21.15.133:0/1430924547 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.278+0000 7f89f7400640 1 --2- 172.21.15.133:0/1430924547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89dc0039e0 0x7f89f813c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.278+0000 7f89eea00640 1 --2- 172.21.15.133:0/1430924547 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89f814b5b0 0x7f89f814ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.278+0000 7f89f7e00640 1 -- 172.21.15.133:0/1430924547 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89f814b5b0 msgr2=0x7f89f814ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.278+0000 7f89f7e00640 1 --2- 172.21.15.133:0/1430924547 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89f814b5b0 0x7f89f814ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.278+0000 7f89f7e00640 1 -- 172.21.15.133:0/1430924547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89dc0039e0 msgr2=0x7f89f813c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.278+0000 7f89f7e00640 1 --2- 172.21.15.133:0/1430924547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89dc0039e0 0x7f89f813c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.278+0000 7f89f7e00640 1 -- 172.21.15.133:0/1430924547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f813ca60 con 0x7f89f814e4c0 2024-03-07T06:51:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.278+0000 7f89f7e00640 1 --2- 172.21.15.133:0/1430924547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89f814e4c0 0x7f89f814e940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f89f0011490 tx=0x7f89f0011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.278+0000 7f89efe00640 1 -- 172.21.15.133:0/1430924547 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0003610 con 0x7f89f814e4c0 2024-03-07T06:51:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.278+0000 7f89efe00640 1 -- 172.21.15.133:0/1430924547 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f001b070 con 0x7f89f814e4c0 2024-03-07T06:51:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.278+0000 7f89efe00640 1 -- 172.21.15.133:0/1430924547 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f00039f0 con 0x7f89f814e4c0 2024-03-07T06:51:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.278+0000 7f89fe000640 1 -- 172.21.15.133:0/1430924547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f814f240 con 0x7f89f814e4c0 2024-03-07T06:51:16.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.278+0000 7f89fe000640 1 -- 172.21.15.133:0/1430924547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f814f800 con 0x7f89f814e4c0 2024-03-07T06:51:16.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.279+0000 7f89fe000640 1 -- 172.21.15.133:0/1430924547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f80ccf60 con 0x7f89f814e4c0 2024-03-07T06:51:16.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.280+0000 7f89efe00640 1 -- 172.21.15.133:0/1430924547 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f89f0003b90 con 0x7f89f814e4c0 2024-03-07T06:51:16.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.281+0000 7f89efe00640 1 --2- 172.21.15.133:0/1430924547 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f89d406b500 0x7f89d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:16.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.281+0000 7f89efe00640 1 -- 172.21.15.133:0/1430924547 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89f0089f20 con 0x7f89f814e4c0 2024-03-07T06:51:16.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.282+0000 7f89f7400640 1 --2- 172.21.15.133:0/1430924547 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f89d406b500 0x7f89d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:16.281 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.283+0000 7f89f7400640 1 --2- 172.21.15.133:0/1430924547 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f89d406b500 0x7f89d406d9c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f89e800a860 tx=0x7f89e8009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:16.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.286+0000 7f89efe00640 1 -- 172.21.15.133:0/1430924547 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f89f80ccf60 con 0x7f89f814e4c0 2024-03-07T06:51:16.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:16.519+0000 7f89fe000640 1 -- 172.21.15.133:0/1430924547 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f89f8062c90 con 0x7f89f814e4c0 2024-03-07T06:51:17.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.188+0000 7f89efe00640 1 -- 172.21.15.133:0/1430924547 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f89f0024020 con 0x7f89f814e4c0 2024-03-07T06:51:17.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.191+0000 7f89fe000640 1 -- 172.21.15.133:0/1430924547 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f89d406b500 msgr2=0x7f89d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:17.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.191+0000 7f89fe000640 1 --2- 172.21.15.133:0/1430924547 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f89d406b500 0x7f89d406d9c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f89e800a860 tx=0x7f89e8009030 comp rx=0 tx=0).stop 2024-03-07T06:51:17.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.191+0000 7f89fe000640 1 -- 172.21.15.133:0/1430924547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89f814e4c0 msgr2=0x7f89f814e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:17.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.191+0000 7f89fe000640 1 --2- 172.21.15.133:0/1430924547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89f814e4c0 0x7f89f814e940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f89f0011490 tx=0x7f89f0011950 comp rx=0 tx=0).stop 2024-03-07T06:51:17.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.191+0000 7f89fe000640 1 -- 172.21.15.133:0/1430924547 shutdown_connections 2024-03-07T06:51:17.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.191+0000 7f89fe000640 1 --2- 172.21.15.133:0/1430924547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89f814e4c0 0x7f89f814e940 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:17.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.191+0000 7f89fe000640 1 --2- 172.21.15.133:0/1430924547 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89f814b5b0 0x7f89f814ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:17.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.191+0000 7f89fe000640 1 --2- 172.21.15.133:0/1430924547 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f89d406b500 0x7f89d406d9c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:17.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.191+0000 7f89fe000640 1 --2- 172.21.15.133:0/1430924547 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89dc0039e0 0x7f89f813c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:17.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.191+0000 7f89fe000640 1 -- 172.21.15.133:0/1430924547 >> 172.21.15.133:0/1430924547 conn(0x7f89f80b1d30 msgr2=0x7f89f81500a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:17.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.192+0000 7f89fe000640 1 -- 172.21.15.133:0/1430924547 shutdown_connections 2024-03-07T06:51:17.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.192+0000 7f89fe000640 1 -- 172.21.15.133:0/1430924547 wait complete. 2024-03-07T06:51:17.210 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-07T06:51:17.210 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-07T06:51:17.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.423+0000 7f8d9b200640 1 Processor -- start 2024-03-07T06:51:17.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.423+0000 7f8d9b200640 1 -- start start 2024-03-07T06:51:17.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.425+0000 7f8d9b200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d9c159880 0x7f8d9c0cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:17.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.425+0000 7f8d9b200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9c160720 con 0x7f8d9c0ccff0 2024-03-07T06:51:17.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.425+0000 7f8d9b200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9c0cde00 con 0x7f8d9c0b4f10 2024-03-07T06:51:17.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.425+0000 7f8d9b200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0cd560 con 0x7f8d9c159880 2024-03-07T06:51:17.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.425+0000 7f8d99400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8d9c0ccff0 0x7f8d9c160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53200/0 (socket says 172.21.15.133:53200) 2024-03-07T06:51:17.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.425+0000 7f8d99400640 1 -- 172.21.15.133:0/778754000 learned_addr learned my addr 172.21.15.133:0/778754000 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:17.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d99e00640 1 --2- 172.21.15.133:0/778754000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d9c159880 0x7f8d9c0cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:17.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 -- 172.21.15.133:0/778754000 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1914014404 0 0) 0x7f8d9c160720 con 0x7f8d9c0ccff0 2024-03-07T06:51:17.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 -- 172.21.15.133:0/778754000 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d80003520 con 0x7f8d9c0ccff0 2024-03-07T06:51:17.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 -- 172.21.15.133:0/778754000 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3368690456 0 0) 0x7f8d80003520 con 0x7f8d9c0ccff0 2024-03-07T06:51:17.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 -- 172.21.15.133:0/778754000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d9c159880 msgr2=0x7f8d9c0cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:17.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 --2- 172.21.15.133:0/778754000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d9c159880 0x7f8d9c0cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:17.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 -- 172.21.15.133:0/778754000 >> v1:172.21.15.161:6789/0 conn(0x7f8d9c0b4f10 legacy=0x7f8d9c15c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:17.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 -- 172.21.15.133:0/778754000 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c0cd840 con 0x7f8d9c0ccff0 2024-03-07T06:51:17.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 -- 172.21.15.133:0/778754000 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f8d88001670 con 0x7f8d9c0ccff0 2024-03-07T06:51:17.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 -- 172.21.15.133:0/778754000 >> v1:172.21.15.133:6789/0 conn(0x7f8d9c0ccff0 legacy=0x7f8d9c160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:17.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 --2- 172.21.15.133:0/778754000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d800039e0 0x7f8d80003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:17.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 --2- 172.21.15.133:0/778754000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d80004800 0x7f8d80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:17.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 --2- 172.21.15.133:0/778754000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d9c159880 0x7f8d800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:17.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 -- 172.21.15.133:0/778754000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0cd560 con 0x7f8d80004800 2024-03-07T06:51:17.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 -- 172.21.15.133:0/778754000 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8000baa0 con 0x7f8d800039e0 2024-03-07T06:51:17.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.426+0000 7f8d98a00640 1 -- 172.21.15.133:0/778754000 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8000bca0 con 0x7f8d9c159880 2024-03-07T06:51:17.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.427+0000 7f8d99400640 1 --2- 172.21.15.133:0/778754000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d80004800 0x7f8d80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:17.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.427+0000 7f8d99e00640 1 --2- 172.21.15.133:0/778754000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d800039e0 0x7f8d80003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:17.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.427+0000 7f8d9a800640 1 --2- 172.21.15.133:0/778754000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d9c159880 0x7f8d800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:17.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.427+0000 7f8d99400640 1 -- 172.21.15.133:0/778754000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d9c159880 msgr2=0x7f8d800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:17.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.427+0000 7f8d99400640 1 --2- 172.21.15.133:0/778754000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d9c159880 0x7f8d800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:17.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.427+0000 7f8d99400640 1 -- 172.21.15.133:0/778754000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d800039e0 msgr2=0x7f8d80003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:17.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.427+0000 7f8d99400640 1 --2- 172.21.15.133:0/778754000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d800039e0 0x7f8d80003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:17.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.427+0000 7f8d99400640 1 -- 172.21.15.133:0/778754000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d80003520 con 0x7f8d80004800 2024-03-07T06:51:17.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.427+0000 7f8d99e00640 1 --2- 172.21.15.133:0/778754000 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d800039e0 0x7f8d80003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:17.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.427+0000 7f8d9a800640 1 --2- 172.21.15.133:0/778754000 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d9c159880 0x7f8d800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:17.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.428+0000 7f8d99400640 1 --2- 172.21.15.133:0/778754000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d80004800 0x7f8d80004c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800c160 tx=0x7f8d80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e114fc58cd5ceeb server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:17.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.428+0000 7f8d98a00640 1 -- 172.21.15.133:0/778754000 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88007c60 con 0x7f8d80004800 2024-03-07T06:51:17.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.428+0000 7f8d98a00640 1 -- 172.21.15.133:0/778754000 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d88030070 con 0x7f8d80004800 2024-03-07T06:51:17.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.428+0000 7f8d98a00640 1 -- 172.21.15.133:0/778754000 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88005d70 con 0x7f8d80004800 2024-03-07T06:51:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.429+0000 7f8d9b200640 1 -- 172.21.15.133:0/778754000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d80004800 msgr2=0x7f8d80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.429+0000 7f8d9b200640 1 --2- 172.21.15.133:0/778754000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d80004800 0x7f8d80004c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800c160 tx=0x7f8d80005830 comp rx=0 tx=0).stop 2024-03-07T06:51:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.429+0000 7f8d9a800640 1 -- 172.21.15.133:0/778754000 reap_dead start 2024-03-07T06:51:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.429+0000 7f8d9b200640 1 -- 172.21.15.133:0/778754000 shutdown_connections 2024-03-07T06:51:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.429+0000 7f8d9b200640 1 -- 172.21.15.133:0/778754000 >> 172.21.15.133:0/778754000 conn(0x7f8d9c071860 msgr2=0x7f8d9c071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.429+0000 7f8d9b200640 1 -- 172.21.15.133:0/778754000 shutdown_connections 2024-03-07T06:51:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.429+0000 7f8d9b200640 1 -- 172.21.15.133:0/778754000 wait complete. 2024-03-07T06:51:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.431+0000 7f8d9b200640 1 Processor -- start 2024-03-07T06:51:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.431+0000 7f8d9b200640 1 -- start start 2024-03-07T06:51:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d9b200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d80004800 0x7f8d9c14d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:17.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d9b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d9c148530 0x7f8d9c1f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:17.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d9b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d9c1489b0 0x7f8d9c1f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:17.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d9b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c159c60 con 0x7f8d9c148530 2024-03-07T06:51:17.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d9b200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c15faf0 con 0x7f8d9c1489b0 2024-03-07T06:51:17.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d9b200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9c0cc6f0 con 0x7f8d80004800 2024-03-07T06:51:17.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d99400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d9c148530 0x7f8d9c1f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:17.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d99400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d9c148530 0x7f8d9c1f70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46316/0 (socket says 172.21.15.133:46316) 2024-03-07T06:51:17.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d99400640 1 -- 172.21.15.133:0/649923675 learned_addr learned my addr 172.21.15.133:0/649923675 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:17.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d99e00640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d80004800 0x7f8d9c14d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:17.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d9a800640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d9c1489b0 0x7f8d9c1f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:17.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d99400640 1 -- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d80004800 msgr2=0x7f8d9c14d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:17.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d99400640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d80004800 0x7f8d9c14d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:17.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d99400640 1 -- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d9c1489b0 msgr2=0x7f8d9c1f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:17.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d99400640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d9c1489b0 0x7f8d9c1f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:17.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d99400640 1 -- 172.21.15.133:0/649923675 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c149af0 con 0x7f8d9c148530 2024-03-07T06:51:17.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.432+0000 7f8d99e00640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d80004800 0x7f8d9c14d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:17.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.433+0000 7f8d99400640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d9c148530 0x7f8d9c1f70c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800c280 tx=0x7f8d8802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:17.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.433+0000 7f8d92000640 1 -- 172.21.15.133:0/649923675 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88046070 con 0x7f8d9c148530 2024-03-07T06:51:17.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.433+0000 7f8d92000640 1 -- 172.21.15.133:0/649923675 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d88000ed0 con 0x7f8d9c148530 2024-03-07T06:51:17.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.433+0000 7f8d9b200640 1 -- 172.21.15.133:0/649923675 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9c149dd0 con 0x7f8d9c148530 2024-03-07T06:51:17.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.433+0000 7f8d92000640 1 -- 172.21.15.133:0/649923675 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88004e10 con 0x7f8d9c148530 2024-03-07T06:51:17.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.433+0000 7f8d9b200640 1 -- 172.21.15.133:0/649923675 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9c0ad8e0 con 0x7f8d9c148530 2024-03-07T06:51:17.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.434+0000 7f8d9b200640 1 -- 172.21.15.133:0/649923675 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9c067c80 con 0x7f8d9c148530 2024-03-07T06:51:17.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.435+0000 7f8d92000640 1 -- 172.21.15.133:0/649923675 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8d88007070 con 0x7f8d9c148530 2024-03-07T06:51:17.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.436+0000 7f8d92000640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8d7c06b500 0x7f8d7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:17.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.436+0000 7f8d92000640 1 -- 172.21.15.133:0/649923675 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d880ab670 con 0x7f8d9c148530 2024-03-07T06:51:17.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.436+0000 7f8d99e00640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8d7c06b500 0x7f8d7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:17.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.437+0000 7f8d99e00640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8d7c06b500 0x7f8d7c06d9c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009aa0 tx=0x7f8d8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:17.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.440+0000 7f8d92000640 1 -- 172.21.15.133:0/649923675 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8d8807a460 con 0x7f8d9c148530 2024-03-07T06:51:17.673 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:17.674+0000 7f8d9b200640 1 -- 172.21.15.133:0/649923675 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8d9c062c90 con 0x7f8d9c148530 2024-03-07T06:51:18.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.198+0000 7f8d92000640 1 -- 172.21.15.133:0/649923675 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8d88079d60 con 0x7f8d9c148530 2024-03-07T06:51:18.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.201+0000 7f8d9b200640 1 -- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8d7c06b500 msgr2=0x7f8d7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:18.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.201+0000 7f8d9b200640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8d7c06b500 0x7f8d7c06d9c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009aa0 tx=0x7f8d8c008040 comp rx=0 tx=0).stop 2024-03-07T06:51:18.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.201+0000 7f8d9b200640 1 -- 172.21.15.133:0/649923675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d9c148530 msgr2=0x7f8d9c1f70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.201+0000 7f8d9b200640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d9c148530 0x7f8d9c1f70c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800c280 tx=0x7f8d8802e070 comp rx=0 tx=0).stop 2024-03-07T06:51:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.202+0000 7f8d9b200640 1 -- 172.21.15.133:0/649923675 shutdown_connections 2024-03-07T06:51:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.202+0000 7f8d9b200640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8d7c06b500 0x7f8d7c06d9c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.202+0000 7f8d9b200640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d9c1489b0 0x7f8d9c1f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.202+0000 7f8d9b200640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d9c148530 0x7f8d9c1f70c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.202+0000 7f8d9b200640 1 --2- 172.21.15.133:0/649923675 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d80004800 0x7f8d9c14d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.202+0000 7f8d9b200640 1 -- 172.21.15.133:0/649923675 >> 172.21.15.133:0/649923675 conn(0x7f8d9c071860 msgr2=0x7f8d9c0cbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.202+0000 7f8d9b200640 1 -- 172.21.15.133:0/649923675 shutdown_connections 2024-03-07T06:51:18.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.202+0000 7f8d9b200640 1 -- 172.21.15.133:0/649923675 wait complete. 2024-03-07T06:51:18.224 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-07T06:51:18.224 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:51:18.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.428+0000 7ff9b3200640 1 Processor -- start 2024-03-07T06:51:18.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.428+0000 7ff9b3200640 1 -- start start 2024-03-07T06:51:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.430+0000 7ff9b3200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff9b40b70d0 0x7ff9b40b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.430+0000 7ff9b3200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b4143810 con 0x7ff9b40d1af0 2024-03-07T06:51:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.430+0000 7ff9b3200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b4143af0 con 0x7ff9b4140cb0 2024-03-07T06:51:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.430+0000 7ff9b3200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b41435c0 con 0x7ff9b40b70d0 2024-03-07T06:51:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.430+0000 7ff9b1400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff9b40b70d0 0x7ff9b40b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.430+0000 7ff9b2800640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff9b40d1af0 0x7ff9b40d1f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53212/0 (socket says 172.21.15.133:53212) 2024-03-07T06:51:18.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.430+0000 7ff9b2800640 1 -- 172.21.15.133:0/453806284 learned_addr learned my addr 172.21.15.133:0/453806284 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.431+0000 7ff9b1400640 1 -- 172.21.15.133:0/453806284 >> v1:172.21.15.161:6789/0 conn(0x7ff9b4140cb0 legacy=0x7ff9b4162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.431+0000 7ff9b1400640 1 -- 172.21.15.133:0/453806284 >> v1:172.21.15.133:6789/0 conn(0x7ff9b40d1af0 legacy=0x7ff9b40d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.431+0000 7ff9b1400640 1 -- 172.21.15.133:0/453806284 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b4143dd0 con 0x7ff9b40b70d0 2024-03-07T06:51:18.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.431+0000 7ff9b0a00640 1 -- 172.21.15.133:0/453806284 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121196889 0 0) 0x7ff9b4143810 con 0x7ff9b40d1af0 2024-03-07T06:51:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.432+0000 7ff9b1400640 1 --2- 172.21.15.133:0/453806284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b40b70d0 0x7ff9b40b7560 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009d90 tx=0x7ff9a402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a57cdfd8be7dc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.432+0000 7ff9b0a00640 1 -- 172.21.15.133:0/453806284 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4039070 con 0x7ff9b40b70d0 2024-03-07T06:51:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.432+0000 7ff9b0a00640 1 -- 172.21.15.133:0/453806284 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a402cc60 con 0x7ff9b40b70d0 2024-03-07T06:51:18.431 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.432+0000 7ff9b0a00640 1 -- 172.21.15.133:0/453806284 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4034660 con 0x7ff9b40b70d0 2024-03-07T06:51:18.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.433+0000 7ff9b3200640 1 -- 172.21.15.133:0/453806284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b40b70d0 msgr2=0x7ff9b40b7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:18.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.433+0000 7ff9b3200640 1 --2- 172.21.15.133:0/453806284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b40b70d0 0x7ff9b40b7560 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009d90 tx=0x7ff9a402beb0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.433+0000 7ff9b3200640 1 -- 172.21.15.133:0/453806284 shutdown_connections 2024-03-07T06:51:18.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.433+0000 7ff9b3200640 1 --2- 172.21.15.133:0/453806284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b40b70d0 0x7ff9b40b7560 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.433+0000 7ff9b3200640 1 -- 172.21.15.133:0/453806284 >> 172.21.15.133:0/453806284 conn(0x7ff9b40674e0 msgr2=0x7ff9b40d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:18.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.433+0000 7ff9b3200640 1 -- 172.21.15.133:0/453806284 shutdown_connections 2024-03-07T06:51:18.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.433+0000 7ff9b3200640 1 -- 172.21.15.133:0/453806284 wait complete. 2024-03-07T06:51:18.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.435+0000 7ff9b3200640 1 Processor -- start 2024-03-07T06:51:18.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.435+0000 7ff9b3200640 1 -- start start 2024-03-07T06:51:18.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b3200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b40b70d0 0x7ff9b4155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b3200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9b40d1af0 0x7ff9b413fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b4140cb0 0x7ff9b40b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b3200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b4161bb0 con 0x7ff9b4140cb0 2024-03-07T06:51:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b3200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b4144420 con 0x7ff9b40b70d0 2024-03-07T06:51:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b3200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b4161d30 con 0x7ff9b40d1af0 2024-03-07T06:51:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b4140cb0 0x7ff9b40b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b4140cb0 0x7ff9b40b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46328/0 (socket says 172.21.15.133:46328) 2024-03-07T06:51:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b2800640 1 -- 172.21.15.133:0/3681205856 learned_addr learned my addr 172.21.15.133:0/3681205856 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b1e00640 1 --2- 172.21.15.133:0/3681205856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b40b70d0 0x7ff9b4155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b1400640 1 --2- 172.21.15.133:0/3681205856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9b40d1af0 0x7ff9b413fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b2800640 1 -- 172.21.15.133:0/3681205856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9b40d1af0 msgr2=0x7ff9b413fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:18.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b2800640 1 --2- 172.21.15.133:0/3681205856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9b40d1af0 0x7ff9b413fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b2800640 1 -- 172.21.15.133:0/3681205856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b40b70d0 msgr2=0x7ff9b4155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:18.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b2800640 1 --2- 172.21.15.133:0/3681205856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b40b70d0 0x7ff9b4155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.436+0000 7ff9b2800640 1 -- 172.21.15.133:0/3681205856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b41f5510 con 0x7ff9b4140cb0 2024-03-07T06:51:18.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.437+0000 7ff9b2800640 1 --2- 172.21.15.133:0/3681205856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b4140cb0 0x7ff9b40b52b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00c480 tx=0x7ff9ac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:18.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.437+0000 7ff9aa000640 1 -- 172.21.15.133:0/3681205856 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac018070 con 0x7ff9b4140cb0 2024-03-07T06:51:18.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.437+0000 7ff9aa000640 1 -- 172.21.15.133:0/3681205856 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac007500 con 0x7ff9b4140cb0 2024-03-07T06:51:18.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.437+0000 7ff9aa000640 1 -- 172.21.15.133:0/3681205856 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac004a20 con 0x7ff9b4140cb0 2024-03-07T06:51:18.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.437+0000 7ff9b3200640 1 -- 172.21.15.133:0/3681205856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b41f57f0 con 0x7ff9b4140cb0 2024-03-07T06:51:18.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.437+0000 7ff9b3200640 1 -- 172.21.15.133:0/3681205856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b41f5db0 con 0x7ff9b4140cb0 2024-03-07T06:51:18.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.438+0000 7ff9b3200640 1 -- 172.21.15.133:0/3681205856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b4062c90 con 0x7ff9b4140cb0 2024-03-07T06:51:18.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.439+0000 7ff9aa000640 1 -- 172.21.15.133:0/3681205856 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff9ac01b070 con 0x7ff9b4140cb0 2024-03-07T06:51:18.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.440+0000 7ff9aa000640 1 --2- 172.21.15.133:0/3681205856 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff99006b4d0 0x7ff99006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:18.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.440+0000 7ff9aa000640 1 -- 172.21.15.133:0/3681205856 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9ac089540 con 0x7ff9b4140cb0 2024-03-07T06:51:18.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.440+0000 7ff9b1e00640 1 --2- 172.21.15.133:0/3681205856 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff99006b4d0 0x7ff99006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:18.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.441+0000 7ff9b1e00640 1 --2- 172.21.15.133:0/3681205856 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff99006b4d0 0x7ff99006d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff99c006fd0 tx=0x7ff99c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:18.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.444+0000 7ff9aa000640 1 -- 172.21.15.133:0/3681205856 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff9ac0583b0 con 0x7ff9b4140cb0 2024-03-07T06:51:18.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.680+0000 7ff9b3200640 1 -- 172.21.15.133:0/3681205856 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9b4071090 con 0x7ff9b4140cb0 2024-03-07T06:51:18.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.681+0000 7ff9aa000640 1 -- 172.21.15.133:0/3681205856 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4983 (secure 0 0 0) 0x7ff9ac057cb0 con 0x7ff9b4140cb0 2024-03-07T06:51:18.680 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:18.680 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1116649196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1116649196},{"type":"v1","addr":"172.21.15.161:6835","nonce":1116649196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3485625105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3485625105},{"type":"v1","addr":"172.21.15.133:6835","nonce":3485625105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3866978630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3866978630},{"type":"v1","addr":"172.21.15.133:6833","nonce":3866978630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:51:14.594513+0000","modified":"2024-03-07T06:51:17.675044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2885273912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2885273912},{"type":"v1","addr":"172.21.15.161:6837","nonce":2885273912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-03-07T06:51:18.681 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 8 2024-03-07T06:51:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.684+0000 7ff9b3200640 1 -- 172.21.15.133:0/3681205856 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff99006b4d0 msgr2=0x7ff99006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.684+0000 7ff9b3200640 1 --2- 172.21.15.133:0/3681205856 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff99006b4d0 0x7ff99006d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff99c006fd0 tx=0x7ff99c008040 comp rx=0 tx=0).stop 2024-03-07T06:51:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.684+0000 7ff9b3200640 1 -- 172.21.15.133:0/3681205856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b4140cb0 msgr2=0x7ff9b40b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.684+0000 7ff9b3200640 1 --2- 172.21.15.133:0/3681205856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b4140cb0 0x7ff9b40b52b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00c480 tx=0x7ff9ac00c940 comp rx=0 tx=0).stop 2024-03-07T06:51:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.684+0000 7ff9b3200640 1 -- 172.21.15.133:0/3681205856 shutdown_connections 2024-03-07T06:51:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.684+0000 7ff9b3200640 1 --2- 172.21.15.133:0/3681205856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9b4140cb0 0x7ff9b40b52b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.684+0000 7ff9b3200640 1 --2- 172.21.15.133:0/3681205856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9b40d1af0 0x7ff9b413fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.684+0000 7ff9b3200640 1 --2- 172.21.15.133:0/3681205856 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff99006b4d0 0x7ff99006d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.684+0000 7ff9b3200640 1 --2- 172.21.15.133:0/3681205856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b40b70d0 0x7ff9b4155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.684+0000 7ff9b3200640 1 -- 172.21.15.133:0/3681205856 >> 172.21.15.133:0/3681205856 conn(0x7ff9b40674e0 msgr2=0x7ff9b4160980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:18.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.685+0000 7ff9b3200640 1 -- 172.21.15.133:0/3681205856 shutdown_connections 2024-03-07T06:51:18.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.685+0000 7ff9b3200640 1 -- 172.21.15.133:0/3681205856 wait complete. 2024-03-07T06:51:18.702 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:18.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.911+0000 7f88bf400640 1 Processor -- start 2024-03-07T06:51:18.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.911+0000 7f88bf400640 1 -- start start 2024-03-07T06:51:18.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.913+0000 7f88bf400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f88b80cf7d0 0x7f88b80cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:18.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.913+0000 7f88bf400640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b80731b0 con 0x7f88b8072950 2024-03-07T06:51:18.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.913+0000 7f88bf400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b8072150 con 0x7f88b815fbb0 2024-03-07T06:51:18.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.913+0000 7f88bf400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b8072f60 con 0x7f88b80cf7d0 2024-03-07T06:51:18.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.913+0000 7f88b7e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f88b8072950 0x7f88b815c550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53226/0 (socket says 172.21.15.133:53226) 2024-03-07T06:51:18.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.913+0000 7f88b7e00640 1 -- 172.21.15.133:0/3416920222 learned_addr learned my addr 172.21.15.133:0/3416920222 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:18.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.913+0000 7f88bd400640 1 --2- 172.21.15.133:0/3416920222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f88b80cf7d0 0x7f88b80cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:18.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.913+0000 7f88b7400640 1 -- 172.21.15.133:0/3416920222 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024886282 0 0) 0x7f88b80731b0 con 0x7f88b8072950 2024-03-07T06:51:18.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.913+0000 7f88b7400640 1 -- 172.21.15.133:0/3416920222 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88a0003520 con 0x7f88b8072950 2024-03-07T06:51:18.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88b7400640 1 -- 172.21.15.133:0/3416920222 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2471345934 0 0) 0x7f88a0003520 con 0x7f88b8072950 2024-03-07T06:51:18.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88b7400640 1 -- 172.21.15.133:0/3416920222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f88b80cf7d0 msgr2=0x7f88b80cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:18.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88b7400640 1 --2- 172.21.15.133:0/3416920222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f88b80cf7d0 0x7f88b80cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88b7400640 1 -- 172.21.15.133:0/3416920222 >> v1:172.21.15.161:6789/0 conn(0x7f88b815fbb0 legacy=0x7f88b815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:18.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88b7400640 1 -- 172.21.15.133:0/3416920222 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b8072430 con 0x7f88b8072950 2024-03-07T06:51:18.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88b7400640 1 -- 172.21.15.133:0/3416920222 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f88a4001690 con 0x7f88b8072950 2024-03-07T06:51:18.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88b7400640 1 -- 172.21.15.133:0/3416920222 >> v1:172.21.15.133:6789/0 conn(0x7f88b8072950 legacy=0x7f88b815c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:18.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88b7400640 1 --2- 172.21.15.133:0/3416920222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88a00039e0 0x7f88a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:18.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88b7400640 1 --2- 172.21.15.133:0/3416920222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88a0004800 0x7f88a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:18.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88b7400640 1 --2- 172.21.15.133:0/3416920222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f88b80cf7d0 0x7f88a0005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:18.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88b7400640 1 -- 172.21.15.133:0/3416920222 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b8072f60 con 0x7f88a00039e0 2024-03-07T06:51:18.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88b7400640 1 -- 172.21.15.133:0/3416920222 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a000bb80 con 0x7f88a0004800 2024-03-07T06:51:18.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88b7400640 1 -- 172.21.15.133:0/3416920222 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a000bd80 con 0x7f88b80cf7d0 2024-03-07T06:51:18.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88bca00640 1 --2- 172.21.15.133:0/3416920222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88a00039e0 0x7f88a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:18.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.914+0000 7f88b7e00640 1 --2- 172.21.15.133:0/3416920222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88a0004800 0x7f88a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:18.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.915+0000 7f88bca00640 1 -- 172.21.15.133:0/3416920222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f88b80cf7d0 msgr2=0x7f88a0005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:18.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.915+0000 7f88bca00640 1 --2- 172.21.15.133:0/3416920222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f88b80cf7d0 0x7f88a0005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.915+0000 7f88bca00640 1 -- 172.21.15.133:0/3416920222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88a0004800 msgr2=0x7f88a0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:18.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.915+0000 7f88bca00640 1 --2- 172.21.15.133:0/3416920222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88a0004800 0x7f88a0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.915+0000 7f88bca00640 1 -- 172.21.15.133:0/3416920222 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a0003520 con 0x7f88a00039e0 2024-03-07T06:51:18.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.916+0000 7f88bca00640 1 --2- 172.21.15.133:0/3416920222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88a00039e0 0x7f88a0003e00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88ac001750 tx=0x7f88a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a844e5fde95a31 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:18.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.916+0000 7f88b7400640 1 -- 172.21.15.133:0/3416920222 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac001370 con 0x7f88a00039e0 2024-03-07T06:51:18.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.916+0000 7f88b7400640 1 -- 172.21.15.133:0/3416920222 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac0052d0 con 0x7f88a00039e0 2024-03-07T06:51:18.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.916+0000 7f88b7400640 1 -- 172.21.15.133:0/3416920222 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac003830 con 0x7f88a00039e0 2024-03-07T06:51:18.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.916+0000 7f88bf400640 1 -- 172.21.15.133:0/3416920222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88a00039e0 msgr2=0x7f88a0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:18.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.916+0000 7f88bf400640 1 --2- 172.21.15.133:0/3416920222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88a00039e0 0x7f88a0003e00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f88ac001750 tx=0x7f88a0005830 comp rx=0 tx=0).stop 2024-03-07T06:51:18.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.916+0000 7f88bd400640 1 -- 172.21.15.133:0/3416920222 reap_dead start 2024-03-07T06:51:18.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.916+0000 7f88bf400640 1 -- 172.21.15.133:0/3416920222 shutdown_connections 2024-03-07T06:51:18.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.916+0000 7f88bf400640 1 -- 172.21.15.133:0/3416920222 >> 172.21.15.133:0/3416920222 conn(0x7f88b80b3120 msgr2=0x7f88b80b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:18.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.917+0000 7f88bf400640 1 -- 172.21.15.133:0/3416920222 shutdown_connections 2024-03-07T06:51:18.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.917+0000 7f88bf400640 1 -- 172.21.15.133:0/3416920222 wait complete. 2024-03-07T06:51:18.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.918+0000 7f88bf400640 1 Processor -- start 2024-03-07T06:51:18.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.918+0000 7f88bf400640 1 -- start start 2024-03-07T06:51:18.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.919+0000 7f88bf400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88a00039e0 0x7f88b8148810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:18.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.919+0000 7f88bf400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f88b814ba30 0x7f88b80cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:18.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.919+0000 7f88bf400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b80b59c0 0x7f88b80b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.919+0000 7f88bf400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b8073490 con 0x7f88a00039e0 2024-03-07T06:51:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.919+0000 7f88bf400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80cfc00 con 0x7f88b80b59c0 2024-03-07T06:51:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.919+0000 7f88bf400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80b1a00 con 0x7f88b814ba30 2024-03-07T06:51:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.920+0000 7f88bca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88a00039e0 0x7f88b8148810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.920+0000 7f88bca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88a00039e0 0x7f88b8148810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46358/0 (socket says 172.21.15.133:46358) 2024-03-07T06:51:18.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.920+0000 7f88bca00640 1 -- 172.21.15.133:0/459973583 learned_addr learned my addr 172.21.15.133:0/459973583 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.920+0000 7f88bd400640 1 --2- 172.21.15.133:0/459973583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b80b59c0 0x7f88b80b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.920+0000 7f88b7e00640 1 --2- 172.21.15.133:0/459973583 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f88b814ba30 0x7f88b80cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.920+0000 7f88bca00640 1 -- 172.21.15.133:0/459973583 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f88b814ba30 msgr2=0x7f88b80cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.920+0000 7f88bca00640 1 --2- 172.21.15.133:0/459973583 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f88b814ba30 0x7f88b80cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.920+0000 7f88bca00640 1 -- 172.21.15.133:0/459973583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b80b59c0 msgr2=0x7f88b80b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.920+0000 7f88bca00640 1 --2- 172.21.15.133:0/459973583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b80b59c0 0x7f88b80b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:18.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.920+0000 7f88bca00640 1 -- 172.21.15.133:0/459973583 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b80c98f0 con 0x7f88a00039e0 2024-03-07T06:51:18.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.920+0000 7f88bca00640 1 --2- 172.21.15.133:0/459973583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88a00039e0 0x7f88b8148810 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f88ac008fd0 tx=0x7f88ac001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:18.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.921+0000 7f88b4c00640 1 -- 172.21.15.133:0/459973583 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac041070 con 0x7f88a00039e0 2024-03-07T06:51:18.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.921+0000 7f88b4c00640 1 -- 172.21.15.133:0/459973583 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac005480 con 0x7f88a00039e0 2024-03-07T06:51:18.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.921+0000 7f88b4c00640 1 -- 172.21.15.133:0/459973583 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ac001370 con 0x7f88a00039e0 2024-03-07T06:51:18.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.921+0000 7f88bf400640 1 -- 172.21.15.133:0/459973583 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b80cb300 con 0x7f88a00039e0 2024-03-07T06:51:18.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.921+0000 7f88bf400640 1 -- 172.21.15.133:0/459973583 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b80cb8c0 con 0x7f88a00039e0 2024-03-07T06:51:18.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.922+0000 7f88bf400640 1 -- 172.21.15.133:0/459973583 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b80b6380 con 0x7f88a00039e0 2024-03-07T06:51:18.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.923+0000 7f88b4c00640 1 -- 172.21.15.133:0/459973583 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f88ac044070 con 0x7f88a00039e0 2024-03-07T06:51:18.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.923+0000 7f88b4c00640 1 --2- 172.21.15.133:0/459973583 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f889806b4d0 0x7f889806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:18.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.924+0000 7f88b4c00640 1 -- 172.21.15.133:0/459973583 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88ac0abd50 con 0x7f88a00039e0 2024-03-07T06:51:18.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.924+0000 7f88b7e00640 1 --2- 172.21.15.133:0/459973583 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f889806b4d0 0x7f889806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:18.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.925+0000 7f88b7e00640 1 --2- 172.21.15.133:0/459973583 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f889806b4d0 0x7f889806d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88a4001800 tx=0x7f88a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:18.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:18.928+0000 7f88b4c00640 1 -- 172.21.15.133:0/459973583 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f88ac07ab30 con 0x7f88a00039e0 2024-03-07T06:51:19.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.125+0000 7f88bf400640 1 -- 172.21.15.133:0/459973583 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88b8062c90 con 0x7f88a00039e0 2024-03-07T06:51:19.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.127+0000 7f88b4c00640 1 -- 172.21.15.133:0/459973583 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f88ac07a430 con 0x7f88a00039e0 2024-03-07T06:51:19.127 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:19.127 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":21,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:16.174650+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.130+0000 7f88bf400640 1 -- 172.21.15.133:0/459973583 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f889806b4d0 msgr2=0x7f889806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.130+0000 7f88bf400640 1 --2- 172.21.15.133:0/459973583 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f889806b4d0 0x7f889806d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88a4001800 tx=0x7f88a4009040 comp rx=0 tx=0).stop 2024-03-07T06:51:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.130+0000 7f88bf400640 1 -- 172.21.15.133:0/459973583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88a00039e0 msgr2=0x7f88b8148810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.130+0000 7f88bf400640 1 --2- 172.21.15.133:0/459973583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88a00039e0 0x7f88b8148810 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f88ac008fd0 tx=0x7f88ac001120 comp rx=0 tx=0).stop 2024-03-07T06:51:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.131+0000 7f88bf400640 1 -- 172.21.15.133:0/459973583 shutdown_connections 2024-03-07T06:51:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.131+0000 7f88bf400640 1 --2- 172.21.15.133:0/459973583 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f889806b4d0 0x7f889806d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.131+0000 7f88bf400640 1 --2- 172.21.15.133:0/459973583 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f88b80b59c0 0x7f88b80b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.131+0000 7f88bf400640 1 --2- 172.21.15.133:0/459973583 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f88b814ba30 0x7f88b80cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.131+0000 7f88bf400640 1 --2- 172.21.15.133:0/459973583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88a00039e0 0x7f88b8148810 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.131+0000 7f88bf400640 1 -- 172.21.15.133:0/459973583 >> 172.21.15.133:0/459973583 conn(0x7f88b80b3120 msgr2=0x7f88b815eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.131+0000 7f88bf400640 1 -- 172.21.15.133:0/459973583 shutdown_connections 2024-03-07T06:51:19.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.131+0000 7f88bf400640 1 -- 172.21.15.133:0/459973583 wait complete. 2024-03-07T06:51:19.149 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-07T06:51:19.149 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:51:19.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.362+0000 7f94a1a00640 1 Processor -- start 2024-03-07T06:51:19.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.362+0000 7f94a1a00640 1 -- start start 2024-03-07T06:51:19.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.364+0000 7f94a1a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f949c0af950 0x7f949c0b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:19.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.364+0000 7f94a1a00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949c06f140 con 0x7f949c15b1b0 2024-03-07T06:51:19.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.364+0000 7f94a1a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949c06f420 con 0x7f949c0ce5f0 2024-03-07T06:51:19.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.364+0000 7f94a1a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949c071610 con 0x7f949c0af950 2024-03-07T06:51:19.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.364+0000 7f949aa00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f949c15b1b0 0x7f949c0cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53234/0 (socket says 172.21.15.133:53234) 2024-03-07T06:51:19.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.364+0000 7f949aa00640 1 -- 172.21.15.133:0/2818655941 learned_addr learned my addr 172.21.15.133:0/2818655941 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949b400640 1 --2- 172.21.15.133:0/2818655941 >> v2:172.21.15.161:3300/0 conn(0x7f949c0af950 0x7f949c0b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 -- 172.21.15.133:0/2818655941 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220094094 0 0) 0x7f949c06f140 con 0x7f949c15b1b0 2024-03-07T06:51:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 -- 172.21.15.133:0/2818655941 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9480003520 con 0x7f949c15b1b0 2024-03-07T06:51:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 -- 172.21.15.133:0/2818655941 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2054627397 0 0) 0x7f9480003520 con 0x7f949c15b1b0 2024-03-07T06:51:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 -- 172.21.15.133:0/2818655941 >> v2:172.21.15.161:3300/0 conn(0x7f949c0af950 msgr2=0x7f949c0b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 --2- 172.21.15.133:0/2818655941 >> v2:172.21.15.161:3300/0 conn(0x7f949c0af950 0x7f949c0b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 -- 172.21.15.133:0/2818655941 >> v1:172.21.15.161:6789/0 conn(0x7f949c0ce5f0 legacy=0x7f949c071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:19.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 -- 172.21.15.133:0/2818655941 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c071860 con 0x7f949c15b1b0 2024-03-07T06:51:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 -- 172.21.15.133:0/2818655941 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f948c001670 con 0x7f949c15b1b0 2024-03-07T06:51:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 -- 172.21.15.133:0/2818655941 >> v1:172.21.15.133:6789/0 conn(0x7f949c15b1b0 legacy=0x7f949c0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 --2- 172.21.15.133:0/2818655941 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f94800039b0 0x7f9480003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 --2- 172.21.15.133:0/2818655941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9480004800 0x7f9480004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 --2- 172.21.15.133:0/2818655941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94800051c0 0x7f948000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 -- 172.21.15.133:0/2818655941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c071610 con 0x7f94800051c0 2024-03-07T06:51:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 -- 172.21.15.133:0/2818655941 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948000e370 con 0x7f9480004800 2024-03-07T06:51:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949a000640 1 -- 172.21.15.133:0/2818655941 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948000e570 con 0x7f94800039b0 2024-03-07T06:51:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.365+0000 7f949b400640 1 --2- 172.21.15.133:0/2818655941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94800051c0 0x7f948000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.366+0000 7f949b400640 1 -- 172.21.15.133:0/2818655941 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f94800039b0 msgr2=0x7f9480003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.366+0000 7f949b400640 1 --2- 172.21.15.133:0/2818655941 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f94800039b0 0x7f9480003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.366+0000 7f949b400640 1 -- 172.21.15.133:0/2818655941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9480004800 msgr2=0x7f9480004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.366+0000 7f949b400640 1 --2- 172.21.15.133:0/2818655941 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9480004800 0x7f9480004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.366+0000 7f949b400640 1 -- 172.21.15.133:0/2818655941 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9480003520 con 0x7f94800051c0 2024-03-07T06:51:19.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.367+0000 7f949b400640 1 -- 172.21.15.133:0/2818655941 reap_dead start 2024-03-07T06:51:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.367+0000 7f949b400640 1 --2- 172.21.15.133:0/2818655941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94800051c0 0x7f948000dbd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9494009fc0 tx=0x7f9480005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cffd4848bbb6d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.367+0000 7f949a000640 1 -- 172.21.15.133:0/2818655941 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494009c10 con 0x7f94800051c0 2024-03-07T06:51:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.367+0000 7f949a000640 1 -- 172.21.15.133:0/2818655941 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949402cca0 con 0x7f94800051c0 2024-03-07T06:51:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.367+0000 7f949a000640 1 -- 172.21.15.133:0/2818655941 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494003bd0 con 0x7f94800051c0 2024-03-07T06:51:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.368+0000 7f94a1a00640 1 -- 172.21.15.133:0/2818655941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94800051c0 msgr2=0x7f948000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.368+0000 7f94a1a00640 1 --2- 172.21.15.133:0/2818655941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94800051c0 0x7f948000dbd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9494009fc0 tx=0x7f9480005830 comp rx=0 tx=0).stop 2024-03-07T06:51:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.368+0000 7f94a1a00640 1 -- 172.21.15.133:0/2818655941 shutdown_connections 2024-03-07T06:51:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.368+0000 7f94a1a00640 1 --2- 172.21.15.133:0/2818655941 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94800051c0 0x7f948000dbd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.368+0000 7f94a1a00640 1 -- 172.21.15.133:0/2818655941 >> 172.21.15.133:0/2818655941 conn(0x7f949c0b3120 msgr2=0x7f949c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:19.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.368+0000 7f94a1a00640 1 -- 172.21.15.133:0/2818655941 shutdown_connections 2024-03-07T06:51:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.368+0000 7f94a1a00640 1 -- 172.21.15.133:0/2818655941 wait complete. 2024-03-07T06:51:19.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.370+0000 7f94a1a00640 1 Processor -- start 2024-03-07T06:51:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.370+0000 7f94a1a00640 1 -- start start 2024-03-07T06:51:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f94a1a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94800051c0 0x7f949c1f00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f94a1a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f949c15c620 0x7f949c0b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f94a1a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f949c0b4640 0x7f949c1535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f94a1a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c15b710 con 0x7f94800051c0 2024-03-07T06:51:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f94a1a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c15b590 con 0x7f949c0b4640 2024-03-07T06:51:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f94a1a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0b3c10 con 0x7f949c15c620 2024-03-07T06:51:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f949aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94800051c0 0x7f949c1f00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f949aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94800051c0 0x7f949c1f00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46370/0 (socket says 172.21.15.133:46370) 2024-03-07T06:51:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f949aa00640 1 -- 172.21.15.133:0/4080640591 learned_addr learned my addr 172.21.15.133:0/4080640591 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:19.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f949b400640 1 --2- 172.21.15.133:0/4080640591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f949c0b4640 0x7f949c1535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f9493e00640 1 --2- 172.21.15.133:0/4080640591 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f949c15c620 0x7f949c0b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f949aa00640 1 -- 172.21.15.133:0/4080640591 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f949c15c620 msgr2=0x7f949c0b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f949aa00640 1 --2- 172.21.15.133:0/4080640591 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f949c15c620 0x7f949c0b5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f949aa00640 1 -- 172.21.15.133:0/4080640591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f949c0b4640 msgr2=0x7f949c1535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f949aa00640 1 --2- 172.21.15.133:0/4080640591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f949c0b4640 0x7f949c1535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.371+0000 7f949aa00640 1 -- 172.21.15.133:0/4080640591 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c15cb70 con 0x7f94800051c0 2024-03-07T06:51:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.372+0000 7f949aa00640 1 --2- 172.21.15.133:0/4080640591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94800051c0 0x7f949c1f00d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f948c00c6f0 tx=0x7f948c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.372+0000 7f9492a00640 1 -- 172.21.15.133:0/4080640591 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c018070 con 0x7f94800051c0 2024-03-07T06:51:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.372+0000 7f9492a00640 1 -- 172.21.15.133:0/4080640591 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948c0072e0 con 0x7f94800051c0 2024-03-07T06:51:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.372+0000 7f94a1a00640 1 -- 172.21.15.133:0/4080640591 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949c15ce50 con 0x7f94800051c0 2024-03-07T06:51:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.372+0000 7f9492a00640 1 -- 172.21.15.133:0/4080640591 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c0047e0 con 0x7f94800051c0 2024-03-07T06:51:19.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.372+0000 7f94a1a00640 1 -- 172.21.15.133:0/4080640591 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c1efbb0 con 0x7f94800051c0 2024-03-07T06:51:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.373+0000 7f94a1a00640 1 -- 172.21.15.133:0/4080640591 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949c1efdb0 con 0x7f94800051c0 2024-03-07T06:51:19.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.374+0000 7f9492a00640 1 -- 172.21.15.133:0/4080640591 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f948c01b070 con 0x7f94800051c0 2024-03-07T06:51:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.375+0000 7f9492a00640 1 --2- 172.21.15.133:0/4080640591 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f947806b500 0x7f947806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.376+0000 7f9492a00640 1 -- 172.21.15.133:0/4080640591 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f948c089620 con 0x7f94800051c0 2024-03-07T06:51:19.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.376+0000 7f9493e00640 1 --2- 172.21.15.133:0/4080640591 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f947806b500 0x7f947806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:19.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.377+0000 7f9493e00640 1 --2- 172.21.15.133:0/4080640591 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f947806b500 0x7f947806d9c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f949c157d20 tx=0x7f9484009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:19.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.380+0000 7f9492a00640 1 -- 172.21.15.133:0/4080640591 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f949c1efdb0 con 0x7f94800051c0 2024-03-07T06:51:19.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.570+0000 7f94a1a00640 1 -- 172.21.15.133:0/4080640591 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f949c062c90 con 0x7f947806b500 2024-03-07T06:51:19.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.591+0000 7f9492a00640 1 -- 172.21.15.133:0/4080640591 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240071 (secure 0 0 0) 0x7f949c062c90 con 0x7f947806b500 2024-03-07T06:51:19.594 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:19.595 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:51:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.596+0000 7f94a1a00640 1 -- 172.21.15.133:0/4080640591 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f947806b500 msgr2=0x7f947806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.596+0000 7f94a1a00640 1 --2- 172.21.15.133:0/4080640591 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f947806b500 0x7f947806d9c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f949c157d20 tx=0x7f9484009040 comp rx=0 tx=0).stop 2024-03-07T06:51:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.596+0000 7f94a1a00640 1 -- 172.21.15.133:0/4080640591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94800051c0 msgr2=0x7f949c1f00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.596+0000 7f94a1a00640 1 --2- 172.21.15.133:0/4080640591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94800051c0 0x7f949c1f00d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f948c00c6f0 tx=0x7f948c00cbb0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.596+0000 7f94a1a00640 1 -- 172.21.15.133:0/4080640591 shutdown_connections 2024-03-07T06:51:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.596+0000 7f94a1a00640 1 --2- 172.21.15.133:0/4080640591 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f947806b500 0x7f947806d9c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.596+0000 7f94a1a00640 1 --2- 172.21.15.133:0/4080640591 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f949c0b4640 0x7f949c1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.596+0000 7f94a1a00640 1 --2- 172.21.15.133:0/4080640591 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f949c15c620 0x7f949c0b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.596+0000 7f94a1a00640 1 --2- 172.21.15.133:0/4080640591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94800051c0 0x7f949c1f00d0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f948c00c6f0 tx=0x7f948c00cbb0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.597+0000 7f94a1a00640 1 -- 172.21.15.133:0/4080640591 >> 172.21.15.133:0/4080640591 conn(0x7f949c0b3120 msgr2=0x7f949c0cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:19.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.597+0000 7f94a1a00640 1 -- 172.21.15.133:0/4080640591 shutdown_connections 2024-03-07T06:51:19.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.597+0000 7f94a1a00640 1 -- 172.21.15.133:0/4080640591 wait complete. 2024-03-07T06:51:19.615 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-03-07T06:51:18.470591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219556,"kb_used_data":2532,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755164,"statfs":{"total":773094113280,"available":772869287936,"internally_reserved":0,"allocated":2592768,"data_stored":1613325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.344667"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.131807+0000","last_change":"2024-03-07T06:51:13.131807+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.131807+0000","last_undegraded":"2024-03-07T06:51:13.131807+0000","last_fullsized":"2024-03-07T06:51:13.131807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.135007+0000","last_change":"2024-03-07T06:51:13.135007+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.135007+0000","last_undegraded":"2024-03-07T06:51:13.135007+0000","last_fullsized":"2024-03-07T06:51:13.135007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.130942+0000","last_change":"2024-03-07T06:51:13.130942+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.130942+0000","last_undegraded":"2024-03-07T06:51:13.130942+0000","last_fullsized":"2024-03-07T06:51:13.130942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.152124+0000","last_change":"2024-03-07T06:51:13.152124+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.152124+0000","last_undegraded":"2024-03-07T06:51:13.152124+0000","last_fullsized":"2024-03-07T06:51:13.152124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.138886+0000","last_change":"2024-03-07T06:51:13.138886+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.138886+0000","last_undegraded":"2024-03-07T06:51:13.138886+0000","last_fullsized":"2024-03-07T06:51:13.138886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.151197+0000","last_change":"2024-03-07T06:51:13.151197+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.151197+0000","last_undegraded":"2024-03-07T06:51:13.151197+0000","last_fullsized":"2024-03-07T06:51:13.151197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.147533+0000","last_change":"2024-03-07T06:51:13.147533+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.147533+0000","last_undegraded":"2024-03-07T06:51:13.147533+0000","last_fullsized":"2024-03-07T06:51:13.147533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.135577+0000","last_change":"2024-03-07T06:51:13.135577+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.135577+0000","last_undegraded":"2024-03-07T06:51:13.135577+0000","last_fullsized":"2024-03-07T06:51:13.135577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.133983+0000","last_change":"2024-03-07T06:51:13.133983+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.133983+0000","last_undegraded":"2024-03-07T06:51:13.133983+0000","last_fullsized":"2024-03-07T06:51:13.133983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.153228+0000","last_change":"2024-03-07T06:51:13.153228+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.153228+0000","last_undegraded":"2024-03-07T06:51:13.153228+0000","last_fullsized":"2024-03-07T06:51:13.153228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.135108+0000","last_change":"2024-03-07T06:51:13.135108+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.135108+0000","last_undegraded":"2024-03-07T06:51:13.135108+0000","last_fullsized":"2024-03-07T06:51:13.135108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.154393+0000","last_change":"2024-03-07T06:51:13.154393+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.154393+0000","last_undegraded":"2024-03-07T06:51:13.154393+0000","last_fullsized":"2024-03-07T06:51:13.154393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.134885+0000","last_change":"2024-03-07T06:51:13.134885+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.134885+0000","last_undegraded":"2024-03-07T06:51:13.134885+0000","last_fullsized":"2024-03-07T06:51:13.134885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.154552+0000","last_change":"2024-03-07T06:51:13.154552+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.154552+0000","last_undegraded":"2024-03-07T06:51:13.154552+0000","last_fullsized":"2024-03-07T06:51:13.154552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.145614+0000","last_change":"2024-03-07T06:51:13.145614+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.145614+0000","last_undegraded":"2024-03-07T06:51:13.145614+0000","last_fullsized":"2024-03-07T06:51:13.145614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.154252+0000","last_change":"2024-03-07T06:51:13.154252+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.154252+0000","last_undegraded":"2024-03-07T06:51:13.154252+0000","last_fullsized":"2024-03-07T06:51:13.154252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-03-07T06:51:13.147631+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:51:13.147631+0000","last_peered":"2024-03-07T06:51:13.147631+0000","last_clean":"2024-03-07T06:51:13.147631+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:51:13.147631+0000","last_undegraded":"2024-03-07T06:51:13.147631+0000","last_fullsized":"2024-03-07T06:51:13.147631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.151267+0000","last_change":"2024-03-07T06:51:13.151267+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.151267+0000","last_undegraded":"2024-03-07T06:51:13.151267+0000","last_fullsized":"2024-03-07T06:51:13.151267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.141256+0000","last_change":"2024-03-07T06:51:13.141256+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.141256+0000","last_undegraded":"2024-03-07T06:51:13.141256+0000","last_fullsized":"2024-03-07T06:51:13.141256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.155367+0000","last_change":"2024-03-07T06:51:13.155367+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.155367+0000","last_undegraded":"2024-03-07T06:51:13.155367+0000","last_fullsized":"2024-03-07T06:51:13.155367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.151378+0000","last_change":"2024-03-07T06:51:13.151378+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.151378+0000","last_undegraded":"2024-03-07T06:51:13.151378+0000","last_fullsized":"2024-03-07T06:51:13.151378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.144999+0000","last_change":"2024-03-07T06:51:13.144999+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.144999+0000","last_undegraded":"2024-03-07T06:51:13.144999+0000","last_fullsized":"2024-03-07T06:51:13.144999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.143900+0000","last_change":"2024-03-07T06:51:13.143900+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.143900+0000","last_undegraded":"2024-03-07T06:51:13.143900+0000","last_fullsized":"2024-03-07T06:51:13.143900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.145725+0000","last_change":"2024-03-07T06:51:13.145725+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.145725+0000","last_undegraded":"2024-03-07T06:51:13.145725+0000","last_fullsized":"2024-03-07T06:51:13.145725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.136804+0000","last_change":"2024-03-07T06:51:13.136804+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.136804+0000","last_undegraded":"2024-03-07T06:51:13.136804+0000","last_fullsized":"2024-03-07T06:51:13.136804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.142872+0000","last_change":"2024-03-07T06:51:13.142872+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.142872+0000","last_undegraded":"2024-03-07T06:51:13.142872+0000","last_fullsized":"2024-03-07T06:51:13.142872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.144981+0000","last_change":"2024-03-07T06:51:13.144981+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.144981+0000","last_undegraded":"2024-03-07T06:51:13.144981+0000","last_fullsized":"2024-03-07T06:51:13.144981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.145384+0000","last_change":"2024-03-07T06:51:13.145384+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.145384+0000","last_undegraded":"2024-03-07T06:51:13.145384+0000","last_fullsized":"2024-03-07T06:51:13.145384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.142224+0000","last_change":"2024-03-07T06:51:13.142224+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.142224+0000","last_undegraded":"2024-03-07T06:51:13.142224+0000","last_fullsized":"2024-03-07T06:51:13.142224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.141804+0000","last_change":"2024-03-07T06:51:13.141804+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.141804+0000","last_undegraded":"2024-03-07T06:51:13.141804+0000","last_fullsized":"2024-03-07T06:51:13.141804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.146901+0000","last_change":"2024-03-07T06:51:13.146901+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.146901+0000","last_undegraded":"2024-03-07T06:51:13.146901+0000","last_fullsized":"2024-03-07T06:51:13.146901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.130977+0000","last_change":"2024-03-07T06:51:13.130977+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.130977+0000","last_undegraded":"2024-03-07T06:51:13.130977+0000","last_fullsized":"2024-03-07T06:51:13.130977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.134194+0000","last_change":"2024-03-07T06:51:13.134194+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.134194+0000","last_undegraded":"2024-03-07T06:51:13.134194+0000","last_fullsized":"2024-03-07T06:51:13.134194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28384,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343456,"statfs":{"total":96636764160,"available":96607698944,"internally_reserved":0,"allocated":761856,"data_stored":640806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":761856,"data_stored":640806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:51:19.619 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:51:19.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.859+0000 7f87b3800640 1 Processor -- start 2024-03-07T06:51:19.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.859+0000 7f87b3800640 1 -- start start 2024-03-07T06:51:19.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.861+0000 7f87b3800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87ac072950 0x7f87ac0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:19.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.861+0000 7f87b3800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f87ac0cdee0 0x7f87ac15c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:19.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.861+0000 7f87b3800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ac0731b0 con 0x7f87ac15fbb0 2024-03-07T06:51:19.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.861+0000 7f87b3800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ac072f60 con 0x7f87ac0cdee0 2024-03-07T06:51:19.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.861+0000 7f87b3800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac073490 con 0x7f87ac072950 2024-03-07T06:51:19.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.861+0000 7f87b1800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f87ac0cdee0 0x7f87ac15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:19.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.861+0000 7f87b1800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f87ac0cdee0 0x7f87ac15c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46372/0 (socket says 172.21.15.133:46372) 2024-03-07T06:51:19.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.861+0000 7f87b1800640 1 -- 172.21.15.133:0/3034823281 learned_addr learned my addr 172.21.15.133:0/3034823281 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:19.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.861+0000 7f87abe00640 1 --2- 172.21.15.133:0/3034823281 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87ac072950 0x7f87ac0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:19.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.861+0000 7f87b1800640 1 -- 172.21.15.133:0/3034823281 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87ac072950 msgr2=0x7f87ac0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:19.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.861+0000 7f87b1800640 1 --2- 172.21.15.133:0/3034823281 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87ac072950 0x7f87ac0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.861+0000 7f87b1800640 1 -- 172.21.15.133:0/3034823281 >> v1:172.21.15.161:6789/0 conn(0x7f87ac15fbb0 legacy=0x7f87ac15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:19.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.862+0000 7f87b1800640 1 -- 172.21.15.133:0/3034823281 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac072150 con 0x7f87ac0cdee0 2024-03-07T06:51:19.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.863+0000 7f87b1800640 1 --2- 172.21.15.133:0/3034823281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87ac0cdee0 0x7f87ac15c550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0730e0 tx=0x7f87a402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823e53508aa897e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:19.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.863+0000 7f87ab400640 1 -- 172.21.15.133:0/3034823281 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a402ca50 con 0x7f87ac0cdee0 2024-03-07T06:51:19.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.863+0000 7f87ab400640 1 -- 172.21.15.133:0/3034823281 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a402cbf0 con 0x7f87ac0cdee0 2024-03-07T06:51:19.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.863+0000 7f87ab400640 1 -- 172.21.15.133:0/3034823281 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a40343f0 con 0x7f87ac0cdee0 2024-03-07T06:51:19.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.863+0000 7f87b3800640 1 -- 172.21.15.133:0/3034823281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87ac0cdee0 msgr2=0x7f87ac15c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:19.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.863+0000 7f87b3800640 1 --2- 172.21.15.133:0/3034823281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87ac0cdee0 0x7f87ac15c550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0730e0 tx=0x7f87a402beb0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.863+0000 7f87b3800640 1 -- 172.21.15.133:0/3034823281 shutdown_connections 2024-03-07T06:51:19.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.863+0000 7f87b3800640 1 --2- 172.21.15.133:0/3034823281 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87ac0cdee0 0x7f87ac15c550 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.863+0000 7f87b3800640 1 --2- 172.21.15.133:0/3034823281 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87ac072950 0x7f87ac0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.864+0000 7f87b3800640 1 -- 172.21.15.133:0/3034823281 >> 172.21.15.133:0/3034823281 conn(0x7f87ac0b3120 msgr2=0x7f87ac0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:19.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.864+0000 7f87b3800640 1 -- 172.21.15.133:0/3034823281 shutdown_connections 2024-03-07T06:51:19.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.864+0000 7f87b3800640 1 -- 172.21.15.133:0/3034823281 wait complete. 2024-03-07T06:51:19.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.865+0000 7f87b3800640 1 Processor -- start 2024-03-07T06:51:19.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.866+0000 7f87b3800640 1 -- start start 2024-03-07T06:51:19.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.866+0000 7f87b3800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87ac0cdee0 0x7f87ac14a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:19.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.866+0000 7f87b3800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87ac15fbb0 0x7f87ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:19.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.866+0000 7f87b3800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87ac06e830 0x7f87ac0b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:19.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87b3800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac160980 con 0x7f87ac06e830 2024-03-07T06:51:19.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87b3800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac0ce2f0 con 0x7f87ac0cdee0 2024-03-07T06:51:19.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87b3800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac0b1a00 con 0x7f87ac15fbb0 2024-03-07T06:51:19.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87b1800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87ac06e830 0x7f87ac0b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:19.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87b1800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87ac06e830 0x7f87ac0b57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46388/0 (socket says 172.21.15.133:46388) 2024-03-07T06:51:19.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87b1800640 1 -- 172.21.15.133:0/1098139238 learned_addr learned my addr 172.21.15.133:0/1098139238 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:19.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87abe00640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87ac15fbb0 0x7f87ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:19.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87b0e00640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87ac0cdee0 0x7f87ac14a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:19.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87b1800640 1 -- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87ac15fbb0 msgr2=0x7f87ac06e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:19.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87b1800640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87ac15fbb0 0x7f87ac06e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87b1800640 1 -- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87ac0cdee0 msgr2=0x7f87ac14a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:19.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87b1800640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87ac0cdee0 0x7f87ac14a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:19.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87b1800640 1 -- 172.21.15.133:0/1098139238 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac0b43d0 con 0x7f87ac06e830 2024-03-07T06:51:19.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87abe00640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87ac15fbb0 0x7f87ac06e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:51:19.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.867+0000 7f87b1800640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87ac06e830 0x7f87ac0b57d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f87a4036040 tx=0x7f87a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:19.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.868+0000 7f87a8c00640 1 -- 172.21.15.133:0/1098139238 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4043020 con 0x7f87ac06e830 2024-03-07T06:51:19.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.868+0000 7f87a8c00640 1 -- 172.21.15.133:0/1098139238 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a403e450 con 0x7f87ac06e830 2024-03-07T06:51:19.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.868+0000 7f87a8c00640 1 -- 172.21.15.133:0/1098139238 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4009510 con 0x7f87ac06e830 2024-03-07T06:51:19.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.868+0000 7f87b3800640 1 -- 172.21.15.133:0/1098139238 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ac0b46b0 con 0x7f87ac06e830 2024-03-07T06:51:19.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.868+0000 7f87b3800640 1 -- 172.21.15.133:0/1098139238 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ac0b4c70 con 0x7f87ac06e830 2024-03-07T06:51:19.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.869+0000 7f87b3800640 1 -- 172.21.15.133:0/1098139238 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ac062c90 con 0x7f87ac06e830 2024-03-07T06:51:19.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.870+0000 7f87a8c00640 1 -- 172.21.15.133:0/1098139238 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f87a4046020 con 0x7f87ac06e830 2024-03-07T06:51:19.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.871+0000 7f87a8c00640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f878c06b4d0 0x7f878c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:19.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.871+0000 7f87a8c00640 1 -- 172.21.15.133:0/1098139238 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87a40aca20 con 0x7f87ac06e830 2024-03-07T06:51:19.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.871+0000 7f87b0e00640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f878c06b4d0 0x7f878c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:19.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.872+0000 7f87b0e00640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f878c06b4d0 0x7f878c06d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f87a0000f60 tx=0x7f87a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:19.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:19.875+0000 7f87a8c00640 1 -- 172.21.15.133:0/1098139238 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f87a404d030 con 0x7f87ac06e830 2024-03-07T06:51:20.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.070+0000 7f87b3800640 1 -- 172.21.15.133:0/1098139238 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87ac0ae360 con 0x7f878c06b4d0 2024-03-07T06:51:20.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.087+0000 7f87a8c00640 1 -- 172.21.15.133:0/1098139238 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240071 (secure 0 0 0) 0x7f87ac0ae360 con 0x7f878c06b4d0 2024-03-07T06:51:20.090 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:20.091 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:51:20.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.093+0000 7f87b3800640 1 -- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f878c06b4d0 msgr2=0x7f878c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:20.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.093+0000 7f87b3800640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f878c06b4d0 0x7f878c06d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f87a0000f60 tx=0x7f87a0009040 comp rx=0 tx=0).stop 2024-03-07T06:51:20.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.093+0000 7f87b3800640 1 -- 172.21.15.133:0/1098139238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87ac06e830 msgr2=0x7f87ac0b57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.093+0000 7f87b3800640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87ac06e830 0x7f87ac0b57d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f87a4036040 tx=0x7f87a4004a40 comp rx=0 tx=0).stop 2024-03-07T06:51:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.093+0000 7f87b3800640 1 -- 172.21.15.133:0/1098139238 shutdown_connections 2024-03-07T06:51:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.093+0000 7f87b3800640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87ac06e830 0x7f87ac0b57d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.093+0000 7f87b3800640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87ac15fbb0 0x7f87ac06e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.093+0000 7f87b3800640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f878c06b4d0 0x7f878c06d990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.093+0000 7f87b3800640 1 --2- 172.21.15.133:0/1098139238 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87ac0cdee0 0x7f87ac14a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.093+0000 7f87b3800640 1 -- 172.21.15.133:0/1098139238 >> 172.21.15.133:0/1098139238 conn(0x7f87ac0b3120 msgr2=0x7f87ac074130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:20.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.093+0000 7f87b3800640 1 -- 172.21.15.133:0/1098139238 shutdown_connections 2024-03-07T06:51:20.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.093+0000 7f87b3800640 1 -- 172.21.15.133:0/1098139238 wait complete. 2024-03-07T06:51:20.113 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-03-07T06:51:18.470591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219556,"kb_used_data":2532,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755164,"statfs":{"total":773094113280,"available":772869287936,"internally_reserved":0,"allocated":2592768,"data_stored":1613325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":14,"num_write_kb":262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.344667"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.131807+0000","last_change":"2024-03-07T06:51:13.131807+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.131807+0000","last_undegraded":"2024-03-07T06:51:13.131807+0000","last_fullsized":"2024-03-07T06:51:13.131807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.135007+0000","last_change":"2024-03-07T06:51:13.135007+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.135007+0000","last_undegraded":"2024-03-07T06:51:13.135007+0000","last_fullsized":"2024-03-07T06:51:13.135007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.130942+0000","last_change":"2024-03-07T06:51:13.130942+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.130942+0000","last_undegraded":"2024-03-07T06:51:13.130942+0000","last_fullsized":"2024-03-07T06:51:13.130942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.152124+0000","last_change":"2024-03-07T06:51:13.152124+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.152124+0000","last_undegraded":"2024-03-07T06:51:13.152124+0000","last_fullsized":"2024-03-07T06:51:13.152124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.138886+0000","last_change":"2024-03-07T06:51:13.138886+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.138886+0000","last_undegraded":"2024-03-07T06:51:13.138886+0000","last_fullsized":"2024-03-07T06:51:13.138886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.151197+0000","last_change":"2024-03-07T06:51:13.151197+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.151197+0000","last_undegraded":"2024-03-07T06:51:13.151197+0000","last_fullsized":"2024-03-07T06:51:13.151197+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.147533+0000","last_change":"2024-03-07T06:51:13.147533+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.147533+0000","last_undegraded":"2024-03-07T06:51:13.147533+0000","last_fullsized":"2024-03-07T06:51:13.147533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.135577+0000","last_change":"2024-03-07T06:51:13.135577+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.135577+0000","last_undegraded":"2024-03-07T06:51:13.135577+0000","last_fullsized":"2024-03-07T06:51:13.135577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.133983+0000","last_change":"2024-03-07T06:51:13.133983+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.133983+0000","last_undegraded":"2024-03-07T06:51:13.133983+0000","last_fullsized":"2024-03-07T06:51:13.133983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.153228+0000","last_change":"2024-03-07T06:51:13.153228+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.153228+0000","last_undegraded":"2024-03-07T06:51:13.153228+0000","last_fullsized":"2024-03-07T06:51:13.153228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.135108+0000","last_change":"2024-03-07T06:51:13.135108+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.135108+0000","last_undegraded":"2024-03-07T06:51:13.135108+0000","last_fullsized":"2024-03-07T06:51:13.135108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.154393+0000","last_change":"2024-03-07T06:51:13.154393+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.154393+0000","last_undegraded":"2024-03-07T06:51:13.154393+0000","last_fullsized":"2024-03-07T06:51:13.154393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.134885+0000","last_change":"2024-03-07T06:51:13.134885+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.134885+0000","last_undegraded":"2024-03-07T06:51:13.134885+0000","last_fullsized":"2024-03-07T06:51:13.134885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.154552+0000","last_change":"2024-03-07T06:51:13.154552+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.154552+0000","last_undegraded":"2024-03-07T06:51:13.154552+0000","last_fullsized":"2024-03-07T06:51:13.154552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.145614+0000","last_change":"2024-03-07T06:51:13.145614+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.145614+0000","last_undegraded":"2024-03-07T06:51:13.145614+0000","last_fullsized":"2024-03-07T06:51:13.145614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.154252+0000","last_change":"2024-03-07T06:51:13.154252+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.154252+0000","last_undegraded":"2024-03-07T06:51:13.154252+0000","last_fullsized":"2024-03-07T06:51:13.154252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-03-07T06:51:13.147631+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:51:13.147631+0000","last_peered":"2024-03-07T06:51:13.147631+0000","last_clean":"2024-03-07T06:51:13.147631+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:51:13.147631+0000","last_undegraded":"2024-03-07T06:51:13.147631+0000","last_fullsized":"2024-03-07T06:51:13.147631+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.151267+0000","last_change":"2024-03-07T06:51:13.151267+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.151267+0000","last_undegraded":"2024-03-07T06:51:13.151267+0000","last_fullsized":"2024-03-07T06:51:13.151267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.141256+0000","last_change":"2024-03-07T06:51:13.141256+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.141256+0000","last_undegraded":"2024-03-07T06:51:13.141256+0000","last_fullsized":"2024-03-07T06:51:13.141256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.155367+0000","last_change":"2024-03-07T06:51:13.155367+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.155367+0000","last_undegraded":"2024-03-07T06:51:13.155367+0000","last_fullsized":"2024-03-07T06:51:13.155367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.151378+0000","last_change":"2024-03-07T06:51:13.151378+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.151378+0000","last_undegraded":"2024-03-07T06:51:13.151378+0000","last_fullsized":"2024-03-07T06:51:13.151378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.144999+0000","last_change":"2024-03-07T06:51:13.144999+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.144999+0000","last_undegraded":"2024-03-07T06:51:13.144999+0000","last_fullsized":"2024-03-07T06:51:13.144999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.143900+0000","last_change":"2024-03-07T06:51:13.143900+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.143900+0000","last_undegraded":"2024-03-07T06:51:13.143900+0000","last_fullsized":"2024-03-07T06:51:13.143900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.145725+0000","last_change":"2024-03-07T06:51:13.145725+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.145725+0000","last_undegraded":"2024-03-07T06:51:13.145725+0000","last_fullsized":"2024-03-07T06:51:13.145725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.136804+0000","last_change":"2024-03-07T06:51:13.136804+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.136804+0000","last_undegraded":"2024-03-07T06:51:13.136804+0000","last_fullsized":"2024-03-07T06:51:13.136804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.142872+0000","last_change":"2024-03-07T06:51:13.142872+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.142872+0000","last_undegraded":"2024-03-07T06:51:13.142872+0000","last_fullsized":"2024-03-07T06:51:13.142872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.144981+0000","last_change":"2024-03-07T06:51:13.144981+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.144981+0000","last_undegraded":"2024-03-07T06:51:13.144981+0000","last_fullsized":"2024-03-07T06:51:13.144981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.145384+0000","last_change":"2024-03-07T06:51:13.145384+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.145384+0000","last_undegraded":"2024-03-07T06:51:13.145384+0000","last_fullsized":"2024-03-07T06:51:13.145384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.142224+0000","last_change":"2024-03-07T06:51:13.142224+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.142224+0000","last_undegraded":"2024-03-07T06:51:13.142224+0000","last_fullsized":"2024-03-07T06:51:13.142224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.141804+0000","last_change":"2024-03-07T06:51:13.141804+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.141804+0000","last_undegraded":"2024-03-07T06:51:13.141804+0000","last_fullsized":"2024-03-07T06:51:13.141804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.146901+0000","last_change":"2024-03-07T06:51:13.146901+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.146901+0000","last_undegraded":"2024-03-07T06:51:13.146901+0000","last_fullsized":"2024-03-07T06:51:13.146901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.130977+0000","last_change":"2024-03-07T06:51:13.130977+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.130977+0000","last_undegraded":"2024-03-07T06:51:13.130977+0000","last_fullsized":"2024-03-07T06:51:13.130977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:51:14.122791+0000","last_change":"2024-03-07T06:51:14.122791+0000","last_active":"2024-03-07T06:51:14.122791+0000","last_peered":"2024-03-07T06:51:14.122791+0000","last_clean":"2024-03-07T06:51:14.122791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:14.122791+0000","last_undegraded":"2024-03-07T06:51:14.122791+0000","last_fullsized":"2024-03-07T06:51:14.122791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2024-03-07T06:51:13.134194+0000","last_change":"2024-03-07T06:51:13.134194+0000","last_active":"2024-03-07T06:51:13.110616+0000","last_peered":"2024-03-07T06:51:13.110616+0000","last_clean":"2024-03-07T06:51:13.110616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:51:13.134194+0000","last_undegraded":"2024-03-07T06:51:13.134194+0000","last_fullsized":"2024-03-07T06:51:13.134194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28384,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343456,"statfs":{"total":96636764160,"available":96607698944,"internally_reserved":0,"allocated":761856,"data_stored":640806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":56255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":761856,"data_stored":640806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:51:20.120 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:51:20.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.365+0000 7f9368200640 1 Processor -- start 2024-03-07T06:51:20.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.365+0000 7f9368200640 1 -- start start 2024-03-07T06:51:20.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.367+0000 7f9368200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f936015b1b0 0x7f93600b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:20.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.367+0000 7f9368200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936006f140 con 0x7f93600ce5f0 2024-03-07T06:51:20.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.367+0000 7f9368200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936006f420 con 0x7f93600af950 2024-03-07T06:51:20.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.367+0000 7f9368200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9360071610 con 0x7f936015b1b0 2024-03-07T06:51:20.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.367+0000 7f9366200640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f93600ce5f0 0x7f9360071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:53240/0 (socket says 172.21.15.133:53240) 2024-03-07T06:51:20.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.367+0000 7f9366200640 1 -- 172.21.15.133:0/1561343105 learned_addr learned my addr 172.21.15.133:0/1561343105 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:20.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f9365800640 1 --2- 172.21.15.133:0/1561343105 >> v2:172.21.15.161:3300/0 conn(0x7f936015b1b0 0x7f93600b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:20.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 -- 172.21.15.133:0/1561343105 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345745765 0 0) 0x7f936006f140 con 0x7f93600ce5f0 2024-03-07T06:51:20.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 -- 172.21.15.133:0/1561343105 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9348003520 con 0x7f93600ce5f0 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 -- 172.21.15.133:0/1561343105 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2590203831 0 0) 0x7f9348003520 con 0x7f93600ce5f0 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 -- 172.21.15.133:0/1561343105 >> v2:172.21.15.161:3300/0 conn(0x7f936015b1b0 msgr2=0x7f93600b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 --2- 172.21.15.133:0/1561343105 >> v2:172.21.15.161:3300/0 conn(0x7f936015b1b0 0x7f93600b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 -- 172.21.15.133:0/1561343105 >> v1:172.21.15.161:6789/0 conn(0x7f93600af950 legacy=0x7f93600cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 -- 172.21.15.133:0/1561343105 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9360071860 con 0x7f93600ce5f0 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 -- 172.21.15.133:0/1561343105 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f93580019f0 con 0x7f93600ce5f0 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 -- 172.21.15.133:0/1561343105 >> v1:172.21.15.133:6789/0 conn(0x7f93600ce5f0 legacy=0x7f9360071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 --2- 172.21.15.133:0/1561343105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93480039b0 0x7f9348003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 --2- 172.21.15.133:0/1561343105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9348004800 0x7f9348004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 --2- 172.21.15.133:0/1561343105 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93480051c0 0x7f934800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 -- 172.21.15.133:0/1561343105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9360071610 con 0x7f9348004800 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 -- 172.21.15.133:0/1561343105 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934800c160 con 0x7f93480039b0 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.368+0000 7f935fe00640 1 -- 172.21.15.133:0/1561343105 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934800c360 con 0x7f93480051c0 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.369+0000 7f9364e00640 1 --2- 172.21.15.133:0/1561343105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9348004800 0x7f9348004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:20.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.369+0000 7f9365800640 1 --2- 172.21.15.133:0/1561343105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93480039b0 0x7f9348003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:20.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.369+0000 7f9366200640 1 --2- 172.21.15.133:0/1561343105 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93480051c0 0x7f934800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:20.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.369+0000 7f9364e00640 1 -- 172.21.15.133:0/1561343105 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93480051c0 msgr2=0x7f934800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:20.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.369+0000 7f9364e00640 1 --2- 172.21.15.133:0/1561343105 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93480051c0 0x7f934800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.369+0000 7f9364e00640 1 -- 172.21.15.133:0/1561343105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93480039b0 msgr2=0x7f9348003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:20.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.369+0000 7f9364e00640 1 --2- 172.21.15.133:0/1561343105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93480039b0 0x7f9348003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.369+0000 7f9364e00640 1 -- 172.21.15.133:0/1561343105 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9348003520 con 0x7f9348004800 2024-03-07T06:51:20.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.369+0000 7f9366200640 1 --2- 172.21.15.133:0/1561343105 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93480051c0 0x7f934800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:20.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.369+0000 7f9366200640 1 -- 172.21.15.133:0/1561343105 reap_dead start 2024-03-07T06:51:20.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.370+0000 7f9364e00640 1 --2- 172.21.15.133:0/1561343105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9348004800 0x7f9348004c80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f934c001750 tx=0x7f9348005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01f2270d7a28983 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:20.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.370+0000 7f935fe00640 1 -- 172.21.15.133:0/1561343105 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c001370 con 0x7f9348004800 2024-03-07T06:51:20.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.370+0000 7f935fe00640 1 -- 172.21.15.133:0/1561343105 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934c0052d0 con 0x7f9348004800 2024-03-07T06:51:20.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.370+0000 7f935fe00640 1 -- 172.21.15.133:0/1561343105 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c003830 con 0x7f9348004800 2024-03-07T06:51:20.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.371+0000 7f9368200640 1 -- 172.21.15.133:0/1561343105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9348004800 msgr2=0x7f9348004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:20.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.371+0000 7f9368200640 1 --2- 172.21.15.133:0/1561343105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9348004800 0x7f9348004c80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f934c001750 tx=0x7f9348005830 comp rx=0 tx=0).stop 2024-03-07T06:51:20.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.371+0000 7f9368200640 1 -- 172.21.15.133:0/1561343105 shutdown_connections 2024-03-07T06:51:20.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.371+0000 7f9368200640 1 --2- 172.21.15.133:0/1561343105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9348004800 0x7f9348004c80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.371+0000 7f9368200640 1 -- 172.21.15.133:0/1561343105 >> 172.21.15.133:0/1561343105 conn(0x7f93600b3120 msgr2=0x7f93600b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:20.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.371+0000 7f9368200640 1 -- 172.21.15.133:0/1561343105 shutdown_connections 2024-03-07T06:51:20.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.371+0000 7f9368200640 1 -- 172.21.15.133:0/1561343105 wait complete. 2024-03-07T06:51:20.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.373+0000 7f9368200640 1 Processor -- start 2024-03-07T06:51:20.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.373+0000 7f9368200640 1 -- start start 2024-03-07T06:51:20.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.374+0000 7f9368200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9348004800 0x7f936015c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:20.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.374+0000 7f9368200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f936015e180 0x7f93600b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:20.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.374+0000 7f9368200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93600ae680 0x7f93601efd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:20.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.374+0000 7f9368200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936015b710 con 0x7f936015e180 2024-03-07T06:51:20.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.374+0000 7f9368200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936015b590 con 0x7f93600ae680 2024-03-07T06:51:20.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.374+0000 7f9368200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93600b3c10 con 0x7f9348004800 2024-03-07T06:51:20.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.374+0000 7f9364e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f936015e180 0x7f93600b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:20.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.374+0000 7f9364e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f936015e180 0x7f93600b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46400/0 (socket says 172.21.15.133:46400) 2024-03-07T06:51:20.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.374+0000 7f9364e00640 1 -- 172.21.15.133:0/3004990946 learned_addr learned my addr 172.21.15.133:0/3004990946 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:20.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.375+0000 7f9365800640 1 --2- 172.21.15.133:0/3004990946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9348004800 0x7f936015c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:20.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.375+0000 7f9366200640 1 --2- 172.21.15.133:0/3004990946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93600ae680 0x7f93601efd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:20.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.375+0000 7f9364e00640 1 -- 172.21.15.133:0/3004990946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9348004800 msgr2=0x7f936015c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:20.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.375+0000 7f9364e00640 1 --2- 172.21.15.133:0/3004990946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9348004800 0x7f936015c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.375+0000 7f9364e00640 1 -- 172.21.15.133:0/3004990946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93600ae680 msgr2=0x7f93601efd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:20.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.375+0000 7f9364e00640 1 --2- 172.21.15.133:0/3004990946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93600ae680 0x7f93601efd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.375+0000 7f9364e00640 1 -- 172.21.15.133:0/3004990946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936015bae0 con 0x7f936015e180 2024-03-07T06:51:20.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.375+0000 7f9364e00640 1 --2- 172.21.15.133:0/3004990946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f936015e180 0x7f93600b06f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f934c000c00 tx=0x7f934c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:20.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.375+0000 7f935d600640 1 -- 172.21.15.133:0/3004990946 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c041020 con 0x7f936015e180 2024-03-07T06:51:20.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.375+0000 7f935d600640 1 -- 172.21.15.133:0/3004990946 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934c005480 con 0x7f936015e180 2024-03-07T06:51:20.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.375+0000 7f9368200640 1 -- 172.21.15.133:0/3004990946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936015bdc0 con 0x7f936015e180 2024-03-07T06:51:20.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.375+0000 7f935d600640 1 -- 172.21.15.133:0/3004990946 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c007a30 con 0x7f936015e180 2024-03-07T06:51:20.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.375+0000 7f9368200640 1 -- 172.21.15.133:0/3004990946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93600b0c30 con 0x7f936015e180 2024-03-07T06:51:20.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.377+0000 7f9368200640 1 -- 172.21.15.133:0/3004990946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9360062c90 con 0x7f936015e180 2024-03-07T06:51:20.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.377+0000 7f935d600640 1 -- 172.21.15.133:0/3004990946 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f934c044020 con 0x7f936015e180 2024-03-07T06:51:20.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.378+0000 7f935d600640 1 --2- 172.21.15.133:0/3004990946 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f934006b4d0 0x7f934006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:20.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.378+0000 7f935d600640 1 -- 172.21.15.133:0/3004990946 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f934c0abb90 con 0x7f936015e180 2024-03-07T06:51:20.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.379+0000 7f9365800640 1 --2- 172.21.15.133:0/3004990946 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f934006b4d0 0x7f934006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:20.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.380+0000 7f9365800640 1 --2- 172.21.15.133:0/3004990946 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f934006b4d0 0x7f934006d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f93540099d0 tx=0x7f9354008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:20.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.382+0000 7f935d600640 1 -- 172.21.15.133:0/3004990946 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f934c04b030 con 0x7f936015e180 2024-03-07T06:51:20.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.579+0000 7f9368200640 1 -- 172.21.15.133:0/3004990946 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f936015b890 con 0x7f934006b4d0 2024-03-07T06:51:20.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.608+0000 7f935d600640 1 -- 172.21.15.133:0/3004990946 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251947 (secure 0 0 0) 0x7f936015b890 con 0x7f934006b4d0 2024-03-07T06:51:20.610 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:20.612 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:51:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.614+0000 7f9368200640 1 -- 172.21.15.133:0/3004990946 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f934006b4d0 msgr2=0x7f934006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.614+0000 7f9368200640 1 --2- 172.21.15.133:0/3004990946 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f934006b4d0 0x7f934006d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f93540099d0 tx=0x7f9354008040 comp rx=0 tx=0).stop 2024-03-07T06:51:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.614+0000 7f9368200640 1 -- 172.21.15.133:0/3004990946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f936015e180 msgr2=0x7f93600b06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.614+0000 7f9368200640 1 --2- 172.21.15.133:0/3004990946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f936015e180 0x7f93600b06f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f934c000c00 tx=0x7f934c001120 comp rx=0 tx=0).stop 2024-03-07T06:51:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.614+0000 7f9368200640 1 -- 172.21.15.133:0/3004990946 shutdown_connections 2024-03-07T06:51:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.614+0000 7f9368200640 1 --2- 172.21.15.133:0/3004990946 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f934006b4d0 0x7f934006d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.614+0000 7f9368200640 1 --2- 172.21.15.133:0/3004990946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93600ae680 0x7f93601efd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.614+0000 7f9368200640 1 --2- 172.21.15.133:0/3004990946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f936015e180 0x7f93600b06f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.614+0000 7f9368200640 1 --2- 172.21.15.133:0/3004990946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9348004800 0x7f936015c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.614+0000 7f9368200640 1 -- 172.21.15.133:0/3004990946 >> 172.21.15.133:0/3004990946 conn(0x7f93600b3120 msgr2=0x7f93600b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.614+0000 7f9368200640 1 -- 172.21.15.133:0/3004990946 shutdown_connections 2024-03-07T06:51:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.614+0000 7f9368200640 1 -- 172.21.15.133:0/3004990946 wait complete. 2024-03-07T06:51:20.632 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-07T06:51:20.471309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219088,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755632,"statfs":{"total":773094113280,"available":772869767168,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.339070"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190674+0000","last_change":"2024-03-07T06:51:15.202253+0000","last_active":"2024-03-07T06:51:16.190674+0000","last_peered":"2024-03-07T06:51:16.190674+0000","last_clean":"2024-03-07T06:51:16.190674+0000","last_became_active":"2024-03-07T06:51:15.201848+0000","last_became_peered":"2024-03-07T06:51:15.201848+0000","last_unstale":"2024-03-07T06:51:16.190674+0000","last_undegraded":"2024-03-07T06:51:16.190674+0000","last_fullsized":"2024-03-07T06:51:16.190674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:41:54.645832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205980+0000","last_change":"2024-03-07T06:51:15.188301+0000","last_active":"2024-03-07T06:51:16.205980+0000","last_peered":"2024-03-07T06:51:16.205980+0000","last_clean":"2024-03-07T06:51:16.205980+0000","last_became_active":"2024-03-07T06:51:15.188021+0000","last_became_peered":"2024-03-07T06:51:15.188021+0000","last_unstale":"2024-03-07T06:51:16.205980+0000","last_undegraded":"2024-03-07T06:51:16.205980+0000","last_fullsized":"2024-03-07T06:51:16.205980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412687+0000","last_change":"2024-03-07T06:51:15.188296+0000","last_active":"2024-03-07T06:51:17.412687+0000","last_peered":"2024-03-07T06:51:17.412687+0000","last_clean":"2024-03-07T06:51:17.412687+0000","last_became_active":"2024-03-07T06:51:15.187954+0000","last_became_peered":"2024-03-07T06:51:15.187954+0000","last_unstale":"2024-03-07T06:51:17.412687+0000","last_undegraded":"2024-03-07T06:51:17.412687+0000","last_fullsized":"2024-03-07T06:51:17.412687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414882+0000","last_change":"2024-03-07T06:51:15.200894+0000","last_active":"2024-03-07T06:51:17.414882+0000","last_peered":"2024-03-07T06:51:17.414882+0000","last_clean":"2024-03-07T06:51:17.414882+0000","last_became_active":"2024-03-07T06:51:15.200617+0000","last_became_peered":"2024-03-07T06:51:15.200617+0000","last_unstale":"2024-03-07T06:51:17.414882+0000","last_undegraded":"2024-03-07T06:51:17.414882+0000","last_fullsized":"2024-03-07T06:51:17.414882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190864+0000","last_change":"2024-03-07T06:51:15.189383+0000","last_active":"2024-03-07T06:51:16.190864+0000","last_peered":"2024-03-07T06:51:16.190864+0000","last_clean":"2024-03-07T06:51:16.190864+0000","last_became_active":"2024-03-07T06:51:15.188192+0000","last_became_peered":"2024-03-07T06:51:15.188192+0000","last_unstale":"2024-03-07T06:51:16.190864+0000","last_undegraded":"2024-03-07T06:51:16.190864+0000","last_fullsized":"2024-03-07T06:51:16.190864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:07:16.800545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414731+0000","last_change":"2024-03-07T06:51:15.195882+0000","last_active":"2024-03-07T06:51:17.414731+0000","last_peered":"2024-03-07T06:51:17.414731+0000","last_clean":"2024-03-07T06:51:17.414731+0000","last_became_active":"2024-03-07T06:51:15.195572+0000","last_became_peered":"2024-03-07T06:51:15.195572+0000","last_unstale":"2024-03-07T06:51:17.414731+0000","last_undegraded":"2024-03-07T06:51:17.414731+0000","last_fullsized":"2024-03-07T06:51:17.414731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189735+0000","last_change":"2024-03-07T06:51:15.190861+0000","last_active":"2024-03-07T06:51:16.189735+0000","last_peered":"2024-03-07T06:51:16.189735+0000","last_clean":"2024-03-07T06:51:16.189735+0000","last_became_active":"2024-03-07T06:51:15.190546+0000","last_became_peered":"2024-03-07T06:51:15.190546+0000","last_unstale":"2024-03-07T06:51:16.189735+0000","last_undegraded":"2024-03-07T06:51:16.189735+0000","last_fullsized":"2024-03-07T06:51:16.189735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:44:23.301530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415109+0000","last_change":"2024-03-07T06:51:15.202655+0000","last_active":"2024-03-07T06:51:17.415109+0000","last_peered":"2024-03-07T06:51:17.415109+0000","last_clean":"2024-03-07T06:51:17.415109+0000","last_became_active":"2024-03-07T06:51:15.197943+0000","last_became_peered":"2024-03-07T06:51:15.197943+0000","last_unstale":"2024-03-07T06:51:17.415109+0000","last_undegraded":"2024-03-07T06:51:17.415109+0000","last_fullsized":"2024-03-07T06:51:17.415109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190628+0000","last_change":"2024-03-07T06:51:15.209333+0000","last_active":"2024-03-07T06:51:16.190628+0000","last_peered":"2024-03-07T06:51:16.190628+0000","last_clean":"2024-03-07T06:51:16.190628+0000","last_became_active":"2024-03-07T06:51:15.209034+0000","last_became_peered":"2024-03-07T06:51:15.209034+0000","last_unstale":"2024-03-07T06:51:16.190628+0000","last_undegraded":"2024-03-07T06:51:16.190628+0000","last_fullsized":"2024-03-07T06:51:16.190628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:31:37.292017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193770+0000","last_change":"2024-03-07T06:51:15.196320+0000","last_active":"2024-03-07T06:51:16.193770+0000","last_peered":"2024-03-07T06:51:16.193770+0000","last_clean":"2024-03-07T06:51:16.193770+0000","last_became_active":"2024-03-07T06:51:15.196029+0000","last_became_peered":"2024-03-07T06:51:15.196029+0000","last_unstale":"2024-03-07T06:51:16.193770+0000","last_undegraded":"2024-03-07T06:51:16.193770+0000","last_fullsized":"2024-03-07T06:51:16.193770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192627+0000","last_change":"2024-03-07T06:51:15.212042+0000","last_active":"2024-03-07T06:51:16.192627+0000","last_peered":"2024-03-07T06:51:16.192627+0000","last_clean":"2024-03-07T06:51:16.192627+0000","last_became_active":"2024-03-07T06:51:15.211737+0000","last_became_peered":"2024-03-07T06:51:15.211737+0000","last_unstale":"2024-03-07T06:51:16.192627+0000","last_undegraded":"2024-03-07T06:51:16.192627+0000","last_fullsized":"2024-03-07T06:51:16.192627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192585+0000","last_change":"2024-03-07T06:51:15.200183+0000","last_active":"2024-03-07T06:51:16.192585+0000","last_peered":"2024-03-07T06:51:16.192585+0000","last_clean":"2024-03-07T06:51:16.192585+0000","last_became_active":"2024-03-07T06:51:15.199322+0000","last_became_peered":"2024-03-07T06:51:15.199322+0000","last_unstale":"2024-03-07T06:51:16.192585+0000","last_undegraded":"2024-03-07T06:51:16.192585+0000","last_fullsized":"2024-03-07T06:51:16.192585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:11.295266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415528+0000","last_change":"2024-03-07T06:51:15.201141+0000","last_active":"2024-03-07T06:51:17.415528+0000","last_peered":"2024-03-07T06:51:17.415528+0000","last_clean":"2024-03-07T06:51:17.415528+0000","last_became_active":"2024-03-07T06:51:15.200965+0000","last_became_peered":"2024-03-07T06:51:15.200965+0000","last_unstale":"2024-03-07T06:51:17.415528+0000","last_undegraded":"2024-03-07T06:51:17.415528+0000","last_fullsized":"2024-03-07T06:51:17.415528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412986+0000","last_change":"2024-03-07T06:51:15.191043+0000","last_active":"2024-03-07T06:51:17.412986+0000","last_peered":"2024-03-07T06:51:17.412986+0000","last_clean":"2024-03-07T06:51:17.412986+0000","last_became_active":"2024-03-07T06:51:15.190777+0000","last_became_peered":"2024-03-07T06:51:15.190777+0000","last_unstale":"2024-03-07T06:51:17.412986+0000","last_undegraded":"2024-03-07T06:51:17.412986+0000","last_fullsized":"2024-03-07T06:51:17.412986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205186+0000","last_change":"2024-03-07T06:51:15.178908+0000","last_active":"2024-03-07T06:51:16.205186+0000","last_peered":"2024-03-07T06:51:16.205186+0000","last_clean":"2024-03-07T06:51:16.205186+0000","last_became_active":"2024-03-07T06:51:15.178387+0000","last_became_peered":"2024-03-07T06:51:15.178387+0000","last_unstale":"2024-03-07T06:51:16.205186+0000","last_undegraded":"2024-03-07T06:51:16.205186+0000","last_fullsized":"2024-03-07T06:51:16.205186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414392+0000","last_change":"2024-03-07T06:51:15.195007+0000","last_active":"2024-03-07T06:51:17.414392+0000","last_peered":"2024-03-07T06:51:17.414392+0000","last_clean":"2024-03-07T06:51:17.414392+0000","last_became_active":"2024-03-07T06:51:15.194624+0000","last_became_peered":"2024-03-07T06:51:15.194624+0000","last_unstale":"2024-03-07T06:51:17.414392+0000","last_undegraded":"2024-03-07T06:51:17.414392+0000","last_fullsized":"2024-03-07T06:51:17.414392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.416108+0000","last_change":"2024-03-07T06:51:15.206405+0000","last_active":"2024-03-07T06:51:17.416108+0000","last_peered":"2024-03-07T06:51:17.416108+0000","last_clean":"2024-03-07T06:51:17.416108+0000","last_became_active":"2024-03-07T06:51:15.205493+0000","last_became_peered":"2024-03-07T06:51:15.205493+0000","last_unstale":"2024-03-07T06:51:17.416108+0000","last_undegraded":"2024-03-07T06:51:17.416108+0000","last_fullsized":"2024-03-07T06:51:17.416108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:43:58.927044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413141+0000","last_change":"2024-03-07T06:51:15.195307+0000","last_active":"2024-03-07T06:51:17.413141+0000","last_peered":"2024-03-07T06:51:17.413141+0000","last_clean":"2024-03-07T06:51:17.413141+0000","last_became_active":"2024-03-07T06:51:15.194990+0000","last_became_peered":"2024-03-07T06:51:15.194990+0000","last_unstale":"2024-03-07T06:51:17.413141+0000","last_undegraded":"2024-03-07T06:51:17.413141+0000","last_fullsized":"2024-03-07T06:51:17.413141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:11.295266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189287+0000","last_change":"2024-03-07T06:51:15.197914+0000","last_active":"2024-03-07T06:51:16.189287+0000","last_peered":"2024-03-07T06:51:16.189287+0000","last_clean":"2024-03-07T06:51:16.189287+0000","last_became_active":"2024-03-07T06:51:15.196938+0000","last_became_peered":"2024-03-07T06:51:15.196938+0000","last_unstale":"2024-03-07T06:51:16.189287+0000","last_undegraded":"2024-03-07T06:51:16.189287+0000","last_fullsized":"2024-03-07T06:51:16.189287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:56.465649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192765+0000","last_change":"2024-03-07T06:51:15.199775+0000","last_active":"2024-03-07T06:51:16.192765+0000","last_peered":"2024-03-07T06:51:16.192765+0000","last_clean":"2024-03-07T06:51:16.192765+0000","last_became_active":"2024-03-07T06:51:15.199447+0000","last_became_peered":"2024-03-07T06:51:15.199447+0000","last_unstale":"2024-03-07T06:51:16.192765+0000","last_undegraded":"2024-03-07T06:51:16.192765+0000","last_fullsized":"2024-03-07T06:51:16.192765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413641+0000","last_change":"2024-03-07T06:51:15.191968+0000","last_active":"2024-03-07T06:51:17.413641+0000","last_peered":"2024-03-07T06:51:17.413641+0000","last_clean":"2024-03-07T06:51:17.413641+0000","last_became_active":"2024-03-07T06:51:15.191587+0000","last_became_peered":"2024-03-07T06:51:15.191587+0000","last_unstale":"2024-03-07T06:51:17.413641+0000","last_undegraded":"2024-03-07T06:51:17.413641+0000","last_fullsized":"2024-03-07T06:51:17.413641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415294+0000","last_change":"2024-03-07T06:51:15.206834+0000","last_active":"2024-03-07T06:51:17.415294+0000","last_peered":"2024-03-07T06:51:17.415294+0000","last_clean":"2024-03-07T06:51:17.415294+0000","last_became_active":"2024-03-07T06:51:15.198815+0000","last_became_peered":"2024-03-07T06:51:15.198815+0000","last_unstale":"2024-03-07T06:51:17.415294+0000","last_undegraded":"2024-03-07T06:51:17.415294+0000","last_fullsized":"2024-03-07T06:51:17.415294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415387+0000","last_change":"2024-03-07T06:51:15.203302+0000","last_active":"2024-03-07T06:51:17.415387+0000","last_peered":"2024-03-07T06:51:17.415387+0000","last_clean":"2024-03-07T06:51:17.415387+0000","last_became_active":"2024-03-07T06:51:15.203041+0000","last_became_peered":"2024-03-07T06:51:15.203041+0000","last_unstale":"2024-03-07T06:51:17.415387+0000","last_undegraded":"2024-03-07T06:51:17.415387+0000","last_fullsized":"2024-03-07T06:51:17.415387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414478+0000","last_change":"2024-03-07T06:51:15.195865+0000","last_active":"2024-03-07T06:51:17.414478+0000","last_peered":"2024-03-07T06:51:17.414478+0000","last_clean":"2024-03-07T06:51:17.414478+0000","last_became_active":"2024-03-07T06:51:15.194043+0000","last_became_peered":"2024-03-07T06:51:15.194043+0000","last_unstale":"2024-03-07T06:51:17.414478+0000","last_undegraded":"2024-03-07T06:51:17.414478+0000","last_fullsized":"2024-03-07T06:51:17.414478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415922+0000","last_change":"2024-03-07T06:51:15.203857+0000","last_active":"2024-03-07T06:51:17.415922+0000","last_peered":"2024-03-07T06:51:17.415922+0000","last_clean":"2024-03-07T06:51:17.415922+0000","last_became_active":"2024-03-07T06:51:15.203562+0000","last_became_peered":"2024-03-07T06:51:15.203562+0000","last_unstale":"2024-03-07T06:51:17.415922+0000","last_undegraded":"2024-03-07T06:51:17.415922+0000","last_fullsized":"2024-03-07T06:51:17.415922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:34:40.365080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193687+0000","last_change":"2024-03-07T06:51:15.210400+0000","last_active":"2024-03-07T06:51:16.193687+0000","last_peered":"2024-03-07T06:51:16.193687+0000","last_clean":"2024-03-07T06:51:16.193687+0000","last_became_active":"2024-03-07T06:51:15.209692+0000","last_became_peered":"2024-03-07T06:51:15.209692+0000","last_unstale":"2024-03-07T06:51:16.193687+0000","last_undegraded":"2024-03-07T06:51:16.193687+0000","last_fullsized":"2024-03-07T06:51:16.193687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205792+0000","last_change":"2024-03-07T06:51:15.190106+0000","last_active":"2024-03-07T06:51:16.205792+0000","last_peered":"2024-03-07T06:51:16.205792+0000","last_clean":"2024-03-07T06:51:16.205792+0000","last_became_active":"2024-03-07T06:51:15.189498+0000","last_became_peered":"2024-03-07T06:51:15.189498+0000","last_unstale":"2024-03-07T06:51:16.205792+0000","last_undegraded":"2024-03-07T06:51:16.205792+0000","last_fullsized":"2024-03-07T06:51:16.205792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415671+0000","last_change":"2024-03-07T06:51:15.203374+0000","last_active":"2024-03-07T06:51:17.415671+0000","last_peered":"2024-03-07T06:51:17.415671+0000","last_clean":"2024-03-07T06:51:17.415671+0000","last_became_active":"2024-03-07T06:51:15.202849+0000","last_became_peered":"2024-03-07T06:51:15.202849+0000","last_unstale":"2024-03-07T06:51:17.415671+0000","last_undegraded":"2024-03-07T06:51:17.415671+0000","last_fullsized":"2024-03-07T06:51:17.415671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413542+0000","last_change":"2024-03-07T06:51:15.200244+0000","last_active":"2024-03-07T06:51:17.413542+0000","last_peered":"2024-03-07T06:51:17.413542+0000","last_clean":"2024-03-07T06:51:17.413542+0000","last_became_active":"2024-03-07T06:51:15.198483+0000","last_became_peered":"2024-03-07T06:51:15.198483+0000","last_unstale":"2024-03-07T06:51:17.413542+0000","last_undegraded":"2024-03-07T06:51:17.413542+0000","last_fullsized":"2024-03-07T06:51:17.413542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415577+0000","last_change":"2024-03-07T06:51:15.195170+0000","last_active":"2024-03-07T06:51:17.415577+0000","last_peered":"2024-03-07T06:51:17.415577+0000","last_clean":"2024-03-07T06:51:17.415577+0000","last_became_active":"2024-03-07T06:51:15.194900+0000","last_became_peered":"2024-03-07T06:51:15.194900+0000","last_unstale":"2024-03-07T06:51:17.415577+0000","last_undegraded":"2024-03-07T06:51:17.415577+0000","last_fullsized":"2024-03-07T06:51:17.415577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:44:23.301530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414305+0000","last_change":"2024-03-07T06:51:15.197411+0000","last_active":"2024-03-07T06:51:17.414305+0000","last_peered":"2024-03-07T06:51:17.414305+0000","last_clean":"2024-03-07T06:51:17.414305+0000","last_became_active":"2024-03-07T06:51:15.194699+0000","last_became_peered":"2024-03-07T06:51:15.194699+0000","last_unstale":"2024-03-07T06:51:17.414305+0000","last_undegraded":"2024-03-07T06:51:17.414305+0000","last_fullsized":"2024-03-07T06:51:17.414305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:43.574428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.195220+0000","last_change":"2024-03-07T06:51:15.201331+0000","last_active":"2024-03-07T06:51:16.195220+0000","last_peered":"2024-03-07T06:51:16.195220+0000","last_clean":"2024-03-07T06:51:16.195220+0000","last_became_active":"2024-03-07T06:51:15.200758+0000","last_became_peered":"2024-03-07T06:51:15.200758+0000","last_unstale":"2024-03-07T06:51:16.195220+0000","last_undegraded":"2024-03-07T06:51:16.195220+0000","last_fullsized":"2024-03-07T06:51:16.195220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192348+0000","last_change":"2024-03-07T06:51:15.180977+0000","last_active":"2024-03-07T06:51:16.192348+0000","last_peered":"2024-03-07T06:51:16.192348+0000","last_clean":"2024-03-07T06:51:16.192348+0000","last_became_active":"2024-03-07T06:51:15.180359+0000","last_became_peered":"2024-03-07T06:51:15.180359+0000","last_unstale":"2024-03-07T06:51:16.192348+0000","last_undegraded":"2024-03-07T06:51:16.192348+0000","last_fullsized":"2024-03-07T06:51:16.192348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:10.776380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415597+0000","last_change":"2024-03-07T06:51:14.171537+0000","last_active":"2024-03-07T06:51:17.415597+0000","last_peered":"2024-03-07T06:51:17.415597+0000","last_clean":"2024-03-07T06:51:17.415597+0000","last_became_active":"2024-03-07T06:51:14.171105+0000","last_became_peered":"2024-03-07T06:51:14.171105+0000","last_unstale":"2024-03-07T06:51:17.415597+0000","last_undegraded":"2024-03-07T06:51:17.415597+0000","last_fullsized":"2024-03-07T06:51:17.415597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413743+0000","last_change":"2024-03-07T06:51:15.198729+0000","last_active":"2024-03-07T06:51:17.413743+0000","last_peered":"2024-03-07T06:51:17.413743+0000","last_clean":"2024-03-07T06:51:17.413743+0000","last_became_active":"2024-03-07T06:51:15.198412+0000","last_became_peered":"2024-03-07T06:51:15.198412+0000","last_unstale":"2024-03-07T06:51:17.413743+0000","last_undegraded":"2024-03-07T06:51:17.413743+0000","last_fullsized":"2024-03-07T06:51:17.413743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192432+0000","last_change":"2024-03-07T06:51:14.172967+0000","last_active":"2024-03-07T06:51:16.192432+0000","last_peered":"2024-03-07T06:51:16.192432+0000","last_clean":"2024-03-07T06:51:16.192432+0000","last_became_active":"2024-03-07T06:51:14.172706+0000","last_became_peered":"2024-03-07T06:51:14.172706+0000","last_unstale":"2024-03-07T06:51:16.192432+0000","last_undegraded":"2024-03-07T06:51:16.192432+0000","last_fullsized":"2024-03-07T06:51:16.192432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415155+0000","last_change":"2024-03-07T06:51:14.167651+0000","last_active":"2024-03-07T06:51:17.415155+0000","last_peered":"2024-03-07T06:51:17.415155+0000","last_clean":"2024-03-07T06:51:17.415155+0000","last_became_active":"2024-03-07T06:51:14.166984+0000","last_became_peered":"2024-03-07T06:51:14.166984+0000","last_unstale":"2024-03-07T06:51:17.415155+0000","last_undegraded":"2024-03-07T06:51:17.415155+0000","last_fullsized":"2024-03-07T06:51:17.415155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415067+0000","last_change":"2024-03-07T06:51:15.202906+0000","last_active":"2024-03-07T06:51:17.415067+0000","last_peered":"2024-03-07T06:51:17.415067+0000","last_clean":"2024-03-07T06:51:17.415067+0000","last_became_active":"2024-03-07T06:51:15.200219+0000","last_became_peered":"2024-03-07T06:51:15.200219+0000","last_unstale":"2024-03-07T06:51:17.415067+0000","last_undegraded":"2024-03-07T06:51:17.415067+0000","last_fullsized":"2024-03-07T06:51:17.415067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414528+0000","last_change":"2024-03-07T06:51:14.175653+0000","last_active":"2024-03-07T06:51:17.414528+0000","last_peered":"2024-03-07T06:51:17.414528+0000","last_clean":"2024-03-07T06:51:17.414528+0000","last_became_active":"2024-03-07T06:51:14.175295+0000","last_became_peered":"2024-03-07T06:51:14.175295+0000","last_unstale":"2024-03-07T06:51:17.414528+0000","last_undegraded":"2024-03-07T06:51:17.414528+0000","last_fullsized":"2024-03-07T06:51:17.414528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205033+0000","last_change":"2024-03-07T06:51:15.190580+0000","last_active":"2024-03-07T06:51:16.205033+0000","last_peered":"2024-03-07T06:51:16.205033+0000","last_clean":"2024-03-07T06:51:16.205033+0000","last_became_active":"2024-03-07T06:51:15.189190+0000","last_became_peered":"2024-03-07T06:51:15.189190+0000","last_unstale":"2024-03-07T06:51:16.205033+0000","last_undegraded":"2024-03-07T06:51:16.205033+0000","last_fullsized":"2024-03-07T06:51:16.205033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193565+0000","last_change":"2024-03-07T06:51:14.173746+0000","last_active":"2024-03-07T06:51:16.193565+0000","last_peered":"2024-03-07T06:51:16.193565+0000","last_clean":"2024-03-07T06:51:16.193565+0000","last_became_active":"2024-03-07T06:51:14.173285+0000","last_became_peered":"2024-03-07T06:51:14.173285+0000","last_unstale":"2024-03-07T06:51:16.193565+0000","last_undegraded":"2024-03-07T06:51:16.193565+0000","last_fullsized":"2024-03-07T06:51:16.193565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415198+0000","last_change":"2024-03-07T06:51:15.215525+0000","last_active":"2024-03-07T06:51:17.415198+0000","last_peered":"2024-03-07T06:51:17.415198+0000","last_clean":"2024-03-07T06:51:17.415198+0000","last_became_active":"2024-03-07T06:51:15.215050+0000","last_became_peered":"2024-03-07T06:51:15.215050+0000","last_unstale":"2024-03-07T06:51:17.415198+0000","last_undegraded":"2024-03-07T06:51:17.415198+0000","last_fullsized":"2024-03-07T06:51:17.415198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:22:38.973224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414959+0000","last_change":"2024-03-07T06:51:14.168633+0000","last_active":"2024-03-07T06:51:17.414959+0000","last_peered":"2024-03-07T06:51:17.414959+0000","last_clean":"2024-03-07T06:51:17.414959+0000","last_became_active":"2024-03-07T06:51:14.168293+0000","last_became_peered":"2024-03-07T06:51:14.168293+0000","last_unstale":"2024-03-07T06:51:17.414959+0000","last_undegraded":"2024-03-07T06:51:17.414959+0000","last_fullsized":"2024-03-07T06:51:17.414959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412761+0000","last_change":"2024-03-07T06:51:15.208732+0000","last_active":"2024-03-07T06:51:17.412761+0000","last_peered":"2024-03-07T06:51:17.412761+0000","last_clean":"2024-03-07T06:51:17.412761+0000","last_became_active":"2024-03-07T06:51:15.208472+0000","last_became_peered":"2024-03-07T06:51:15.208472+0000","last_unstale":"2024-03-07T06:51:17.412761+0000","last_undegraded":"2024-03-07T06:51:17.412761+0000","last_fullsized":"2024-03-07T06:51:17.412761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205600+0000","last_change":"2024-03-07T06:51:14.171631+0000","last_active":"2024-03-07T06:51:16.205600+0000","last_peered":"2024-03-07T06:51:16.205600+0000","last_clean":"2024-03-07T06:51:16.205600+0000","last_became_active":"2024-03-07T06:51:14.171058+0000","last_became_peered":"2024-03-07T06:51:14.171058+0000","last_unstale":"2024-03-07T06:51:16.205600+0000","last_undegraded":"2024-03-07T06:51:16.205600+0000","last_fullsized":"2024-03-07T06:51:16.205600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193446+0000","last_change":"2024-03-07T06:51:15.195194+0000","last_active":"2024-03-07T06:51:16.193446+0000","last_peered":"2024-03-07T06:51:16.193446+0000","last_clean":"2024-03-07T06:51:16.193446+0000","last_became_active":"2024-03-07T06:51:15.191103+0000","last_became_peered":"2024-03-07T06:51:15.191103+0000","last_unstale":"2024-03-07T06:51:16.193446+0000","last_undegraded":"2024-03-07T06:51:16.193446+0000","last_fullsized":"2024-03-07T06:51:16.193446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:40:51.935140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192876+0000","last_change":"2024-03-07T06:51:14.166640+0000","last_active":"2024-03-07T06:51:16.192876+0000","last_peered":"2024-03-07T06:51:16.192876+0000","last_clean":"2024-03-07T06:51:16.192876+0000","last_became_active":"2024-03-07T06:51:14.166329+0000","last_became_peered":"2024-03-07T06:51:14.166329+0000","last_unstale":"2024-03-07T06:51:16.192876+0000","last_undegraded":"2024-03-07T06:51:16.192876+0000","last_fullsized":"2024-03-07T06:51:16.192876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188809+0000","last_change":"2024-03-07T06:51:15.193638+0000","last_active":"2024-03-07T06:51:16.188809+0000","last_peered":"2024-03-07T06:51:16.188809+0000","last_clean":"2024-03-07T06:51:16.188809+0000","last_became_active":"2024-03-07T06:51:15.193354+0000","last_became_peered":"2024-03-07T06:51:15.193354+0000","last_unstale":"2024-03-07T06:51:16.188809+0000","last_undegraded":"2024-03-07T06:51:16.188809+0000","last_fullsized":"2024-03-07T06:51:16.188809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413807+0000","last_change":"2024-03-07T06:51:14.167158+0000","last_active":"2024-03-07T06:51:17.413807+0000","last_peered":"2024-03-07T06:51:17.413807+0000","last_clean":"2024-03-07T06:51:17.413807+0000","last_became_active":"2024-03-07T06:51:14.166835+0000","last_became_peered":"2024-03-07T06:51:14.166835+0000","last_unstale":"2024-03-07T06:51:17.413807+0000","last_undegraded":"2024-03-07T06:51:17.413807+0000","last_fullsized":"2024-03-07T06:51:17.413807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:38:27.101103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413891+0000","last_change":"2024-03-07T06:51:15.199375+0000","last_active":"2024-03-07T06:51:17.413891+0000","last_peered":"2024-03-07T06:51:17.413891+0000","last_clean":"2024-03-07T06:51:17.413891+0000","last_became_active":"2024-03-07T06:51:15.199060+0000","last_became_peered":"2024-03-07T06:51:15.199060+0000","last_unstale":"2024-03-07T06:51:17.413891+0000","last_undegraded":"2024-03-07T06:51:17.413891+0000","last_fullsized":"2024-03-07T06:51:17.413891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415423+0000","last_change":"2024-03-07T06:51:14.170514+0000","last_active":"2024-03-07T06:51:17.415423+0000","last_peered":"2024-03-07T06:51:17.415423+0000","last_clean":"2024-03-07T06:51:17.415423+0000","last_became_active":"2024-03-07T06:51:14.169542+0000","last_became_peered":"2024-03-07T06:51:14.169542+0000","last_unstale":"2024-03-07T06:51:17.415423+0000","last_undegraded":"2024-03-07T06:51:17.415423+0000","last_fullsized":"2024-03-07T06:51:17.415423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191234+0000","last_change":"2024-03-07T06:51:15.208081+0000","last_active":"2024-03-07T06:51:16.191234+0000","last_peered":"2024-03-07T06:51:16.191234+0000","last_clean":"2024-03-07T06:51:16.191234+0000","last_became_active":"2024-03-07T06:51:15.207758+0000","last_became_peered":"2024-03-07T06:51:15.207758+0000","last_unstale":"2024-03-07T06:51:16.191234+0000","last_undegraded":"2024-03-07T06:51:16.191234+0000","last_fullsized":"2024-03-07T06:51:16.191234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:43:16.081728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191798+0000","last_change":"2024-03-07T06:51:14.168470+0000","last_active":"2024-03-07T06:51:16.191798+0000","last_peered":"2024-03-07T06:51:16.191798+0000","last_clean":"2024-03-07T06:51:16.191798+0000","last_became_active":"2024-03-07T06:51:14.168190+0000","last_became_peered":"2024-03-07T06:51:14.168190+0000","last_unstale":"2024-03-07T06:51:16.191798+0000","last_undegraded":"2024-03-07T06:51:16.191798+0000","last_fullsized":"2024-03-07T06:51:16.191798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192778+0000","last_change":"2024-03-07T06:51:15.201105+0000","last_active":"2024-03-07T06:51:16.192778+0000","last_peered":"2024-03-07T06:51:16.192778+0000","last_clean":"2024-03-07T06:51:16.192778+0000","last_became_active":"2024-03-07T06:51:15.200404+0000","last_became_peered":"2024-03-07T06:51:15.200404+0000","last_unstale":"2024-03-07T06:51:16.192778+0000","last_undegraded":"2024-03-07T06:51:16.192778+0000","last_fullsized":"2024-03-07T06:51:16.192778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412017+0000","last_change":"2024-03-07T06:51:14.154100+0000","last_active":"2024-03-07T06:51:17.412017+0000","last_peered":"2024-03-07T06:51:17.412017+0000","last_clean":"2024-03-07T06:51:17.412017+0000","last_became_active":"2024-03-07T06:51:14.152648+0000","last_became_peered":"2024-03-07T06:51:14.152648+0000","last_unstale":"2024-03-07T06:51:17.412017+0000","last_undegraded":"2024-03-07T06:51:17.412017+0000","last_fullsized":"2024-03-07T06:51:17.412017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189886+0000","last_change":"2024-03-07T06:51:15.203942+0000","last_active":"2024-03-07T06:51:16.189886+0000","last_peered":"2024-03-07T06:51:16.189886+0000","last_clean":"2024-03-07T06:51:16.189886+0000","last_became_active":"2024-03-07T06:51:15.203671+0000","last_became_peered":"2024-03-07T06:51:15.203671+0000","last_unstale":"2024-03-07T06:51:16.189886+0000","last_undegraded":"2024-03-07T06:51:16.189886+0000","last_fullsized":"2024-03-07T06:51:16.189886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:02:55.966834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193630+0000","last_change":"2024-03-07T06:51:14.164166+0000","last_active":"2024-03-07T06:51:16.193630+0000","last_peered":"2024-03-07T06:51:16.193630+0000","last_clean":"2024-03-07T06:51:16.193630+0000","last_became_active":"2024-03-07T06:51:14.163821+0000","last_became_peered":"2024-03-07T06:51:14.163821+0000","last_unstale":"2024-03-07T06:51:16.193630+0000","last_undegraded":"2024-03-07T06:51:16.193630+0000","last_fullsized":"2024-03-07T06:51:16.193630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:38:27.101103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190408+0000","last_change":"2024-03-07T06:51:15.207392+0000","last_active":"2024-03-07T06:51:16.190408+0000","last_peered":"2024-03-07T06:51:16.190408+0000","last_clean":"2024-03-07T06:51:16.190408+0000","last_became_active":"2024-03-07T06:51:15.203049+0000","last_became_peered":"2024-03-07T06:51:15.203049+0000","last_unstale":"2024-03-07T06:51:16.190408+0000","last_undegraded":"2024-03-07T06:51:16.190408+0000","last_fullsized":"2024-03-07T06:51:16.190408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:43:58.927044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413191+0000","last_change":"2024-03-07T06:51:14.162546+0000","last_active":"2024-03-07T06:51:17.413191+0000","last_peered":"2024-03-07T06:51:17.413191+0000","last_clean":"2024-03-07T06:51:17.413191+0000","last_became_active":"2024-03-07T06:51:14.161082+0000","last_became_peered":"2024-03-07T06:51:14.161082+0000","last_unstale":"2024-03-07T06:51:17.413191+0000","last_undegraded":"2024-03-07T06:51:17.413191+0000","last_fullsized":"2024-03-07T06:51:17.413191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190725+0000","last_change":"2024-03-07T06:51:15.208036+0000","last_active":"2024-03-07T06:51:16.190725+0000","last_peered":"2024-03-07T06:51:16.190725+0000","last_clean":"2024-03-07T06:51:16.190725+0000","last_became_active":"2024-03-07T06:51:15.206837+0000","last_became_peered":"2024-03-07T06:51:15.206837+0000","last_unstale":"2024-03-07T06:51:16.190725+0000","last_undegraded":"2024-03-07T06:51:16.190725+0000","last_fullsized":"2024-03-07T06:51:16.190725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:55:33.816984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205890+0000","last_change":"2024-03-07T06:51:14.168546+0000","last_active":"2024-03-07T06:51:16.205890+0000","last_peered":"2024-03-07T06:51:16.205890+0000","last_clean":"2024-03-07T06:51:16.205890+0000","last_became_active":"2024-03-07T06:51:14.168238+0000","last_became_peered":"2024-03-07T06:51:14.168238+0000","last_unstale":"2024-03-07T06:51:16.205890+0000","last_undegraded":"2024-03-07T06:51:16.205890+0000","last_fullsized":"2024-03-07T06:51:16.205890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415347+0000","last_change":"2024-03-07T06:51:15.204629+0000","last_active":"2024-03-07T06:51:17.415347+0000","last_peered":"2024-03-07T06:51:17.415347+0000","last_clean":"2024-03-07T06:51:17.415347+0000","last_became_active":"2024-03-07T06:51:15.203615+0000","last_became_peered":"2024-03-07T06:51:15.203615+0000","last_unstale":"2024-03-07T06:51:17.415347+0000","last_undegraded":"2024-03-07T06:51:17.415347+0000","last_fullsized":"2024-03-07T06:51:17.415347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:38:28.416311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412767+0000","last_change":"2024-03-07T06:51:14.170179+0000","last_active":"2024-03-07T06:51:17.412767+0000","last_peered":"2024-03-07T06:51:17.412767+0000","last_clean":"2024-03-07T06:51:17.412767+0000","last_became_active":"2024-03-07T06:51:14.169797+0000","last_became_peered":"2024-03-07T06:51:14.169797+0000","last_unstale":"2024-03-07T06:51:17.412767+0000","last_undegraded":"2024-03-07T06:51:17.412767+0000","last_fullsized":"2024-03-07T06:51:17.412767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188955+0000","last_change":"2024-03-07T06:51:15.212961+0000","last_active":"2024-03-07T06:51:16.188955+0000","last_peered":"2024-03-07T06:51:16.188955+0000","last_clean":"2024-03-07T06:51:16.188955+0000","last_became_active":"2024-03-07T06:51:15.212691+0000","last_became_peered":"2024-03-07T06:51:15.212691+0000","last_unstale":"2024-03-07T06:51:16.188955+0000","last_undegraded":"2024-03-07T06:51:16.188955+0000","last_fullsized":"2024-03-07T06:51:16.188955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:38:28.416311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188951+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:51:16.188951+0000","last_peered":"2024-03-07T06:51:16.188951+0000","last_clean":"2024-03-07T06:51:16.188951+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:51:16.188951+0000","last_undegraded":"2024-03-07T06:51:16.188951+0000","last_fullsized":"2024-03-07T06:51:16.188951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414355+0000","last_change":"2024-03-07T06:51:14.172051+0000","last_active":"2024-03-07T06:51:17.414355+0000","last_peered":"2024-03-07T06:51:17.414355+0000","last_clean":"2024-03-07T06:51:17.414355+0000","last_became_active":"2024-03-07T06:51:14.170736+0000","last_became_peered":"2024-03-07T06:51:14.170736+0000","last_unstale":"2024-03-07T06:51:17.414355+0000","last_undegraded":"2024-03-07T06:51:17.414355+0000","last_fullsized":"2024-03-07T06:51:17.414355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415537+0000","last_change":"2024-03-07T06:51:15.208661+0000","last_active":"2024-03-07T06:51:17.415537+0000","last_peered":"2024-03-07T06:51:17.415537+0000","last_clean":"2024-03-07T06:51:17.415537+0000","last_became_active":"2024-03-07T06:51:15.208394+0000","last_became_peered":"2024-03-07T06:51:15.208394+0000","last_unstale":"2024-03-07T06:51:17.415537+0000","last_undegraded":"2024-03-07T06:51:17.415537+0000","last_fullsized":"2024-03-07T06:51:17.415537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188640+0000","last_change":"2024-03-07T06:51:14.163185+0000","last_active":"2024-03-07T06:51:16.188640+0000","last_peered":"2024-03-07T06:51:16.188640+0000","last_clean":"2024-03-07T06:51:16.188640+0000","last_became_active":"2024-03-07T06:51:14.162494+0000","last_became_peered":"2024-03-07T06:51:14.162494+0000","last_unstale":"2024-03-07T06:51:16.188640+0000","last_undegraded":"2024-03-07T06:51:16.188640+0000","last_fullsized":"2024-03-07T06:51:16.188640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412672+0000","last_change":"2024-03-07T06:51:15.193638+0000","last_active":"2024-03-07T06:51:17.412672+0000","last_peered":"2024-03-07T06:51:17.412672+0000","last_clean":"2024-03-07T06:51:17.412672+0000","last_became_active":"2024-03-07T06:51:15.193379+0000","last_became_peered":"2024-03-07T06:51:15.193379+0000","last_unstale":"2024-03-07T06:51:17.412672+0000","last_undegraded":"2024-03-07T06:51:17.412672+0000","last_fullsized":"2024-03-07T06:51:17.412672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:16:10.967500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412927+0000","last_change":"2024-03-07T06:51:14.163793+0000","last_active":"2024-03-07T06:51:17.412927+0000","last_peered":"2024-03-07T06:51:17.412927+0000","last_clean":"2024-03-07T06:51:17.412927+0000","last_became_active":"2024-03-07T06:51:14.162059+0000","last_became_peered":"2024-03-07T06:51:14.162059+0000","last_unstale":"2024-03-07T06:51:17.412927+0000","last_undegraded":"2024-03-07T06:51:17.412927+0000","last_fullsized":"2024-03-07T06:51:17.412927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205257+0000","last_change":"2024-03-07T06:51:15.187522+0000","last_active":"2024-03-07T06:51:16.205257+0000","last_peered":"2024-03-07T06:51:16.205257+0000","last_clean":"2024-03-07T06:51:16.205257+0000","last_became_active":"2024-03-07T06:51:15.179882+0000","last_became_peered":"2024-03-07T06:51:15.179882+0000","last_unstale":"2024-03-07T06:51:16.205257+0000","last_undegraded":"2024-03-07T06:51:16.205257+0000","last_fullsized":"2024-03-07T06:51:16.205257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415629+0000","last_change":"2024-03-07T06:51:14.164766+0000","last_active":"2024-03-07T06:51:17.415629+0000","last_peered":"2024-03-07T06:51:17.415629+0000","last_clean":"2024-03-07T06:51:17.415629+0000","last_became_active":"2024-03-07T06:51:14.164397+0000","last_became_peered":"2024-03-07T06:51:14.164397+0000","last_unstale":"2024-03-07T06:51:17.415629+0000","last_undegraded":"2024-03-07T06:51:17.415629+0000","last_fullsized":"2024-03-07T06:51:17.415629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192589+0000","last_change":"2024-03-07T06:51:15.190608+0000","last_active":"2024-03-07T06:51:16.192589+0000","last_peered":"2024-03-07T06:51:16.192589+0000","last_clean":"2024-03-07T06:51:16.192589+0000","last_became_active":"2024-03-07T06:51:15.190326+0000","last_became_peered":"2024-03-07T06:51:15.190326+0000","last_unstale":"2024-03-07T06:51:16.192589+0000","last_undegraded":"2024-03-07T06:51:16.192589+0000","last_fullsized":"2024-03-07T06:51:16.192589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:43.574428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205896+0000","last_change":"2024-03-07T06:51:14.164402+0000","last_active":"2024-03-07T06:51:16.205896+0000","last_peered":"2024-03-07T06:51:16.205896+0000","last_clean":"2024-03-07T06:51:16.205896+0000","last_became_active":"2024-03-07T06:51:14.164081+0000","last_became_peered":"2024-03-07T06:51:14.164081+0000","last_unstale":"2024-03-07T06:51:16.205896+0000","last_undegraded":"2024-03-07T06:51:16.205896+0000","last_fullsized":"2024-03-07T06:51:16.205896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415903+0000","last_change":"2024-03-07T06:51:15.208063+0000","last_active":"2024-03-07T06:51:17.415903+0000","last_peered":"2024-03-07T06:51:17.415903+0000","last_clean":"2024-03-07T06:51:17.415903+0000","last_became_active":"2024-03-07T06:51:15.207782+0000","last_became_peered":"2024-03-07T06:51:15.207782+0000","last_unstale":"2024-03-07T06:51:17.415903+0000","last_undegraded":"2024-03-07T06:51:17.415903+0000","last_fullsized":"2024-03-07T06:51:17.415903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:02:55.966834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189333+0000","last_change":"2024-03-07T06:51:14.156560+0000","last_active":"2024-03-07T06:51:16.189333+0000","last_peered":"2024-03-07T06:51:16.189333+0000","last_clean":"2024-03-07T06:51:16.189333+0000","last_became_active":"2024-03-07T06:51:14.156040+0000","last_became_peered":"2024-03-07T06:51:14.156040+0000","last_unstale":"2024-03-07T06:51:16.189333+0000","last_undegraded":"2024-03-07T06:51:16.189333+0000","last_fullsized":"2024-03-07T06:51:16.189333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:07:44.222048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205750+0000","last_change":"2024-03-07T06:51:15.180806+0000","last_active":"2024-03-07T06:51:16.205750+0000","last_peered":"2024-03-07T06:51:16.205750+0000","last_clean":"2024-03-07T06:51:16.205750+0000","last_became_active":"2024-03-07T06:51:15.180402+0000","last_became_peered":"2024-03-07T06:51:15.180402+0000","last_unstale":"2024-03-07T06:51:16.205750+0000","last_undegraded":"2024-03-07T06:51:16.205750+0000","last_fullsized":"2024-03-07T06:51:16.205750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205976+0000","last_change":"2024-03-07T06:51:14.168110+0000","last_active":"2024-03-07T06:51:16.205976+0000","last_peered":"2024-03-07T06:51:16.205976+0000","last_clean":"2024-03-07T06:51:16.205976+0000","last_became_active":"2024-03-07T06:51:14.167759+0000","last_became_peered":"2024-03-07T06:51:14.167759+0000","last_unstale":"2024-03-07T06:51:16.205976+0000","last_undegraded":"2024-03-07T06:51:16.205976+0000","last_fullsized":"2024-03-07T06:51:16.205976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.416085+0000","last_change":"2024-03-07T06:51:15.199832+0000","last_active":"2024-03-07T06:51:17.416085+0000","last_peered":"2024-03-07T06:51:17.416085+0000","last_clean":"2024-03-07T06:51:17.416085+0000","last_became_active":"2024-03-07T06:51:15.199572+0000","last_became_peered":"2024-03-07T06:51:15.199572+0000","last_unstale":"2024-03-07T06:51:17.416085+0000","last_undegraded":"2024-03-07T06:51:17.416085+0000","last_fullsized":"2024-03-07T06:51:17.416085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193650+0000","last_change":"2024-03-07T06:51:14.173549+0000","last_active":"2024-03-07T06:51:16.193650+0000","last_peered":"2024-03-07T06:51:16.193650+0000","last_clean":"2024-03-07T06:51:16.193650+0000","last_became_active":"2024-03-07T06:51:14.173251+0000","last_became_peered":"2024-03-07T06:51:14.173251+0000","last_unstale":"2024-03-07T06:51:16.193650+0000","last_undegraded":"2024-03-07T06:51:16.193650+0000","last_fullsized":"2024-03-07T06:51:16.193650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413748+0000","last_change":"2024-03-07T06:51:15.195178+0000","last_active":"2024-03-07T06:51:17.413748+0000","last_peered":"2024-03-07T06:51:17.413748+0000","last_clean":"2024-03-07T06:51:17.413748+0000","last_became_active":"2024-03-07T06:51:15.194907+0000","last_became_peered":"2024-03-07T06:51:15.194907+0000","last_unstale":"2024-03-07T06:51:17.413748+0000","last_undegraded":"2024-03-07T06:51:17.413748+0000","last_fullsized":"2024-03-07T06:51:17.413748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188590+0000","last_change":"2024-03-07T06:51:14.156566+0000","last_active":"2024-03-07T06:51:16.188590+0000","last_peered":"2024-03-07T06:51:16.188590+0000","last_clean":"2024-03-07T06:51:16.188590+0000","last_became_active":"2024-03-07T06:51:14.156261+0000","last_became_peered":"2024-03-07T06:51:14.156261+0000","last_unstale":"2024-03-07T06:51:16.188590+0000","last_undegraded":"2024-03-07T06:51:16.188590+0000","last_fullsized":"2024-03-07T06:51:16.188590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193424+0000","last_change":"2024-03-07T06:51:15.183065+0000","last_active":"2024-03-07T06:51:16.193424+0000","last_peered":"2024-03-07T06:51:16.193424+0000","last_clean":"2024-03-07T06:51:16.193424+0000","last_became_active":"2024-03-07T06:51:15.182189+0000","last_became_peered":"2024-03-07T06:51:15.182189+0000","last_unstale":"2024-03-07T06:51:16.193424+0000","last_undegraded":"2024-03-07T06:51:16.193424+0000","last_fullsized":"2024-03-07T06:51:16.193424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:05:21.883987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205102+0000","last_change":"2024-03-07T06:51:14.169705+0000","last_active":"2024-03-07T06:51:16.205102+0000","last_peered":"2024-03-07T06:51:16.205102+0000","last_clean":"2024-03-07T06:51:16.205102+0000","last_became_active":"2024-03-07T06:51:14.169444+0000","last_became_peered":"2024-03-07T06:51:14.169444+0000","last_unstale":"2024-03-07T06:51:16.205102+0000","last_undegraded":"2024-03-07T06:51:16.205102+0000","last_fullsized":"2024-03-07T06:51:16.205102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415220+0000","last_change":"2024-03-07T06:51:15.197549+0000","last_active":"2024-03-07T06:51:17.415220+0000","last_peered":"2024-03-07T06:51:17.415220+0000","last_clean":"2024-03-07T06:51:17.415220+0000","last_became_active":"2024-03-07T06:51:15.195020+0000","last_became_peered":"2024-03-07T06:51:15.195020+0000","last_unstale":"2024-03-07T06:51:17.415220+0000","last_undegraded":"2024-03-07T06:51:17.415220+0000","last_fullsized":"2024-03-07T06:51:17.415220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205415+0000","last_change":"2024-03-07T06:51:14.170410+0000","last_active":"2024-03-07T06:51:16.205415+0000","last_peered":"2024-03-07T06:51:16.205415+0000","last_clean":"2024-03-07T06:51:16.205415+0000","last_became_active":"2024-03-07T06:51:14.170043+0000","last_became_peered":"2024-03-07T06:51:14.170043+0000","last_unstale":"2024-03-07T06:51:16.205415+0000","last_undegraded":"2024-03-07T06:51:16.205415+0000","last_fullsized":"2024-03-07T06:51:16.205415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415636+0000","last_change":"2024-03-07T06:51:15.193076+0000","last_active":"2024-03-07T06:51:17.415636+0000","last_peered":"2024-03-07T06:51:17.415636+0000","last_clean":"2024-03-07T06:51:17.415636+0000","last_became_active":"2024-03-07T06:51:15.192700+0000","last_became_peered":"2024-03-07T06:51:15.192700+0000","last_unstale":"2024-03-07T06:51:17.415636+0000","last_undegraded":"2024-03-07T06:51:17.415636+0000","last_fullsized":"2024-03-07T06:51:17.415636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191169+0000","last_change":"2024-03-07T06:51:14.173220+0000","last_active":"2024-03-07T06:51:16.191169+0000","last_peered":"2024-03-07T06:51:16.191169+0000","last_clean":"2024-03-07T06:51:16.191169+0000","last_became_active":"2024-03-07T06:51:14.169687+0000","last_became_peered":"2024-03-07T06:51:14.169687+0000","last_unstale":"2024-03-07T06:51:16.191169+0000","last_undegraded":"2024-03-07T06:51:16.191169+0000","last_fullsized":"2024-03-07T06:51:16.191169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:02:40.568300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192059+0000","last_change":"2024-03-07T06:51:15.205150+0000","last_active":"2024-03-07T06:51:16.192059+0000","last_peered":"2024-03-07T06:51:16.192059+0000","last_clean":"2024-03-07T06:51:16.192059+0000","last_became_active":"2024-03-07T06:51:15.204435+0000","last_became_peered":"2024-03-07T06:51:15.204435+0000","last_unstale":"2024-03-07T06:51:16.192059+0000","last_undegraded":"2024-03-07T06:51:16.192059+0000","last_fullsized":"2024-03-07T06:51:16.192059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415532+0000","last_change":"2024-03-07T06:51:15.214214+0000","last_active":"2024-03-07T06:51:17.415532+0000","last_peered":"2024-03-07T06:51:17.415532+0000","last_clean":"2024-03-07T06:51:17.415532+0000","last_became_active":"2024-03-07T06:51:15.209505+0000","last_became_peered":"2024-03-07T06:51:15.209505+0000","last_unstale":"2024-03-07T06:51:17.415532+0000","last_undegraded":"2024-03-07T06:51:17.415532+0000","last_fullsized":"2024-03-07T06:51:17.415532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:56.465649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188962+0000","last_change":"2024-03-07T06:51:14.172650+0000","last_active":"2024-03-07T06:51:16.188962+0000","last_peered":"2024-03-07T06:51:16.188962+0000","last_clean":"2024-03-07T06:51:16.188962+0000","last_became_active":"2024-03-07T06:51:14.170441+0000","last_became_peered":"2024-03-07T06:51:14.170441+0000","last_unstale":"2024-03-07T06:51:16.188962+0000","last_undegraded":"2024-03-07T06:51:16.188962+0000","last_fullsized":"2024-03-07T06:51:16.188962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:05:20.871811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415022+0000","last_change":"2024-03-07T06:51:15.204124+0000","last_active":"2024-03-07T06:51:17.415022+0000","last_peered":"2024-03-07T06:51:17.415022+0000","last_clean":"2024-03-07T06:51:17.415022+0000","last_became_active":"2024-03-07T06:51:15.203713+0000","last_became_peered":"2024-03-07T06:51:15.203713+0000","last_unstale":"2024-03-07T06:51:17.415022+0000","last_undegraded":"2024-03-07T06:51:17.415022+0000","last_fullsized":"2024-03-07T06:51:17.415022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:34:40.365080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189015+0000","last_change":"2024-03-07T06:51:14.167422+0000","last_active":"2024-03-07T06:51:16.189015+0000","last_peered":"2024-03-07T06:51:16.189015+0000","last_clean":"2024-03-07T06:51:16.189015+0000","last_became_active":"2024-03-07T06:51:14.167152+0000","last_became_peered":"2024-03-07T06:51:14.167152+0000","last_unstale":"2024-03-07T06:51:16.189015+0000","last_undegraded":"2024-03-07T06:51:16.189015+0000","last_fullsized":"2024-03-07T06:51:16.189015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:40:50.922964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412609+0000","last_change":"2024-03-07T06:51:15.210848+0000","last_active":"2024-03-07T06:51:17.412609+0000","last_peered":"2024-03-07T06:51:17.412609+0000","last_clean":"2024-03-07T06:51:17.412609+0000","last_became_active":"2024-03-07T06:51:15.210526+0000","last_became_peered":"2024-03-07T06:51:15.210526+0000","last_unstale":"2024-03-07T06:51:17.412609+0000","last_undegraded":"2024-03-07T06:51:17.412609+0000","last_fullsized":"2024-03-07T06:51:17.412609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:40:57.934235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415818+0000","last_change":"2024-03-07T06:51:14.163311+0000","last_active":"2024-03-07T06:51:17.415818+0000","last_peered":"2024-03-07T06:51:17.415818+0000","last_clean":"2024-03-07T06:51:17.415818+0000","last_became_active":"2024-03-07T06:51:14.161332+0000","last_became_peered":"2024-03-07T06:51:14.161332+0000","last_unstale":"2024-03-07T06:51:17.415818+0000","last_undegraded":"2024-03-07T06:51:17.415818+0000","last_fullsized":"2024-03-07T06:51:17.415818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415995+0000","last_change":"2024-03-07T06:51:15.200486+0000","last_active":"2024-03-07T06:51:17.415995+0000","last_peered":"2024-03-07T06:51:17.415995+0000","last_clean":"2024-03-07T06:51:17.415995+0000","last_became_active":"2024-03-07T06:51:15.200171+0000","last_became_peered":"2024-03-07T06:51:15.200171+0000","last_unstale":"2024-03-07T06:51:17.415995+0000","last_undegraded":"2024-03-07T06:51:17.415995+0000","last_fullsized":"2024-03-07T06:51:17.415995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:22:38.973224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412810+0000","last_change":"2024-03-07T06:51:14.169164+0000","last_active":"2024-03-07T06:51:17.412810+0000","last_peered":"2024-03-07T06:51:17.412810+0000","last_clean":"2024-03-07T06:51:17.412810+0000","last_became_active":"2024-03-07T06:51:14.168662+0000","last_became_peered":"2024-03-07T06:51:14.168662+0000","last_unstale":"2024-03-07T06:51:17.412810+0000","last_undegraded":"2024-03-07T06:51:17.412810+0000","last_fullsized":"2024-03-07T06:51:17.412810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:51:20.637 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-07T06:51:20.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.885+0000 7fd11da00640 1 Processor -- start 2024-03-07T06:51:20.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.885+0000 7fd11da00640 1 -- start start 2024-03-07T06:51:20.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.886+0000 7fd11da00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd11815fbb0 0x7fd1180cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:20.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.886+0000 7fd11da00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1180731b0 con 0x7fd1180cdee0 2024-03-07T06:51:20.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.886+0000 7fd11da00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd118072150 con 0x7fd118072950 2024-03-07T06:51:20.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.886+0000 7fd11da00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd118072f60 con 0x7fd11815fbb0 2024-03-07T06:51:20.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.887+0000 7fd116a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd11815fbb0 0x7fd1180cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:20.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.887+0000 7fd116a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd11815fbb0 0x7fd1180cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38410/0 (socket says 172.21.15.133:38410) 2024-03-07T06:51:20.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.887+0000 7fd116a00640 1 -- 172.21.15.133:0/838277328 learned_addr learned my addr 172.21.15.133:0/838277328 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:20.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.887+0000 7fd116a00640 1 -- 172.21.15.133:0/838277328 >> v1:172.21.15.161:6789/0 conn(0x7fd118072950 legacy=0x7fd11815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:20.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.887+0000 7fd116a00640 1 -- 172.21.15.133:0/838277328 >> v1:172.21.15.133:6789/0 conn(0x7fd1180cdee0 legacy=0x7fd11815c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:20.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.887+0000 7fd116a00640 1 -- 172.21.15.133:0/838277328 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd118072430 con 0x7fd11815fbb0 2024-03-07T06:51:20.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.888+0000 7fd116a00640 1 --2- 172.21.15.133:0/838277328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd11815fbb0 0x7fd1180cf7d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd108009870 tx=0x7fd10802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1eab6d9c96dd0d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:20.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.888+0000 7fd116000640 1 -- 172.21.15.133:0/838277328 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd108035020 con 0x7fd11815fbb0 2024-03-07T06:51:20.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.888+0000 7fd116000640 1 -- 172.21.15.133:0/838277328 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd108031870 con 0x7fd11815fbb0 2024-03-07T06:51:20.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.888+0000 7fd116000640 1 -- 172.21.15.133:0/838277328 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd108031c40 con 0x7fd11815fbb0 2024-03-07T06:51:20.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.889+0000 7fd11da00640 1 -- 172.21.15.133:0/838277328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd11815fbb0 msgr2=0x7fd1180cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:20.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.889+0000 7fd11da00640 1 --2- 172.21.15.133:0/838277328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd11815fbb0 0x7fd1180cf7d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd108009870 tx=0x7fd10802b910 comp rx=0 tx=0).stop 2024-03-07T06:51:20.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.889+0000 7fd11da00640 1 -- 172.21.15.133:0/838277328 shutdown_connections 2024-03-07T06:51:20.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.889+0000 7fd11da00640 1 --2- 172.21.15.133:0/838277328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd11815fbb0 0x7fd1180cf7d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.889+0000 7fd11da00640 1 -- 172.21.15.133:0/838277328 >> 172.21.15.133:0/838277328 conn(0x7fd1180b3120 msgr2=0x7fd1180b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:20.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.889+0000 7fd11da00640 1 -- 172.21.15.133:0/838277328 shutdown_connections 2024-03-07T06:51:20.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.889+0000 7fd11da00640 1 -- 172.21.15.133:0/838277328 wait complete. 2024-03-07T06:51:20.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.891+0000 7fd11da00640 1 Processor -- start 2024-03-07T06:51:20.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.891+0000 7fd11da00640 1 -- start start 2024-03-07T06:51:20.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd11da00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1180cdee0 0x7fd118159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:20.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd11da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd11815fbb0 0x7fd118156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:20.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd11da00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1181577f0 0x7fd1181466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:20.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd11da00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1180ce2c0 con 0x7fd11815fbb0 2024-03-07T06:51:20.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd11da00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd118073490 con 0x7fd1181577f0 2024-03-07T06:51:20.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd11da00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1180b1a00 con 0x7fd1180cdee0 2024-03-07T06:51:20.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd10e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd11815fbb0 0x7fd118156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:20.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd10e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd11815fbb0 0x7fd118156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38420/0 (socket says 172.21.15.133:38420) 2024-03-07T06:51:20.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd10e000640 1 -- 172.21.15.133:0/1416424657 learned_addr learned my addr 172.21.15.133:0/1416424657 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:20.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd116a00640 1 --2- 172.21.15.133:0/1416424657 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1180cdee0 0x7fd118159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:20.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd117400640 1 --2- 172.21.15.133:0/1416424657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1181577f0 0x7fd1181466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:20.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd10e000640 1 -- 172.21.15.133:0/1416424657 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1180cdee0 msgr2=0x7fd118159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:20.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd10e000640 1 --2- 172.21.15.133:0/1416424657 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1180cdee0 0x7fd118159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd10e000640 1 -- 172.21.15.133:0/1416424657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1181577f0 msgr2=0x7fd1181466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:20.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd10e000640 1 --2- 172.21.15.133:0/1416424657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1181577f0 0x7fd1181466f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:20.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.892+0000 7fd10e000640 1 -- 172.21.15.133:0/1416424657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11815a400 con 0x7fd11815fbb0 2024-03-07T06:51:20.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.893+0000 7fd10e000640 1 --2- 172.21.15.133:0/1416424657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd11815fbb0 0x7fd118156790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd100009fd0 tx=0x7fd10000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:20.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.893+0000 7fd10f400640 1 -- 172.21.15.133:0/1416424657 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100015020 con 0x7fd11815fbb0 2024-03-07T06:51:20.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.893+0000 7fd10f400640 1 -- 172.21.15.133:0/1416424657 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd100011830 con 0x7fd11815fbb0 2024-03-07T06:51:20.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.893+0000 7fd10f400640 1 -- 172.21.15.133:0/1416424657 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd100011c00 con 0x7fd11815fbb0 2024-03-07T06:51:20.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.893+0000 7fd11da00640 1 -- 172.21.15.133:0/1416424657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd118155cb0 con 0x7fd11815fbb0 2024-03-07T06:51:20.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.893+0000 7fd11da00640 1 -- 172.21.15.133:0/1416424657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd118156270 con 0x7fd11815fbb0 2024-03-07T06:51:20.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.894+0000 7fd11da00640 1 -- 172.21.15.133:0/1416424657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd118156470 con 0x7fd11815fbb0 2024-03-07T06:51:20.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.895+0000 7fd10f400640 1 -- 172.21.15.133:0/1416424657 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd100018020 con 0x7fd11815fbb0 2024-03-07T06:51:20.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.896+0000 7fd10f400640 1 --2- 172.21.15.133:0/1416424657 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd0f406b500 0x7fd0f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:20.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.897+0000 7fd10f400640 1 -- 172.21.15.133:0/1416424657 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd10008a170 con 0x7fd11815fbb0 2024-03-07T06:51:20.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.897+0000 7fd116a00640 1 --2- 172.21.15.133:0/1416424657 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd0f406b500 0x7fd0f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:20.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.898+0000 7fd116a00640 1 --2- 172.21.15.133:0/1416424657 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd0f406b500 0x7fd0f406d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd108036000 tx=0x7fd10803c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:20.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:20.901+0000 7fd10f400640 1 -- 172.21.15.133:0/1416424657 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd118156470 con 0x7fd11815fbb0 2024-03-07T06:51:20.912 INFO:tasks.ceph.mds.d.smithi161.stderr:2024-03-07T06:51:20.913+0000 7f32ca200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-07T06:51:21.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.163+0000 7fd11da00640 1 -- 172.21.15.133:0/1416424657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd118062c90 con 0x7fd11815fbb0 2024-03-07T06:51:21.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.165+0000 7fd10f400640 1 -- 172.21.15.133:0/1416424657 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1194 (secure 0 0 0) 0x7fd10001f020 con 0x7fd11815fbb0 2024-03-07T06:51:21.164 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:21.164 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":31,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":21,"num_osds":8,"num_up_osds":8,"osd_up_since":1709794262,"num_in_osds":8,"osd_in_since":1709794253,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":224825344,"bytes_avail":772869287936,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042,"read_bytes_sec":1724,"write_bytes_sec":50197,"read_op_per_sec":1,"write_op_per_sec":2},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4288}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-03-07T06:50:49.326869+0000","services":{}},"progress_events":{}} 2024-03-07T06:51:21.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.167+0000 7fd11da00640 1 -- 172.21.15.133:0/1416424657 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd0f406b500 msgr2=0x7fd0f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:21.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.168+0000 7fd11da00640 1 --2- 172.21.15.133:0/1416424657 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd0f406b500 0x7fd0f406d9c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd108036000 tx=0x7fd10803c040 comp rx=0 tx=0).stop 2024-03-07T06:51:21.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.168+0000 7fd11da00640 1 -- 172.21.15.133:0/1416424657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd11815fbb0 msgr2=0x7fd118156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:21.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.168+0000 7fd11da00640 1 --2- 172.21.15.133:0/1416424657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd11815fbb0 0x7fd118156790 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd100009fd0 tx=0x7fd10000b870 comp rx=0 tx=0).stop 2024-03-07T06:51:21.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.168+0000 7fd11da00640 1 -- 172.21.15.133:0/1416424657 shutdown_connections 2024-03-07T06:51:21.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.168+0000 7fd11da00640 1 --2- 172.21.15.133:0/1416424657 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd0f406b500 0x7fd0f406d9c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:21.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.168+0000 7fd11da00640 1 --2- 172.21.15.133:0/1416424657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1181577f0 0x7fd1181466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:21.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.168+0000 7fd11da00640 1 --2- 172.21.15.133:0/1416424657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd11815fbb0 0x7fd118156790 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:21.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.168+0000 7fd11da00640 1 --2- 172.21.15.133:0/1416424657 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1180cdee0 0x7fd118159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:21.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.168+0000 7fd11da00640 1 -- 172.21.15.133:0/1416424657 >> 172.21.15.133:0/1416424657 conn(0x7fd1180b3120 msgr2=0x7fd11815ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:21.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.168+0000 7fd11da00640 1 -- 172.21.15.133:0/1416424657 shutdown_connections 2024-03-07T06:51:21.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.168+0000 7fd11da00640 1 -- 172.21.15.133:0/1416424657 wait complete. 2024-03-07T06:51:21.185 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-07T06:51:21.185 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:51:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.387+0000 7fec27400640 1 Processor -- start 2024-03-07T06:51:21.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.387+0000 7fec27400640 1 -- start start 2024-03-07T06:51:21.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.388+0000 7fec27400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec20072950 0x7fec200cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:21.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.388+0000 7fec27400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fec200cdee0 0x7fec2015c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:21.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.388+0000 7fec27400640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec200731b0 con 0x7fec2015fbb0 2024-03-07T06:51:21.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.388+0000 7fec27400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec20072f60 con 0x7fec200cdee0 2024-03-07T06:51:21.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.388+0000 7fec27400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec20073490 con 0x7fec20072950 2024-03-07T06:51:21.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec24a00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fec2015fbb0 0x7fec2015ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44068/0 (socket says 172.21.15.133:44068) 2024-03-07T06:51:21.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec24a00640 1 -- 172.21.15.133:0/168050382 learned_addr learned my addr 172.21.15.133:0/168050382 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec17e00640 1 --2- 172.21.15.133:0/168050382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec20072950 0x7fec200cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec25400640 1 --2- 172.21.15.133:0/168050382 >> v2:172.21.15.161:3300/0 conn(0x7fec200cdee0 0x7fec2015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec1fe00640 1 -- 172.21.15.133:0/168050382 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037855495 0 0) 0x7fec200731b0 con 0x7fec2015fbb0 2024-03-07T06:51:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec1fe00640 1 -- 172.21.15.133:0/168050382 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec04003520 con 0x7fec2015fbb0 2024-03-07T06:51:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec1fe00640 1 -- 172.21.15.133:0/168050382 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2015710296 0 0) 0x7fec04003520 con 0x7fec2015fbb0 2024-03-07T06:51:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec1fe00640 1 -- 172.21.15.133:0/168050382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec20072950 msgr2=0x7fec200cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec1fe00640 1 --2- 172.21.15.133:0/168050382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec20072950 0x7fec200cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec1fe00640 1 -- 172.21.15.133:0/168050382 >> v2:172.21.15.161:3300/0 conn(0x7fec200cdee0 msgr2=0x7fec2015c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec1fe00640 1 --2- 172.21.15.133:0/168050382 >> v2:172.21.15.161:3300/0 conn(0x7fec200cdee0 0x7fec2015c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec1fe00640 1 -- 172.21.15.133:0/168050382 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec20072150 con 0x7fec2015fbb0 2024-03-07T06:51:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec1fe00640 1 -- 172.21.15.133:0/168050382 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fec10001670 con 0x7fec2015fbb0 2024-03-07T06:51:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec1fe00640 1 -- 172.21.15.133:0/168050382 >> v1:172.21.15.133:6789/0 conn(0x7fec2015fbb0 legacy=0x7fec2015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:21.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec1fe00640 1 --2- 172.21.15.133:0/168050382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec040039b0 0x7fec04003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec1fe00640 1 --2- 172.21.15.133:0/168050382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec04004800 0x7fec04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.389+0000 7fec1fe00640 1 --2- 172.21.15.133:0/168050382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec040051c0 0x7fec0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.390+0000 7fec1fe00640 1 -- 172.21.15.133:0/168050382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec20072f60 con 0x7fec040039b0 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.390+0000 7fec1fe00640 1 -- 172.21.15.133:0/168050382 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec20073490 con 0x7fec04004800 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.390+0000 7fec1fe00640 1 -- 172.21.15.133:0/168050382 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0400bf00 con 0x7fec040051c0 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.390+0000 7fec24a00640 1 --2- 172.21.15.133:0/168050382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec040039b0 0x7fec04003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.390+0000 7fec25400640 1 --2- 172.21.15.133:0/168050382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec040051c0 0x7fec0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.390+0000 7fec17e00640 1 --2- 172.21.15.133:0/168050382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec04004800 0x7fec04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.390+0000 7fec24a00640 1 -- 172.21.15.133:0/168050382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec040051c0 msgr2=0x7fec0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.390+0000 7fec24a00640 1 --2- 172.21.15.133:0/168050382 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec040051c0 0x7fec0400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.390+0000 7fec24a00640 1 -- 172.21.15.133:0/168050382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec04004800 msgr2=0x7fec04004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.390+0000 7fec24a00640 1 --2- 172.21.15.133:0/168050382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec04004800 0x7fec04004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.390+0000 7fec24a00640 1 -- 172.21.15.133:0/168050382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec04003520 con 0x7fec040039b0 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.390+0000 7fec17e00640 1 --2- 172.21.15.133:0/168050382 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec04004800 0x7fec04004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:21.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.391+0000 7fec24a00640 1 --2- 172.21.15.133:0/168050382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec040039b0 0x7fec04003df0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fec1000c160 tx=0x7fec04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b7c2338b7e919cb server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.391+0000 7fec1fe00640 1 -- 172.21.15.133:0/168050382 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec10007c80 con 0x7fec040039b0 2024-03-07T06:51:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.391+0000 7fec1fe00640 1 -- 172.21.15.133:0/168050382 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec10030070 con 0x7fec040039b0 2024-03-07T06:51:21.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.391+0000 7fec1fe00640 1 -- 172.21.15.133:0/168050382 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec10005cf0 con 0x7fec040039b0 2024-03-07T06:51:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.392+0000 7fec27400640 1 -- 172.21.15.133:0/168050382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec040039b0 msgr2=0x7fec04003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.392+0000 7fec27400640 1 --2- 172.21.15.133:0/168050382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec040039b0 0x7fec04003df0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fec1000c160 tx=0x7fec04005830 comp rx=0 tx=0).stop 2024-03-07T06:51:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.392+0000 7fec25400640 1 -- 172.21.15.133:0/168050382 reap_dead start 2024-03-07T06:51:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.392+0000 7fec27400640 1 -- 172.21.15.133:0/168050382 shutdown_connections 2024-03-07T06:51:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.392+0000 7fec27400640 1 -- 172.21.15.133:0/168050382 >> 172.21.15.133:0/168050382 conn(0x7fec200b3120 msgr2=0x7fec200b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.392+0000 7fec27400640 1 -- 172.21.15.133:0/168050382 shutdown_connections 2024-03-07T06:51:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.392+0000 7fec27400640 1 -- 172.21.15.133:0/168050382 wait complete. 2024-03-07T06:51:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.394+0000 7fec27400640 1 Processor -- start 2024-03-07T06:51:21.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.394+0000 7fec27400640 1 -- start start 2024-03-07T06:51:21.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.395+0000 7fec27400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec040039b0 0x7fec2009b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:21.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.395+0000 7fec27400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec20145aa0 0x7fec2009bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:21.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.395+0000 7fec27400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec2009c1d0 0x7fec2015e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.395+0000 7fec27400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec20160980 con 0x7fec20145aa0 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.395+0000 7fec27400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec200ce2f0 con 0x7fec2009c1d0 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.395+0000 7fec27400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec200b1a00 con 0x7fec040039b0 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.395+0000 7fec17e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec20145aa0 0x7fec2009bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.395+0000 7fec17e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec20145aa0 0x7fec2009bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38438/0 (socket says 172.21.15.133:38438) 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.395+0000 7fec17e00640 1 -- 172.21.15.133:0/3226543673 learned_addr learned my addr 172.21.15.133:0/3226543673 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.395+0000 7fec25400640 1 --2- 172.21.15.133:0/3226543673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec2009c1d0 0x7fec2015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.395+0000 7fec24a00640 1 --2- 172.21.15.133:0/3226543673 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec040039b0 0x7fec2009b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.396+0000 7fec17e00640 1 -- 172.21.15.133:0/3226543673 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec040039b0 msgr2=0x7fec2009b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.396+0000 7fec17e00640 1 --2- 172.21.15.133:0/3226543673 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec040039b0 0x7fec2009b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.396+0000 7fec17e00640 1 -- 172.21.15.133:0/3226543673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec2009c1d0 msgr2=0x7fec2015e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.396+0000 7fec17e00640 1 --2- 172.21.15.133:0/3226543673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec2009c1d0 0x7fec2015e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.396+0000 7fec17e00640 1 -- 172.21.15.133:0/3226543673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2015ef60 con 0x7fec20145aa0 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.396+0000 7fec17e00640 1 --2- 172.21.15.133:0/3226543673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec20145aa0 0x7fec2009bc90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec080108b0 tx=0x7fec08010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:21.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.396+0000 7fec1d600640 1 -- 172.21.15.133:0/3226543673 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0800f040 con 0x7fec20145aa0 2024-03-07T06:51:21.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.396+0000 7fec1d600640 1 -- 172.21.15.133:0/3226543673 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0800eb60 con 0x7fec20145aa0 2024-03-07T06:51:21.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.396+0000 7fec1d600640 1 -- 172.21.15.133:0/3226543673 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0801c400 con 0x7fec20145aa0 2024-03-07T06:51:21.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.396+0000 7fec27400640 1 -- 172.21.15.133:0/3226543673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2009ab00 con 0x7fec20145aa0 2024-03-07T06:51:21.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.396+0000 7fec27400640 1 -- 172.21.15.133:0/3226543673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec20145f20 con 0x7fec20145aa0 2024-03-07T06:51:21.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.397+0000 7fec27400640 1 -- 172.21.15.133:0/3226543673 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec20062c90 con 0x7fec20145aa0 2024-03-07T06:51:21.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.398+0000 7fec1d600640 1 -- 172.21.15.133:0/3226543673 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fec0801c5a0 con 0x7fec20145aa0 2024-03-07T06:51:21.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.400+0000 7fec1d600640 1 --2- 172.21.15.133:0/3226543673 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7febfc06b4d0 0x7febfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:21.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.400+0000 7fec1d600640 1 -- 172.21.15.133:0/3226543673 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec0808a8e0 con 0x7fec20145aa0 2024-03-07T06:51:21.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.400+0000 7fec24a00640 1 --2- 172.21.15.133:0/3226543673 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7febfc06b4d0 0x7febfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:21.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.401+0000 7fec24a00640 1 --2- 172.21.15.133:0/3226543673 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7febfc06b4d0 0x7febfc06d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fec10034000 tx=0x7fec10007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:21.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.404+0000 7fec1d600640 1 -- 172.21.15.133:0/3226543673 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fec08018030 con 0x7fec20145aa0 2024-03-07T06:51:21.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.595+0000 7fec27400640 1 -- 172.21.15.133:0/3226543673 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec20072d40 con 0x7febfc06b4d0 2024-03-07T06:51:21.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.630+0000 7fec1d600640 1 -- 172.21.15.133:0/3226543673 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251947 (secure 0 0 0) 0x7fec10082840 con 0x7febfc06b4d0 2024-03-07T06:51:21.632 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:21.634 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:51:21.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.635+0000 7fec27400640 1 -- 172.21.15.133:0/3226543673 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7febfc06b4d0 msgr2=0x7febfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:21.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.635+0000 7fec27400640 1 --2- 172.21.15.133:0/3226543673 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7febfc06b4d0 0x7febfc06d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fec10034000 tx=0x7fec10007000 comp rx=0 tx=0).stop 2024-03-07T06:51:21.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.635+0000 7fec27400640 1 -- 172.21.15.133:0/3226543673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec20145aa0 msgr2=0x7fec2009bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:21.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.635+0000 7fec27400640 1 --2- 172.21.15.133:0/3226543673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec20145aa0 0x7fec2009bc90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec080108b0 tx=0x7fec08010d70 comp rx=0 tx=0).stop 2024-03-07T06:51:21.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.635+0000 7fec27400640 1 -- 172.21.15.133:0/3226543673 shutdown_connections 2024-03-07T06:51:21.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.635+0000 7fec27400640 1 --2- 172.21.15.133:0/3226543673 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7febfc06b4d0 0x7febfc06d990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:21.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.635+0000 7fec27400640 1 --2- 172.21.15.133:0/3226543673 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fec2009c1d0 0x7fec2015e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:21.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.635+0000 7fec27400640 1 --2- 172.21.15.133:0/3226543673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec20145aa0 0x7fec2009bc90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:21.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.635+0000 7fec27400640 1 --2- 172.21.15.133:0/3226543673 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fec040039b0 0x7fec2009b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:21.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.636+0000 7fec27400640 1 -- 172.21.15.133:0/3226543673 >> 172.21.15.133:0/3226543673 conn(0x7fec200b3120 msgr2=0x7fec200716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:21.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.636+0000 7fec27400640 1 -- 172.21.15.133:0/3226543673 shutdown_connections 2024-03-07T06:51:21.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:21.636+0000 7fec27400640 1 -- 172.21.15.133:0/3226543673 wait complete. 2024-03-07T06:51:21.654 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-07T06:51:20.471309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219088,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755632,"statfs":{"total":773094113280,"available":772869767168,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.339070"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190674+0000","last_change":"2024-03-07T06:51:15.202253+0000","last_active":"2024-03-07T06:51:16.190674+0000","last_peered":"2024-03-07T06:51:16.190674+0000","last_clean":"2024-03-07T06:51:16.190674+0000","last_became_active":"2024-03-07T06:51:15.201848+0000","last_became_peered":"2024-03-07T06:51:15.201848+0000","last_unstale":"2024-03-07T06:51:16.190674+0000","last_undegraded":"2024-03-07T06:51:16.190674+0000","last_fullsized":"2024-03-07T06:51:16.190674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:41:54.645832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205980+0000","last_change":"2024-03-07T06:51:15.188301+0000","last_active":"2024-03-07T06:51:16.205980+0000","last_peered":"2024-03-07T06:51:16.205980+0000","last_clean":"2024-03-07T06:51:16.205980+0000","last_became_active":"2024-03-07T06:51:15.188021+0000","last_became_peered":"2024-03-07T06:51:15.188021+0000","last_unstale":"2024-03-07T06:51:16.205980+0000","last_undegraded":"2024-03-07T06:51:16.205980+0000","last_fullsized":"2024-03-07T06:51:16.205980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412687+0000","last_change":"2024-03-07T06:51:15.188296+0000","last_active":"2024-03-07T06:51:17.412687+0000","last_peered":"2024-03-07T06:51:17.412687+0000","last_clean":"2024-03-07T06:51:17.412687+0000","last_became_active":"2024-03-07T06:51:15.187954+0000","last_became_peered":"2024-03-07T06:51:15.187954+0000","last_unstale":"2024-03-07T06:51:17.412687+0000","last_undegraded":"2024-03-07T06:51:17.412687+0000","last_fullsized":"2024-03-07T06:51:17.412687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414882+0000","last_change":"2024-03-07T06:51:15.200894+0000","last_active":"2024-03-07T06:51:17.414882+0000","last_peered":"2024-03-07T06:51:17.414882+0000","last_clean":"2024-03-07T06:51:17.414882+0000","last_became_active":"2024-03-07T06:51:15.200617+0000","last_became_peered":"2024-03-07T06:51:15.200617+0000","last_unstale":"2024-03-07T06:51:17.414882+0000","last_undegraded":"2024-03-07T06:51:17.414882+0000","last_fullsized":"2024-03-07T06:51:17.414882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190864+0000","last_change":"2024-03-07T06:51:15.189383+0000","last_active":"2024-03-07T06:51:16.190864+0000","last_peered":"2024-03-07T06:51:16.190864+0000","last_clean":"2024-03-07T06:51:16.190864+0000","last_became_active":"2024-03-07T06:51:15.188192+0000","last_became_peered":"2024-03-07T06:51:15.188192+0000","last_unstale":"2024-03-07T06:51:16.190864+0000","last_undegraded":"2024-03-07T06:51:16.190864+0000","last_fullsized":"2024-03-07T06:51:16.190864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:07:16.800545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414731+0000","last_change":"2024-03-07T06:51:15.195882+0000","last_active":"2024-03-07T06:51:17.414731+0000","last_peered":"2024-03-07T06:51:17.414731+0000","last_clean":"2024-03-07T06:51:17.414731+0000","last_became_active":"2024-03-07T06:51:15.195572+0000","last_became_peered":"2024-03-07T06:51:15.195572+0000","last_unstale":"2024-03-07T06:51:17.414731+0000","last_undegraded":"2024-03-07T06:51:17.414731+0000","last_fullsized":"2024-03-07T06:51:17.414731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189735+0000","last_change":"2024-03-07T06:51:15.190861+0000","last_active":"2024-03-07T06:51:16.189735+0000","last_peered":"2024-03-07T06:51:16.189735+0000","last_clean":"2024-03-07T06:51:16.189735+0000","last_became_active":"2024-03-07T06:51:15.190546+0000","last_became_peered":"2024-03-07T06:51:15.190546+0000","last_unstale":"2024-03-07T06:51:16.189735+0000","last_undegraded":"2024-03-07T06:51:16.189735+0000","last_fullsized":"2024-03-07T06:51:16.189735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:44:23.301530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415109+0000","last_change":"2024-03-07T06:51:15.202655+0000","last_active":"2024-03-07T06:51:17.415109+0000","last_peered":"2024-03-07T06:51:17.415109+0000","last_clean":"2024-03-07T06:51:17.415109+0000","last_became_active":"2024-03-07T06:51:15.197943+0000","last_became_peered":"2024-03-07T06:51:15.197943+0000","last_unstale":"2024-03-07T06:51:17.415109+0000","last_undegraded":"2024-03-07T06:51:17.415109+0000","last_fullsized":"2024-03-07T06:51:17.415109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190628+0000","last_change":"2024-03-07T06:51:15.209333+0000","last_active":"2024-03-07T06:51:16.190628+0000","last_peered":"2024-03-07T06:51:16.190628+0000","last_clean":"2024-03-07T06:51:16.190628+0000","last_became_active":"2024-03-07T06:51:15.209034+0000","last_became_peered":"2024-03-07T06:51:15.209034+0000","last_unstale":"2024-03-07T06:51:16.190628+0000","last_undegraded":"2024-03-07T06:51:16.190628+0000","last_fullsized":"2024-03-07T06:51:16.190628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:31:37.292017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193770+0000","last_change":"2024-03-07T06:51:15.196320+0000","last_active":"2024-03-07T06:51:16.193770+0000","last_peered":"2024-03-07T06:51:16.193770+0000","last_clean":"2024-03-07T06:51:16.193770+0000","last_became_active":"2024-03-07T06:51:15.196029+0000","last_became_peered":"2024-03-07T06:51:15.196029+0000","last_unstale":"2024-03-07T06:51:16.193770+0000","last_undegraded":"2024-03-07T06:51:16.193770+0000","last_fullsized":"2024-03-07T06:51:16.193770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192627+0000","last_change":"2024-03-07T06:51:15.212042+0000","last_active":"2024-03-07T06:51:16.192627+0000","last_peered":"2024-03-07T06:51:16.192627+0000","last_clean":"2024-03-07T06:51:16.192627+0000","last_became_active":"2024-03-07T06:51:15.211737+0000","last_became_peered":"2024-03-07T06:51:15.211737+0000","last_unstale":"2024-03-07T06:51:16.192627+0000","last_undegraded":"2024-03-07T06:51:16.192627+0000","last_fullsized":"2024-03-07T06:51:16.192627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192585+0000","last_change":"2024-03-07T06:51:15.200183+0000","last_active":"2024-03-07T06:51:16.192585+0000","last_peered":"2024-03-07T06:51:16.192585+0000","last_clean":"2024-03-07T06:51:16.192585+0000","last_became_active":"2024-03-07T06:51:15.199322+0000","last_became_peered":"2024-03-07T06:51:15.199322+0000","last_unstale":"2024-03-07T06:51:16.192585+0000","last_undegraded":"2024-03-07T06:51:16.192585+0000","last_fullsized":"2024-03-07T06:51:16.192585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:11.295266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415528+0000","last_change":"2024-03-07T06:51:15.201141+0000","last_active":"2024-03-07T06:51:17.415528+0000","last_peered":"2024-03-07T06:51:17.415528+0000","last_clean":"2024-03-07T06:51:17.415528+0000","last_became_active":"2024-03-07T06:51:15.200965+0000","last_became_peered":"2024-03-07T06:51:15.200965+0000","last_unstale":"2024-03-07T06:51:17.415528+0000","last_undegraded":"2024-03-07T06:51:17.415528+0000","last_fullsized":"2024-03-07T06:51:17.415528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412986+0000","last_change":"2024-03-07T06:51:15.191043+0000","last_active":"2024-03-07T06:51:17.412986+0000","last_peered":"2024-03-07T06:51:17.412986+0000","last_clean":"2024-03-07T06:51:17.412986+0000","last_became_active":"2024-03-07T06:51:15.190777+0000","last_became_peered":"2024-03-07T06:51:15.190777+0000","last_unstale":"2024-03-07T06:51:17.412986+0000","last_undegraded":"2024-03-07T06:51:17.412986+0000","last_fullsized":"2024-03-07T06:51:17.412986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205186+0000","last_change":"2024-03-07T06:51:15.178908+0000","last_active":"2024-03-07T06:51:16.205186+0000","last_peered":"2024-03-07T06:51:16.205186+0000","last_clean":"2024-03-07T06:51:16.205186+0000","last_became_active":"2024-03-07T06:51:15.178387+0000","last_became_peered":"2024-03-07T06:51:15.178387+0000","last_unstale":"2024-03-07T06:51:16.205186+0000","last_undegraded":"2024-03-07T06:51:16.205186+0000","last_fullsized":"2024-03-07T06:51:16.205186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414392+0000","last_change":"2024-03-07T06:51:15.195007+0000","last_active":"2024-03-07T06:51:17.414392+0000","last_peered":"2024-03-07T06:51:17.414392+0000","last_clean":"2024-03-07T06:51:17.414392+0000","last_became_active":"2024-03-07T06:51:15.194624+0000","last_became_peered":"2024-03-07T06:51:15.194624+0000","last_unstale":"2024-03-07T06:51:17.414392+0000","last_undegraded":"2024-03-07T06:51:17.414392+0000","last_fullsized":"2024-03-07T06:51:17.414392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.416108+0000","last_change":"2024-03-07T06:51:15.206405+0000","last_active":"2024-03-07T06:51:17.416108+0000","last_peered":"2024-03-07T06:51:17.416108+0000","last_clean":"2024-03-07T06:51:17.416108+0000","last_became_active":"2024-03-07T06:51:15.205493+0000","last_became_peered":"2024-03-07T06:51:15.205493+0000","last_unstale":"2024-03-07T06:51:17.416108+0000","last_undegraded":"2024-03-07T06:51:17.416108+0000","last_fullsized":"2024-03-07T06:51:17.416108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:43:58.927044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413141+0000","last_change":"2024-03-07T06:51:15.195307+0000","last_active":"2024-03-07T06:51:17.413141+0000","last_peered":"2024-03-07T06:51:17.413141+0000","last_clean":"2024-03-07T06:51:17.413141+0000","last_became_active":"2024-03-07T06:51:15.194990+0000","last_became_peered":"2024-03-07T06:51:15.194990+0000","last_unstale":"2024-03-07T06:51:17.413141+0000","last_undegraded":"2024-03-07T06:51:17.413141+0000","last_fullsized":"2024-03-07T06:51:17.413141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:11.295266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189287+0000","last_change":"2024-03-07T06:51:15.197914+0000","last_active":"2024-03-07T06:51:16.189287+0000","last_peered":"2024-03-07T06:51:16.189287+0000","last_clean":"2024-03-07T06:51:16.189287+0000","last_became_active":"2024-03-07T06:51:15.196938+0000","last_became_peered":"2024-03-07T06:51:15.196938+0000","last_unstale":"2024-03-07T06:51:16.189287+0000","last_undegraded":"2024-03-07T06:51:16.189287+0000","last_fullsized":"2024-03-07T06:51:16.189287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:56.465649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192765+0000","last_change":"2024-03-07T06:51:15.199775+0000","last_active":"2024-03-07T06:51:16.192765+0000","last_peered":"2024-03-07T06:51:16.192765+0000","last_clean":"2024-03-07T06:51:16.192765+0000","last_became_active":"2024-03-07T06:51:15.199447+0000","last_became_peered":"2024-03-07T06:51:15.199447+0000","last_unstale":"2024-03-07T06:51:16.192765+0000","last_undegraded":"2024-03-07T06:51:16.192765+0000","last_fullsized":"2024-03-07T06:51:16.192765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413641+0000","last_change":"2024-03-07T06:51:15.191968+0000","last_active":"2024-03-07T06:51:17.413641+0000","last_peered":"2024-03-07T06:51:17.413641+0000","last_clean":"2024-03-07T06:51:17.413641+0000","last_became_active":"2024-03-07T06:51:15.191587+0000","last_became_peered":"2024-03-07T06:51:15.191587+0000","last_unstale":"2024-03-07T06:51:17.413641+0000","last_undegraded":"2024-03-07T06:51:17.413641+0000","last_fullsized":"2024-03-07T06:51:17.413641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415294+0000","last_change":"2024-03-07T06:51:15.206834+0000","last_active":"2024-03-07T06:51:17.415294+0000","last_peered":"2024-03-07T06:51:17.415294+0000","last_clean":"2024-03-07T06:51:17.415294+0000","last_became_active":"2024-03-07T06:51:15.198815+0000","last_became_peered":"2024-03-07T06:51:15.198815+0000","last_unstale":"2024-03-07T06:51:17.415294+0000","last_undegraded":"2024-03-07T06:51:17.415294+0000","last_fullsized":"2024-03-07T06:51:17.415294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415387+0000","last_change":"2024-03-07T06:51:15.203302+0000","last_active":"2024-03-07T06:51:17.415387+0000","last_peered":"2024-03-07T06:51:17.415387+0000","last_clean":"2024-03-07T06:51:17.415387+0000","last_became_active":"2024-03-07T06:51:15.203041+0000","last_became_peered":"2024-03-07T06:51:15.203041+0000","last_unstale":"2024-03-07T06:51:17.415387+0000","last_undegraded":"2024-03-07T06:51:17.415387+0000","last_fullsized":"2024-03-07T06:51:17.415387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414478+0000","last_change":"2024-03-07T06:51:15.195865+0000","last_active":"2024-03-07T06:51:17.414478+0000","last_peered":"2024-03-07T06:51:17.414478+0000","last_clean":"2024-03-07T06:51:17.414478+0000","last_became_active":"2024-03-07T06:51:15.194043+0000","last_became_peered":"2024-03-07T06:51:15.194043+0000","last_unstale":"2024-03-07T06:51:17.414478+0000","last_undegraded":"2024-03-07T06:51:17.414478+0000","last_fullsized":"2024-03-07T06:51:17.414478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415922+0000","last_change":"2024-03-07T06:51:15.203857+0000","last_active":"2024-03-07T06:51:17.415922+0000","last_peered":"2024-03-07T06:51:17.415922+0000","last_clean":"2024-03-07T06:51:17.415922+0000","last_became_active":"2024-03-07T06:51:15.203562+0000","last_became_peered":"2024-03-07T06:51:15.203562+0000","last_unstale":"2024-03-07T06:51:17.415922+0000","last_undegraded":"2024-03-07T06:51:17.415922+0000","last_fullsized":"2024-03-07T06:51:17.415922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:34:40.365080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193687+0000","last_change":"2024-03-07T06:51:15.210400+0000","last_active":"2024-03-07T06:51:16.193687+0000","last_peered":"2024-03-07T06:51:16.193687+0000","last_clean":"2024-03-07T06:51:16.193687+0000","last_became_active":"2024-03-07T06:51:15.209692+0000","last_became_peered":"2024-03-07T06:51:15.209692+0000","last_unstale":"2024-03-07T06:51:16.193687+0000","last_undegraded":"2024-03-07T06:51:16.193687+0000","last_fullsized":"2024-03-07T06:51:16.193687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205792+0000","last_change":"2024-03-07T06:51:15.190106+0000","last_active":"2024-03-07T06:51:16.205792+0000","last_peered":"2024-03-07T06:51:16.205792+0000","last_clean":"2024-03-07T06:51:16.205792+0000","last_became_active":"2024-03-07T06:51:15.189498+0000","last_became_peered":"2024-03-07T06:51:15.189498+0000","last_unstale":"2024-03-07T06:51:16.205792+0000","last_undegraded":"2024-03-07T06:51:16.205792+0000","last_fullsized":"2024-03-07T06:51:16.205792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415671+0000","last_change":"2024-03-07T06:51:15.203374+0000","last_active":"2024-03-07T06:51:17.415671+0000","last_peered":"2024-03-07T06:51:17.415671+0000","last_clean":"2024-03-07T06:51:17.415671+0000","last_became_active":"2024-03-07T06:51:15.202849+0000","last_became_peered":"2024-03-07T06:51:15.202849+0000","last_unstale":"2024-03-07T06:51:17.415671+0000","last_undegraded":"2024-03-07T06:51:17.415671+0000","last_fullsized":"2024-03-07T06:51:17.415671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413542+0000","last_change":"2024-03-07T06:51:15.200244+0000","last_active":"2024-03-07T06:51:17.413542+0000","last_peered":"2024-03-07T06:51:17.413542+0000","last_clean":"2024-03-07T06:51:17.413542+0000","last_became_active":"2024-03-07T06:51:15.198483+0000","last_became_peered":"2024-03-07T06:51:15.198483+0000","last_unstale":"2024-03-07T06:51:17.413542+0000","last_undegraded":"2024-03-07T06:51:17.413542+0000","last_fullsized":"2024-03-07T06:51:17.413542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415577+0000","last_change":"2024-03-07T06:51:15.195170+0000","last_active":"2024-03-07T06:51:17.415577+0000","last_peered":"2024-03-07T06:51:17.415577+0000","last_clean":"2024-03-07T06:51:17.415577+0000","last_became_active":"2024-03-07T06:51:15.194900+0000","last_became_peered":"2024-03-07T06:51:15.194900+0000","last_unstale":"2024-03-07T06:51:17.415577+0000","last_undegraded":"2024-03-07T06:51:17.415577+0000","last_fullsized":"2024-03-07T06:51:17.415577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:44:23.301530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414305+0000","last_change":"2024-03-07T06:51:15.197411+0000","last_active":"2024-03-07T06:51:17.414305+0000","last_peered":"2024-03-07T06:51:17.414305+0000","last_clean":"2024-03-07T06:51:17.414305+0000","last_became_active":"2024-03-07T06:51:15.194699+0000","last_became_peered":"2024-03-07T06:51:15.194699+0000","last_unstale":"2024-03-07T06:51:17.414305+0000","last_undegraded":"2024-03-07T06:51:17.414305+0000","last_fullsized":"2024-03-07T06:51:17.414305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:43.574428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.195220+0000","last_change":"2024-03-07T06:51:15.201331+0000","last_active":"2024-03-07T06:51:16.195220+0000","last_peered":"2024-03-07T06:51:16.195220+0000","last_clean":"2024-03-07T06:51:16.195220+0000","last_became_active":"2024-03-07T06:51:15.200758+0000","last_became_peered":"2024-03-07T06:51:15.200758+0000","last_unstale":"2024-03-07T06:51:16.195220+0000","last_undegraded":"2024-03-07T06:51:16.195220+0000","last_fullsized":"2024-03-07T06:51:16.195220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192348+0000","last_change":"2024-03-07T06:51:15.180977+0000","last_active":"2024-03-07T06:51:16.192348+0000","last_peered":"2024-03-07T06:51:16.192348+0000","last_clean":"2024-03-07T06:51:16.192348+0000","last_became_active":"2024-03-07T06:51:15.180359+0000","last_became_peered":"2024-03-07T06:51:15.180359+0000","last_unstale":"2024-03-07T06:51:16.192348+0000","last_undegraded":"2024-03-07T06:51:16.192348+0000","last_fullsized":"2024-03-07T06:51:16.192348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:10.776380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415597+0000","last_change":"2024-03-07T06:51:14.171537+0000","last_active":"2024-03-07T06:51:17.415597+0000","last_peered":"2024-03-07T06:51:17.415597+0000","last_clean":"2024-03-07T06:51:17.415597+0000","last_became_active":"2024-03-07T06:51:14.171105+0000","last_became_peered":"2024-03-07T06:51:14.171105+0000","last_unstale":"2024-03-07T06:51:17.415597+0000","last_undegraded":"2024-03-07T06:51:17.415597+0000","last_fullsized":"2024-03-07T06:51:17.415597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413743+0000","last_change":"2024-03-07T06:51:15.198729+0000","last_active":"2024-03-07T06:51:17.413743+0000","last_peered":"2024-03-07T06:51:17.413743+0000","last_clean":"2024-03-07T06:51:17.413743+0000","last_became_active":"2024-03-07T06:51:15.198412+0000","last_became_peered":"2024-03-07T06:51:15.198412+0000","last_unstale":"2024-03-07T06:51:17.413743+0000","last_undegraded":"2024-03-07T06:51:17.413743+0000","last_fullsized":"2024-03-07T06:51:17.413743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192432+0000","last_change":"2024-03-07T06:51:14.172967+0000","last_active":"2024-03-07T06:51:16.192432+0000","last_peered":"2024-03-07T06:51:16.192432+0000","last_clean":"2024-03-07T06:51:16.192432+0000","last_became_active":"2024-03-07T06:51:14.172706+0000","last_became_peered":"2024-03-07T06:51:14.172706+0000","last_unstale":"2024-03-07T06:51:16.192432+0000","last_undegraded":"2024-03-07T06:51:16.192432+0000","last_fullsized":"2024-03-07T06:51:16.192432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415155+0000","last_change":"2024-03-07T06:51:14.167651+0000","last_active":"2024-03-07T06:51:17.415155+0000","last_peered":"2024-03-07T06:51:17.415155+0000","last_clean":"2024-03-07T06:51:17.415155+0000","last_became_active":"2024-03-07T06:51:14.166984+0000","last_became_peered":"2024-03-07T06:51:14.166984+0000","last_unstale":"2024-03-07T06:51:17.415155+0000","last_undegraded":"2024-03-07T06:51:17.415155+0000","last_fullsized":"2024-03-07T06:51:17.415155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415067+0000","last_change":"2024-03-07T06:51:15.202906+0000","last_active":"2024-03-07T06:51:17.415067+0000","last_peered":"2024-03-07T06:51:17.415067+0000","last_clean":"2024-03-07T06:51:17.415067+0000","last_became_active":"2024-03-07T06:51:15.200219+0000","last_became_peered":"2024-03-07T06:51:15.200219+0000","last_unstale":"2024-03-07T06:51:17.415067+0000","last_undegraded":"2024-03-07T06:51:17.415067+0000","last_fullsized":"2024-03-07T06:51:17.415067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414528+0000","last_change":"2024-03-07T06:51:14.175653+0000","last_active":"2024-03-07T06:51:17.414528+0000","last_peered":"2024-03-07T06:51:17.414528+0000","last_clean":"2024-03-07T06:51:17.414528+0000","last_became_active":"2024-03-07T06:51:14.175295+0000","last_became_peered":"2024-03-07T06:51:14.175295+0000","last_unstale":"2024-03-07T06:51:17.414528+0000","last_undegraded":"2024-03-07T06:51:17.414528+0000","last_fullsized":"2024-03-07T06:51:17.414528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205033+0000","last_change":"2024-03-07T06:51:15.190580+0000","last_active":"2024-03-07T06:51:16.205033+0000","last_peered":"2024-03-07T06:51:16.205033+0000","last_clean":"2024-03-07T06:51:16.205033+0000","last_became_active":"2024-03-07T06:51:15.189190+0000","last_became_peered":"2024-03-07T06:51:15.189190+0000","last_unstale":"2024-03-07T06:51:16.205033+0000","last_undegraded":"2024-03-07T06:51:16.205033+0000","last_fullsized":"2024-03-07T06:51:16.205033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193565+0000","last_change":"2024-03-07T06:51:14.173746+0000","last_active":"2024-03-07T06:51:16.193565+0000","last_peered":"2024-03-07T06:51:16.193565+0000","last_clean":"2024-03-07T06:51:16.193565+0000","last_became_active":"2024-03-07T06:51:14.173285+0000","last_became_peered":"2024-03-07T06:51:14.173285+0000","last_unstale":"2024-03-07T06:51:16.193565+0000","last_undegraded":"2024-03-07T06:51:16.193565+0000","last_fullsized":"2024-03-07T06:51:16.193565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415198+0000","last_change":"2024-03-07T06:51:15.215525+0000","last_active":"2024-03-07T06:51:17.415198+0000","last_peered":"2024-03-07T06:51:17.415198+0000","last_clean":"2024-03-07T06:51:17.415198+0000","last_became_active":"2024-03-07T06:51:15.215050+0000","last_became_peered":"2024-03-07T06:51:15.215050+0000","last_unstale":"2024-03-07T06:51:17.415198+0000","last_undegraded":"2024-03-07T06:51:17.415198+0000","last_fullsized":"2024-03-07T06:51:17.415198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:22:38.973224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414959+0000","last_change":"2024-03-07T06:51:14.168633+0000","last_active":"2024-03-07T06:51:17.414959+0000","last_peered":"2024-03-07T06:51:17.414959+0000","last_clean":"2024-03-07T06:51:17.414959+0000","last_became_active":"2024-03-07T06:51:14.168293+0000","last_became_peered":"2024-03-07T06:51:14.168293+0000","last_unstale":"2024-03-07T06:51:17.414959+0000","last_undegraded":"2024-03-07T06:51:17.414959+0000","last_fullsized":"2024-03-07T06:51:17.414959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412761+0000","last_change":"2024-03-07T06:51:15.208732+0000","last_active":"2024-03-07T06:51:17.412761+0000","last_peered":"2024-03-07T06:51:17.412761+0000","last_clean":"2024-03-07T06:51:17.412761+0000","last_became_active":"2024-03-07T06:51:15.208472+0000","last_became_peered":"2024-03-07T06:51:15.208472+0000","last_unstale":"2024-03-07T06:51:17.412761+0000","last_undegraded":"2024-03-07T06:51:17.412761+0000","last_fullsized":"2024-03-07T06:51:17.412761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205600+0000","last_change":"2024-03-07T06:51:14.171631+0000","last_active":"2024-03-07T06:51:16.205600+0000","last_peered":"2024-03-07T06:51:16.205600+0000","last_clean":"2024-03-07T06:51:16.205600+0000","last_became_active":"2024-03-07T06:51:14.171058+0000","last_became_peered":"2024-03-07T06:51:14.171058+0000","last_unstale":"2024-03-07T06:51:16.205600+0000","last_undegraded":"2024-03-07T06:51:16.205600+0000","last_fullsized":"2024-03-07T06:51:16.205600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193446+0000","last_change":"2024-03-07T06:51:15.195194+0000","last_active":"2024-03-07T06:51:16.193446+0000","last_peered":"2024-03-07T06:51:16.193446+0000","last_clean":"2024-03-07T06:51:16.193446+0000","last_became_active":"2024-03-07T06:51:15.191103+0000","last_became_peered":"2024-03-07T06:51:15.191103+0000","last_unstale":"2024-03-07T06:51:16.193446+0000","last_undegraded":"2024-03-07T06:51:16.193446+0000","last_fullsized":"2024-03-07T06:51:16.193446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:40:51.935140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192876+0000","last_change":"2024-03-07T06:51:14.166640+0000","last_active":"2024-03-07T06:51:16.192876+0000","last_peered":"2024-03-07T06:51:16.192876+0000","last_clean":"2024-03-07T06:51:16.192876+0000","last_became_active":"2024-03-07T06:51:14.166329+0000","last_became_peered":"2024-03-07T06:51:14.166329+0000","last_unstale":"2024-03-07T06:51:16.192876+0000","last_undegraded":"2024-03-07T06:51:16.192876+0000","last_fullsized":"2024-03-07T06:51:16.192876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188809+0000","last_change":"2024-03-07T06:51:15.193638+0000","last_active":"2024-03-07T06:51:16.188809+0000","last_peered":"2024-03-07T06:51:16.188809+0000","last_clean":"2024-03-07T06:51:16.188809+0000","last_became_active":"2024-03-07T06:51:15.193354+0000","last_became_peered":"2024-03-07T06:51:15.193354+0000","last_unstale":"2024-03-07T06:51:16.188809+0000","last_undegraded":"2024-03-07T06:51:16.188809+0000","last_fullsized":"2024-03-07T06:51:16.188809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413807+0000","last_change":"2024-03-07T06:51:14.167158+0000","last_active":"2024-03-07T06:51:17.413807+0000","last_peered":"2024-03-07T06:51:17.413807+0000","last_clean":"2024-03-07T06:51:17.413807+0000","last_became_active":"2024-03-07T06:51:14.166835+0000","last_became_peered":"2024-03-07T06:51:14.166835+0000","last_unstale":"2024-03-07T06:51:17.413807+0000","last_undegraded":"2024-03-07T06:51:17.413807+0000","last_fullsized":"2024-03-07T06:51:17.413807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:38:27.101103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413891+0000","last_change":"2024-03-07T06:51:15.199375+0000","last_active":"2024-03-07T06:51:17.413891+0000","last_peered":"2024-03-07T06:51:17.413891+0000","last_clean":"2024-03-07T06:51:17.413891+0000","last_became_active":"2024-03-07T06:51:15.199060+0000","last_became_peered":"2024-03-07T06:51:15.199060+0000","last_unstale":"2024-03-07T06:51:17.413891+0000","last_undegraded":"2024-03-07T06:51:17.413891+0000","last_fullsized":"2024-03-07T06:51:17.413891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415423+0000","last_change":"2024-03-07T06:51:14.170514+0000","last_active":"2024-03-07T06:51:17.415423+0000","last_peered":"2024-03-07T06:51:17.415423+0000","last_clean":"2024-03-07T06:51:17.415423+0000","last_became_active":"2024-03-07T06:51:14.169542+0000","last_became_peered":"2024-03-07T06:51:14.169542+0000","last_unstale":"2024-03-07T06:51:17.415423+0000","last_undegraded":"2024-03-07T06:51:17.415423+0000","last_fullsized":"2024-03-07T06:51:17.415423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191234+0000","last_change":"2024-03-07T06:51:15.208081+0000","last_active":"2024-03-07T06:51:16.191234+0000","last_peered":"2024-03-07T06:51:16.191234+0000","last_clean":"2024-03-07T06:51:16.191234+0000","last_became_active":"2024-03-07T06:51:15.207758+0000","last_became_peered":"2024-03-07T06:51:15.207758+0000","last_unstale":"2024-03-07T06:51:16.191234+0000","last_undegraded":"2024-03-07T06:51:16.191234+0000","last_fullsized":"2024-03-07T06:51:16.191234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:43:16.081728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191798+0000","last_change":"2024-03-07T06:51:14.168470+0000","last_active":"2024-03-07T06:51:16.191798+0000","last_peered":"2024-03-07T06:51:16.191798+0000","last_clean":"2024-03-07T06:51:16.191798+0000","last_became_active":"2024-03-07T06:51:14.168190+0000","last_became_peered":"2024-03-07T06:51:14.168190+0000","last_unstale":"2024-03-07T06:51:16.191798+0000","last_undegraded":"2024-03-07T06:51:16.191798+0000","last_fullsized":"2024-03-07T06:51:16.191798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192778+0000","last_change":"2024-03-07T06:51:15.201105+0000","last_active":"2024-03-07T06:51:16.192778+0000","last_peered":"2024-03-07T06:51:16.192778+0000","last_clean":"2024-03-07T06:51:16.192778+0000","last_became_active":"2024-03-07T06:51:15.200404+0000","last_became_peered":"2024-03-07T06:51:15.200404+0000","last_unstale":"2024-03-07T06:51:16.192778+0000","last_undegraded":"2024-03-07T06:51:16.192778+0000","last_fullsized":"2024-03-07T06:51:16.192778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412017+0000","last_change":"2024-03-07T06:51:14.154100+0000","last_active":"2024-03-07T06:51:17.412017+0000","last_peered":"2024-03-07T06:51:17.412017+0000","last_clean":"2024-03-07T06:51:17.412017+0000","last_became_active":"2024-03-07T06:51:14.152648+0000","last_became_peered":"2024-03-07T06:51:14.152648+0000","last_unstale":"2024-03-07T06:51:17.412017+0000","last_undegraded":"2024-03-07T06:51:17.412017+0000","last_fullsized":"2024-03-07T06:51:17.412017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189886+0000","last_change":"2024-03-07T06:51:15.203942+0000","last_active":"2024-03-07T06:51:16.189886+0000","last_peered":"2024-03-07T06:51:16.189886+0000","last_clean":"2024-03-07T06:51:16.189886+0000","last_became_active":"2024-03-07T06:51:15.203671+0000","last_became_peered":"2024-03-07T06:51:15.203671+0000","last_unstale":"2024-03-07T06:51:16.189886+0000","last_undegraded":"2024-03-07T06:51:16.189886+0000","last_fullsized":"2024-03-07T06:51:16.189886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:02:55.966834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193630+0000","last_change":"2024-03-07T06:51:14.164166+0000","last_active":"2024-03-07T06:51:16.193630+0000","last_peered":"2024-03-07T06:51:16.193630+0000","last_clean":"2024-03-07T06:51:16.193630+0000","last_became_active":"2024-03-07T06:51:14.163821+0000","last_became_peered":"2024-03-07T06:51:14.163821+0000","last_unstale":"2024-03-07T06:51:16.193630+0000","last_undegraded":"2024-03-07T06:51:16.193630+0000","last_fullsized":"2024-03-07T06:51:16.193630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:38:27.101103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190408+0000","last_change":"2024-03-07T06:51:15.207392+0000","last_active":"2024-03-07T06:51:16.190408+0000","last_peered":"2024-03-07T06:51:16.190408+0000","last_clean":"2024-03-07T06:51:16.190408+0000","last_became_active":"2024-03-07T06:51:15.203049+0000","last_became_peered":"2024-03-07T06:51:15.203049+0000","last_unstale":"2024-03-07T06:51:16.190408+0000","last_undegraded":"2024-03-07T06:51:16.190408+0000","last_fullsized":"2024-03-07T06:51:16.190408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:43:58.927044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413191+0000","last_change":"2024-03-07T06:51:14.162546+0000","last_active":"2024-03-07T06:51:17.413191+0000","last_peered":"2024-03-07T06:51:17.413191+0000","last_clean":"2024-03-07T06:51:17.413191+0000","last_became_active":"2024-03-07T06:51:14.161082+0000","last_became_peered":"2024-03-07T06:51:14.161082+0000","last_unstale":"2024-03-07T06:51:17.413191+0000","last_undegraded":"2024-03-07T06:51:17.413191+0000","last_fullsized":"2024-03-07T06:51:17.413191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190725+0000","last_change":"2024-03-07T06:51:15.208036+0000","last_active":"2024-03-07T06:51:16.190725+0000","last_peered":"2024-03-07T06:51:16.190725+0000","last_clean":"2024-03-07T06:51:16.190725+0000","last_became_active":"2024-03-07T06:51:15.206837+0000","last_became_peered":"2024-03-07T06:51:15.206837+0000","last_unstale":"2024-03-07T06:51:16.190725+0000","last_undegraded":"2024-03-07T06:51:16.190725+0000","last_fullsized":"2024-03-07T06:51:16.190725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:55:33.816984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205890+0000","last_change":"2024-03-07T06:51:14.168546+0000","last_active":"2024-03-07T06:51:16.205890+0000","last_peered":"2024-03-07T06:51:16.205890+0000","last_clean":"2024-03-07T06:51:16.205890+0000","last_became_active":"2024-03-07T06:51:14.168238+0000","last_became_peered":"2024-03-07T06:51:14.168238+0000","last_unstale":"2024-03-07T06:51:16.205890+0000","last_undegraded":"2024-03-07T06:51:16.205890+0000","last_fullsized":"2024-03-07T06:51:16.205890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415347+0000","last_change":"2024-03-07T06:51:15.204629+0000","last_active":"2024-03-07T06:51:17.415347+0000","last_peered":"2024-03-07T06:51:17.415347+0000","last_clean":"2024-03-07T06:51:17.415347+0000","last_became_active":"2024-03-07T06:51:15.203615+0000","last_became_peered":"2024-03-07T06:51:15.203615+0000","last_unstale":"2024-03-07T06:51:17.415347+0000","last_undegraded":"2024-03-07T06:51:17.415347+0000","last_fullsized":"2024-03-07T06:51:17.415347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:38:28.416311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412767+0000","last_change":"2024-03-07T06:51:14.170179+0000","last_active":"2024-03-07T06:51:17.412767+0000","last_peered":"2024-03-07T06:51:17.412767+0000","last_clean":"2024-03-07T06:51:17.412767+0000","last_became_active":"2024-03-07T06:51:14.169797+0000","last_became_peered":"2024-03-07T06:51:14.169797+0000","last_unstale":"2024-03-07T06:51:17.412767+0000","last_undegraded":"2024-03-07T06:51:17.412767+0000","last_fullsized":"2024-03-07T06:51:17.412767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188955+0000","last_change":"2024-03-07T06:51:15.212961+0000","last_active":"2024-03-07T06:51:16.188955+0000","last_peered":"2024-03-07T06:51:16.188955+0000","last_clean":"2024-03-07T06:51:16.188955+0000","last_became_active":"2024-03-07T06:51:15.212691+0000","last_became_peered":"2024-03-07T06:51:15.212691+0000","last_unstale":"2024-03-07T06:51:16.188955+0000","last_undegraded":"2024-03-07T06:51:16.188955+0000","last_fullsized":"2024-03-07T06:51:16.188955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:38:28.416311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188951+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:51:16.188951+0000","last_peered":"2024-03-07T06:51:16.188951+0000","last_clean":"2024-03-07T06:51:16.188951+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:51:16.188951+0000","last_undegraded":"2024-03-07T06:51:16.188951+0000","last_fullsized":"2024-03-07T06:51:16.188951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414355+0000","last_change":"2024-03-07T06:51:14.172051+0000","last_active":"2024-03-07T06:51:17.414355+0000","last_peered":"2024-03-07T06:51:17.414355+0000","last_clean":"2024-03-07T06:51:17.414355+0000","last_became_active":"2024-03-07T06:51:14.170736+0000","last_became_peered":"2024-03-07T06:51:14.170736+0000","last_unstale":"2024-03-07T06:51:17.414355+0000","last_undegraded":"2024-03-07T06:51:17.414355+0000","last_fullsized":"2024-03-07T06:51:17.414355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415537+0000","last_change":"2024-03-07T06:51:15.208661+0000","last_active":"2024-03-07T06:51:17.415537+0000","last_peered":"2024-03-07T06:51:17.415537+0000","last_clean":"2024-03-07T06:51:17.415537+0000","last_became_active":"2024-03-07T06:51:15.208394+0000","last_became_peered":"2024-03-07T06:51:15.208394+0000","last_unstale":"2024-03-07T06:51:17.415537+0000","last_undegraded":"2024-03-07T06:51:17.415537+0000","last_fullsized":"2024-03-07T06:51:17.415537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188640+0000","last_change":"2024-03-07T06:51:14.163185+0000","last_active":"2024-03-07T06:51:16.188640+0000","last_peered":"2024-03-07T06:51:16.188640+0000","last_clean":"2024-03-07T06:51:16.188640+0000","last_became_active":"2024-03-07T06:51:14.162494+0000","last_became_peered":"2024-03-07T06:51:14.162494+0000","last_unstale":"2024-03-07T06:51:16.188640+0000","last_undegraded":"2024-03-07T06:51:16.188640+0000","last_fullsized":"2024-03-07T06:51:16.188640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412672+0000","last_change":"2024-03-07T06:51:15.193638+0000","last_active":"2024-03-07T06:51:17.412672+0000","last_peered":"2024-03-07T06:51:17.412672+0000","last_clean":"2024-03-07T06:51:17.412672+0000","last_became_active":"2024-03-07T06:51:15.193379+0000","last_became_peered":"2024-03-07T06:51:15.193379+0000","last_unstale":"2024-03-07T06:51:17.412672+0000","last_undegraded":"2024-03-07T06:51:17.412672+0000","last_fullsized":"2024-03-07T06:51:17.412672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:16:10.967500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412927+0000","last_change":"2024-03-07T06:51:14.163793+0000","last_active":"2024-03-07T06:51:17.412927+0000","last_peered":"2024-03-07T06:51:17.412927+0000","last_clean":"2024-03-07T06:51:17.412927+0000","last_became_active":"2024-03-07T06:51:14.162059+0000","last_became_peered":"2024-03-07T06:51:14.162059+0000","last_unstale":"2024-03-07T06:51:17.412927+0000","last_undegraded":"2024-03-07T06:51:17.412927+0000","last_fullsized":"2024-03-07T06:51:17.412927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205257+0000","last_change":"2024-03-07T06:51:15.187522+0000","last_active":"2024-03-07T06:51:16.205257+0000","last_peered":"2024-03-07T06:51:16.205257+0000","last_clean":"2024-03-07T06:51:16.205257+0000","last_became_active":"2024-03-07T06:51:15.179882+0000","last_became_peered":"2024-03-07T06:51:15.179882+0000","last_unstale":"2024-03-07T06:51:16.205257+0000","last_undegraded":"2024-03-07T06:51:16.205257+0000","last_fullsized":"2024-03-07T06:51:16.205257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415629+0000","last_change":"2024-03-07T06:51:14.164766+0000","last_active":"2024-03-07T06:51:17.415629+0000","last_peered":"2024-03-07T06:51:17.415629+0000","last_clean":"2024-03-07T06:51:17.415629+0000","last_became_active":"2024-03-07T06:51:14.164397+0000","last_became_peered":"2024-03-07T06:51:14.164397+0000","last_unstale":"2024-03-07T06:51:17.415629+0000","last_undegraded":"2024-03-07T06:51:17.415629+0000","last_fullsized":"2024-03-07T06:51:17.415629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192589+0000","last_change":"2024-03-07T06:51:15.190608+0000","last_active":"2024-03-07T06:51:16.192589+0000","last_peered":"2024-03-07T06:51:16.192589+0000","last_clean":"2024-03-07T06:51:16.192589+0000","last_became_active":"2024-03-07T06:51:15.190326+0000","last_became_peered":"2024-03-07T06:51:15.190326+0000","last_unstale":"2024-03-07T06:51:16.192589+0000","last_undegraded":"2024-03-07T06:51:16.192589+0000","last_fullsized":"2024-03-07T06:51:16.192589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:43.574428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205896+0000","last_change":"2024-03-07T06:51:14.164402+0000","last_active":"2024-03-07T06:51:16.205896+0000","last_peered":"2024-03-07T06:51:16.205896+0000","last_clean":"2024-03-07T06:51:16.205896+0000","last_became_active":"2024-03-07T06:51:14.164081+0000","last_became_peered":"2024-03-07T06:51:14.164081+0000","last_unstale":"2024-03-07T06:51:16.205896+0000","last_undegraded":"2024-03-07T06:51:16.205896+0000","last_fullsized":"2024-03-07T06:51:16.205896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415903+0000","last_change":"2024-03-07T06:51:15.208063+0000","last_active":"2024-03-07T06:51:17.415903+0000","last_peered":"2024-03-07T06:51:17.415903+0000","last_clean":"2024-03-07T06:51:17.415903+0000","last_became_active":"2024-03-07T06:51:15.207782+0000","last_became_peered":"2024-03-07T06:51:15.207782+0000","last_unstale":"2024-03-07T06:51:17.415903+0000","last_undegraded":"2024-03-07T06:51:17.415903+0000","last_fullsized":"2024-03-07T06:51:17.415903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:02:55.966834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189333+0000","last_change":"2024-03-07T06:51:14.156560+0000","last_active":"2024-03-07T06:51:16.189333+0000","last_peered":"2024-03-07T06:51:16.189333+0000","last_clean":"2024-03-07T06:51:16.189333+0000","last_became_active":"2024-03-07T06:51:14.156040+0000","last_became_peered":"2024-03-07T06:51:14.156040+0000","last_unstale":"2024-03-07T06:51:16.189333+0000","last_undegraded":"2024-03-07T06:51:16.189333+0000","last_fullsized":"2024-03-07T06:51:16.189333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:07:44.222048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205750+0000","last_change":"2024-03-07T06:51:15.180806+0000","last_active":"2024-03-07T06:51:16.205750+0000","last_peered":"2024-03-07T06:51:16.205750+0000","last_clean":"2024-03-07T06:51:16.205750+0000","last_became_active":"2024-03-07T06:51:15.180402+0000","last_became_peered":"2024-03-07T06:51:15.180402+0000","last_unstale":"2024-03-07T06:51:16.205750+0000","last_undegraded":"2024-03-07T06:51:16.205750+0000","last_fullsized":"2024-03-07T06:51:16.205750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205976+0000","last_change":"2024-03-07T06:51:14.168110+0000","last_active":"2024-03-07T06:51:16.205976+0000","last_peered":"2024-03-07T06:51:16.205976+0000","last_clean":"2024-03-07T06:51:16.205976+0000","last_became_active":"2024-03-07T06:51:14.167759+0000","last_became_peered":"2024-03-07T06:51:14.167759+0000","last_unstale":"2024-03-07T06:51:16.205976+0000","last_undegraded":"2024-03-07T06:51:16.205976+0000","last_fullsized":"2024-03-07T06:51:16.205976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.416085+0000","last_change":"2024-03-07T06:51:15.199832+0000","last_active":"2024-03-07T06:51:17.416085+0000","last_peered":"2024-03-07T06:51:17.416085+0000","last_clean":"2024-03-07T06:51:17.416085+0000","last_became_active":"2024-03-07T06:51:15.199572+0000","last_became_peered":"2024-03-07T06:51:15.199572+0000","last_unstale":"2024-03-07T06:51:17.416085+0000","last_undegraded":"2024-03-07T06:51:17.416085+0000","last_fullsized":"2024-03-07T06:51:17.416085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193650+0000","last_change":"2024-03-07T06:51:14.173549+0000","last_active":"2024-03-07T06:51:16.193650+0000","last_peered":"2024-03-07T06:51:16.193650+0000","last_clean":"2024-03-07T06:51:16.193650+0000","last_became_active":"2024-03-07T06:51:14.173251+0000","last_became_peered":"2024-03-07T06:51:14.173251+0000","last_unstale":"2024-03-07T06:51:16.193650+0000","last_undegraded":"2024-03-07T06:51:16.193650+0000","last_fullsized":"2024-03-07T06:51:16.193650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413748+0000","last_change":"2024-03-07T06:51:15.195178+0000","last_active":"2024-03-07T06:51:17.413748+0000","last_peered":"2024-03-07T06:51:17.413748+0000","last_clean":"2024-03-07T06:51:17.413748+0000","last_became_active":"2024-03-07T06:51:15.194907+0000","last_became_peered":"2024-03-07T06:51:15.194907+0000","last_unstale":"2024-03-07T06:51:17.413748+0000","last_undegraded":"2024-03-07T06:51:17.413748+0000","last_fullsized":"2024-03-07T06:51:17.413748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188590+0000","last_change":"2024-03-07T06:51:14.156566+0000","last_active":"2024-03-07T06:51:16.188590+0000","last_peered":"2024-03-07T06:51:16.188590+0000","last_clean":"2024-03-07T06:51:16.188590+0000","last_became_active":"2024-03-07T06:51:14.156261+0000","last_became_peered":"2024-03-07T06:51:14.156261+0000","last_unstale":"2024-03-07T06:51:16.188590+0000","last_undegraded":"2024-03-07T06:51:16.188590+0000","last_fullsized":"2024-03-07T06:51:16.188590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193424+0000","last_change":"2024-03-07T06:51:15.183065+0000","last_active":"2024-03-07T06:51:16.193424+0000","last_peered":"2024-03-07T06:51:16.193424+0000","last_clean":"2024-03-07T06:51:16.193424+0000","last_became_active":"2024-03-07T06:51:15.182189+0000","last_became_peered":"2024-03-07T06:51:15.182189+0000","last_unstale":"2024-03-07T06:51:16.193424+0000","last_undegraded":"2024-03-07T06:51:16.193424+0000","last_fullsized":"2024-03-07T06:51:16.193424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:05:21.883987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205102+0000","last_change":"2024-03-07T06:51:14.169705+0000","last_active":"2024-03-07T06:51:16.205102+0000","last_peered":"2024-03-07T06:51:16.205102+0000","last_clean":"2024-03-07T06:51:16.205102+0000","last_became_active":"2024-03-07T06:51:14.169444+0000","last_became_peered":"2024-03-07T06:51:14.169444+0000","last_unstale":"2024-03-07T06:51:16.205102+0000","last_undegraded":"2024-03-07T06:51:16.205102+0000","last_fullsized":"2024-03-07T06:51:16.205102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415220+0000","last_change":"2024-03-07T06:51:15.197549+0000","last_active":"2024-03-07T06:51:17.415220+0000","last_peered":"2024-03-07T06:51:17.415220+0000","last_clean":"2024-03-07T06:51:17.415220+0000","last_became_active":"2024-03-07T06:51:15.195020+0000","last_became_peered":"2024-03-07T06:51:15.195020+0000","last_unstale":"2024-03-07T06:51:17.415220+0000","last_undegraded":"2024-03-07T06:51:17.415220+0000","last_fullsized":"2024-03-07T06:51:17.415220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205415+0000","last_change":"2024-03-07T06:51:14.170410+0000","last_active":"2024-03-07T06:51:16.205415+0000","last_peered":"2024-03-07T06:51:16.205415+0000","last_clean":"2024-03-07T06:51:16.205415+0000","last_became_active":"2024-03-07T06:51:14.170043+0000","last_became_peered":"2024-03-07T06:51:14.170043+0000","last_unstale":"2024-03-07T06:51:16.205415+0000","last_undegraded":"2024-03-07T06:51:16.205415+0000","last_fullsized":"2024-03-07T06:51:16.205415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415636+0000","last_change":"2024-03-07T06:51:15.193076+0000","last_active":"2024-03-07T06:51:17.415636+0000","last_peered":"2024-03-07T06:51:17.415636+0000","last_clean":"2024-03-07T06:51:17.415636+0000","last_became_active":"2024-03-07T06:51:15.192700+0000","last_became_peered":"2024-03-07T06:51:15.192700+0000","last_unstale":"2024-03-07T06:51:17.415636+0000","last_undegraded":"2024-03-07T06:51:17.415636+0000","last_fullsized":"2024-03-07T06:51:17.415636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191169+0000","last_change":"2024-03-07T06:51:14.173220+0000","last_active":"2024-03-07T06:51:16.191169+0000","last_peered":"2024-03-07T06:51:16.191169+0000","last_clean":"2024-03-07T06:51:16.191169+0000","last_became_active":"2024-03-07T06:51:14.169687+0000","last_became_peered":"2024-03-07T06:51:14.169687+0000","last_unstale":"2024-03-07T06:51:16.191169+0000","last_undegraded":"2024-03-07T06:51:16.191169+0000","last_fullsized":"2024-03-07T06:51:16.191169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:02:40.568300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192059+0000","last_change":"2024-03-07T06:51:15.205150+0000","last_active":"2024-03-07T06:51:16.192059+0000","last_peered":"2024-03-07T06:51:16.192059+0000","last_clean":"2024-03-07T06:51:16.192059+0000","last_became_active":"2024-03-07T06:51:15.204435+0000","last_became_peered":"2024-03-07T06:51:15.204435+0000","last_unstale":"2024-03-07T06:51:16.192059+0000","last_undegraded":"2024-03-07T06:51:16.192059+0000","last_fullsized":"2024-03-07T06:51:16.192059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415532+0000","last_change":"2024-03-07T06:51:15.214214+0000","last_active":"2024-03-07T06:51:17.415532+0000","last_peered":"2024-03-07T06:51:17.415532+0000","last_clean":"2024-03-07T06:51:17.415532+0000","last_became_active":"2024-03-07T06:51:15.209505+0000","last_became_peered":"2024-03-07T06:51:15.209505+0000","last_unstale":"2024-03-07T06:51:17.415532+0000","last_undegraded":"2024-03-07T06:51:17.415532+0000","last_fullsized":"2024-03-07T06:51:17.415532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:56.465649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188962+0000","last_change":"2024-03-07T06:51:14.172650+0000","last_active":"2024-03-07T06:51:16.188962+0000","last_peered":"2024-03-07T06:51:16.188962+0000","last_clean":"2024-03-07T06:51:16.188962+0000","last_became_active":"2024-03-07T06:51:14.170441+0000","last_became_peered":"2024-03-07T06:51:14.170441+0000","last_unstale":"2024-03-07T06:51:16.188962+0000","last_undegraded":"2024-03-07T06:51:16.188962+0000","last_fullsized":"2024-03-07T06:51:16.188962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:05:20.871811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415022+0000","last_change":"2024-03-07T06:51:15.204124+0000","last_active":"2024-03-07T06:51:17.415022+0000","last_peered":"2024-03-07T06:51:17.415022+0000","last_clean":"2024-03-07T06:51:17.415022+0000","last_became_active":"2024-03-07T06:51:15.203713+0000","last_became_peered":"2024-03-07T06:51:15.203713+0000","last_unstale":"2024-03-07T06:51:17.415022+0000","last_undegraded":"2024-03-07T06:51:17.415022+0000","last_fullsized":"2024-03-07T06:51:17.415022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:34:40.365080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189015+0000","last_change":"2024-03-07T06:51:14.167422+0000","last_active":"2024-03-07T06:51:16.189015+0000","last_peered":"2024-03-07T06:51:16.189015+0000","last_clean":"2024-03-07T06:51:16.189015+0000","last_became_active":"2024-03-07T06:51:14.167152+0000","last_became_peered":"2024-03-07T06:51:14.167152+0000","last_unstale":"2024-03-07T06:51:16.189015+0000","last_undegraded":"2024-03-07T06:51:16.189015+0000","last_fullsized":"2024-03-07T06:51:16.189015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:40:50.922964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412609+0000","last_change":"2024-03-07T06:51:15.210848+0000","last_active":"2024-03-07T06:51:17.412609+0000","last_peered":"2024-03-07T06:51:17.412609+0000","last_clean":"2024-03-07T06:51:17.412609+0000","last_became_active":"2024-03-07T06:51:15.210526+0000","last_became_peered":"2024-03-07T06:51:15.210526+0000","last_unstale":"2024-03-07T06:51:17.412609+0000","last_undegraded":"2024-03-07T06:51:17.412609+0000","last_fullsized":"2024-03-07T06:51:17.412609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:40:57.934235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415818+0000","last_change":"2024-03-07T06:51:14.163311+0000","last_active":"2024-03-07T06:51:17.415818+0000","last_peered":"2024-03-07T06:51:17.415818+0000","last_clean":"2024-03-07T06:51:17.415818+0000","last_became_active":"2024-03-07T06:51:14.161332+0000","last_became_peered":"2024-03-07T06:51:14.161332+0000","last_unstale":"2024-03-07T06:51:17.415818+0000","last_undegraded":"2024-03-07T06:51:17.415818+0000","last_fullsized":"2024-03-07T06:51:17.415818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415995+0000","last_change":"2024-03-07T06:51:15.200486+0000","last_active":"2024-03-07T06:51:17.415995+0000","last_peered":"2024-03-07T06:51:17.415995+0000","last_clean":"2024-03-07T06:51:17.415995+0000","last_became_active":"2024-03-07T06:51:15.200171+0000","last_became_peered":"2024-03-07T06:51:15.200171+0000","last_unstale":"2024-03-07T06:51:17.415995+0000","last_undegraded":"2024-03-07T06:51:17.415995+0000","last_fullsized":"2024-03-07T06:51:17.415995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:22:38.973224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412810+0000","last_change":"2024-03-07T06:51:14.169164+0000","last_active":"2024-03-07T06:51:17.412810+0000","last_peered":"2024-03-07T06:51:17.412810+0000","last_clean":"2024-03-07T06:51:17.412810+0000","last_became_active":"2024-03-07T06:51:14.168662+0000","last_became_peered":"2024-03-07T06:51:14.168662+0000","last_unstale":"2024-03-07T06:51:17.412810+0000","last_undegraded":"2024-03-07T06:51:17.412810+0000","last_fullsized":"2024-03-07T06:51:17.412810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:51:24.658 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:51:24.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.880+0000 7fbe6e200640 1 Processor -- start 2024-03-07T06:51:24.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.880+0000 7fbe6e200640 1 -- start start 2024-03-07T06:51:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.881+0000 7fbe6e200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe68072950 0x7fbe680cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.881+0000 7fbe6e200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbe680cdee0 0x7fbe6815c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.881+0000 7fbe6e200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe680731b0 con 0x7fbe6815fbb0 2024-03-07T06:51:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.881+0000 7fbe6e200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe68072f60 con 0x7fbe680cdee0 2024-03-07T06:51:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.881+0000 7fbe6e200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe68073490 con 0x7fbe68072950 2024-03-07T06:51:24.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.882+0000 7fbe67400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fbe6815fbb0 0x7fbe6815ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44084/0 (socket says 172.21.15.133:44084) 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.882+0000 7fbe67400640 1 -- 172.21.15.133:0/3156970428 learned_addr learned my addr 172.21.15.133:0/3156970428 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.882+0000 7fbe66a00640 1 --2- 172.21.15.133:0/3156970428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe68072950 0x7fbe680cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.882+0000 7fbe67e00640 1 --2- 172.21.15.133:0/3156970428 >> v2:172.21.15.161:3300/0 conn(0x7fbe680cdee0 0x7fbe6815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.882+0000 7fbe66000640 1 -- 172.21.15.133:0/3156970428 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068256643 0 0) 0x7fbe680731b0 con 0x7fbe6815fbb0 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.882+0000 7fbe66000640 1 -- 172.21.15.133:0/3156970428 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe4c003520 con 0x7fbe6815fbb0 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66000640 1 -- 172.21.15.133:0/3156970428 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 637573291 0 0) 0x7fbe4c003520 con 0x7fbe6815fbb0 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66000640 1 -- 172.21.15.133:0/3156970428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe68072950 msgr2=0x7fbe680cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66000640 1 --2- 172.21.15.133:0/3156970428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe68072950 0x7fbe680cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66000640 1 -- 172.21.15.133:0/3156970428 >> v2:172.21.15.161:3300/0 conn(0x7fbe680cdee0 msgr2=0x7fbe6815c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66000640 1 --2- 172.21.15.133:0/3156970428 >> v2:172.21.15.161:3300/0 conn(0x7fbe680cdee0 0x7fbe6815c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66000640 1 -- 172.21.15.133:0/3156970428 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe68072150 con 0x7fbe6815fbb0 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66000640 1 -- 172.21.15.133:0/3156970428 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fbe58001670 con 0x7fbe6815fbb0 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66000640 1 -- 172.21.15.133:0/3156970428 >> v1:172.21.15.133:6789/0 conn(0x7fbe6815fbb0 legacy=0x7fbe6815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66000640 1 --2- 172.21.15.133:0/3156970428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe4c0039b0 0x7fbe4c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66000640 1 --2- 172.21.15.133:0/3156970428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c004380 0x7fbe4c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66000640 1 --2- 172.21.15.133:0/3156970428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4c004ed0 0x7fbe4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66000640 1 -- 172.21.15.133:0/3156970428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe68072f60 con 0x7fbe4c004ed0 2024-03-07T06:51:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66000640 1 -- 172.21.15.133:0/3156970428 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe68073490 con 0x7fbe4c004380 2024-03-07T06:51:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66000640 1 -- 172.21.15.133:0/3156970428 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c00bf00 con 0x7fbe4c0039b0 2024-03-07T06:51:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe67e00640 1 --2- 172.21.15.133:0/3156970428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4c004ed0 0x7fbe4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe67400640 1 --2- 172.21.15.133:0/3156970428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe4c0039b0 0x7fbe4c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe66a00640 1 --2- 172.21.15.133:0/3156970428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c004380 0x7fbe4c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe67e00640 1 -- 172.21.15.133:0/3156970428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe4c0039b0 msgr2=0x7fbe4c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe67e00640 1 --2- 172.21.15.133:0/3156970428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe4c0039b0 0x7fbe4c003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe67e00640 1 -- 172.21.15.133:0/3156970428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c004380 msgr2=0x7fbe4c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe67e00640 1 --2- 172.21.15.133:0/3156970428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe4c004380 0x7fbe4c004860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.883+0000 7fbe67e00640 1 -- 172.21.15.133:0/3156970428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c003520 con 0x7fbe4c004ed0 2024-03-07T06:51:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.884+0000 7fbe67e00640 1 --2- 172.21.15.133:0/3156970428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4c004ed0 0x7fbe4c00b9c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009fc0 tx=0x7fbe4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c7f9a7ca923d4e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.885+0000 7fbe66000640 1 -- 172.21.15.133:0/3156970428 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe60039070 con 0x7fbe4c004ed0 2024-03-07T06:51:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.885+0000 7fbe66000640 1 -- 172.21.15.133:0/3156970428 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6002cc30 con 0x7fbe4c004ed0 2024-03-07T06:51:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.885+0000 7fbe66000640 1 -- 172.21.15.133:0/3156970428 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe60009c10 con 0x7fbe4c004ed0 2024-03-07T06:51:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.885+0000 7fbe6e200640 1 -- 172.21.15.133:0/3156970428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4c004ed0 msgr2=0x7fbe4c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.885+0000 7fbe6e200640 1 --2- 172.21.15.133:0/3156970428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4c004ed0 0x7fbe4c00b9c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009fc0 tx=0x7fbe4c005830 comp rx=0 tx=0).stop 2024-03-07T06:51:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.885+0000 7fbe67e00640 1 -- 172.21.15.133:0/3156970428 reap_dead start 2024-03-07T06:51:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.885+0000 7fbe6e200640 1 -- 172.21.15.133:0/3156970428 shutdown_connections 2024-03-07T06:51:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.885+0000 7fbe6e200640 1 -- 172.21.15.133:0/3156970428 >> 172.21.15.133:0/3156970428 conn(0x7fbe680b3120 msgr2=0x7fbe680b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.885+0000 7fbe6e200640 1 -- 172.21.15.133:0/3156970428 shutdown_connections 2024-03-07T06:51:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.885+0000 7fbe6e200640 1 -- 172.21.15.133:0/3156970428 wait complete. 2024-03-07T06:51:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.887+0000 7fbe6e200640 1 Processor -- start 2024-03-07T06:51:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.887+0000 7fbe6e200640 1 -- start start 2024-03-07T06:51:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.888+0000 7fbe6e200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4c004ed0 0x7fbe680caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:24.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.888+0000 7fbe6e200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe680cd0b0 0x7fbe680cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.888+0000 7fbe6e200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe681f3a00 0x7fbe681f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.888+0000 7fbe6e200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe68160980 con 0x7fbe4c004ed0 2024-03-07T06:51:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.888+0000 7fbe6e200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe680ce2f0 con 0x7fbe680cd0b0 2024-03-07T06:51:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.888+0000 7fbe6e200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe680b1a00 con 0x7fbe681f3a00 2024-03-07T06:51:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.888+0000 7fbe67400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4c004ed0 0x7fbe680caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.888+0000 7fbe67400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4c004ed0 0x7fbe680caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38450/0 (socket says 172.21.15.133:38450) 2024-03-07T06:51:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.888+0000 7fbe67400640 1 -- 172.21.15.133:0/782305469 learned_addr learned my addr 172.21.15.133:0/782305469 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.889+0000 7fbe66a00640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe680cd0b0 0x7fbe680cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.889+0000 7fbe67e00640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe681f3a00 0x7fbe681f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.889+0000 7fbe67400640 1 -- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe681f3a00 msgr2=0x7fbe681f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.889+0000 7fbe67400640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe681f3a00 0x7fbe681f3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.889+0000 7fbe67400640 1 -- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe680cd0b0 msgr2=0x7fbe680cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.889+0000 7fbe67400640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe680cd0b0 0x7fbe680cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.889+0000 7fbe67400640 1 -- 172.21.15.133:0/782305469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6815d510 con 0x7fbe4c004ed0 2024-03-07T06:51:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.889+0000 7fbe66a00640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe680cd0b0 0x7fbe680cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.889+0000 7fbe67400640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4c004ed0 0x7fbe680caed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800d7e0 tx=0x7fbe5800dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.889+0000 7fbe5f400640 1 -- 172.21.15.133:0/782305469 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe58019070 con 0x7fbe4c004ed0 2024-03-07T06:51:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.889+0000 7fbe5f400640 1 -- 172.21.15.133:0/782305469 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe580074e0 con 0x7fbe4c004ed0 2024-03-07T06:51:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.889+0000 7fbe5f400640 1 -- 172.21.15.133:0/782305469 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe580049e0 con 0x7fbe4c004ed0 2024-03-07T06:51:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.889+0000 7fbe6e200640 1 -- 172.21.15.133:0/782305469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6815d7f0 con 0x7fbe4c004ed0 2024-03-07T06:51:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.890+0000 7fbe6e200640 1 -- 172.21.15.133:0/782305469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6815e2d0 con 0x7fbe4c004ed0 2024-03-07T06:51:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.891+0000 7fbe6e200640 1 -- 172.21.15.133:0/782305469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe680cda70 con 0x7fbe4c004ed0 2024-03-07T06:51:24.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.892+0000 7fbe5f400640 1 -- 172.21.15.133:0/782305469 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbe58004b80 con 0x7fbe4c004ed0 2024-03-07T06:51:24.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.892+0000 7fbe5f400640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbe4806b500 0x7fbe4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:24.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.893+0000 7fbe5f400640 1 -- 172.21.15.133:0/782305469 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe58089c30 con 0x7fbe4c004ed0 2024-03-07T06:51:24.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.893+0000 7fbe66a00640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbe4806b500 0x7fbe4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:24.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.894+0000 7fbe66a00640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbe4806b500 0x7fbe4806d9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe6815eba0 tx=0x7fbe54008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:24.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:24.897+0000 7fbe5f400640 1 -- 172.21.15.133:0/782305469 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbe58058aa0 con 0x7fbe4c004ed0 2024-03-07T06:51:25.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.091+0000 7fbe6e200640 1 -- 172.21.15.133:0/782305469 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe68062c90 con 0x7fbe4806b500 2024-03-07T06:51:25.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.109+0000 7fbe5f400640 1 -- 172.21.15.133:0/782305469 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7fbe68062c90 con 0x7fbe4806b500 2024-03-07T06:51:25.111 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:25.112 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:51:25.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.114+0000 7fbe6e200640 1 -- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbe4806b500 msgr2=0x7fbe4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:25.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.114+0000 7fbe6e200640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbe4806b500 0x7fbe4806d9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbe6815eba0 tx=0x7fbe54008040 comp rx=0 tx=0).stop 2024-03-07T06:51:25.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.114+0000 7fbe6e200640 1 -- 172.21.15.133:0/782305469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4c004ed0 msgr2=0x7fbe680caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:25.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.114+0000 7fbe6e200640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4c004ed0 0x7fbe680caed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800d7e0 tx=0x7fbe5800dca0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.114+0000 7fbe6e200640 1 -- 172.21.15.133:0/782305469 shutdown_connections 2024-03-07T06:51:25.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.114+0000 7fbe6e200640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe681f3a00 0x7fbe681f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.114+0000 7fbe6e200640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbe4806b500 0x7fbe4806d9c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.114+0000 7fbe6e200640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe680cd0b0 0x7fbe680cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.114+0000 7fbe6e200640 1 --2- 172.21.15.133:0/782305469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe4c004ed0 0x7fbe680caed0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.114+0000 7fbe6e200640 1 -- 172.21.15.133:0/782305469 >> 172.21.15.133:0/782305469 conn(0x7fbe680b3120 msgr2=0x7fbe6815ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:25.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.114+0000 7fbe6e200640 1 -- 172.21.15.133:0/782305469 shutdown_connections 2024-03-07T06:51:25.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.114+0000 7fbe6e200640 1 -- 172.21.15.133:0/782305469 wait complete. 2024-03-07T06:51:25.133 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-03-07T06:51:24.472995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219088,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755632,"statfs":{"total":773094113280,"available":772869767168,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.313208"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190674+0000","last_change":"2024-03-07T06:51:15.202253+0000","last_active":"2024-03-07T06:51:16.190674+0000","last_peered":"2024-03-07T06:51:16.190674+0000","last_clean":"2024-03-07T06:51:16.190674+0000","last_became_active":"2024-03-07T06:51:15.201848+0000","last_became_peered":"2024-03-07T06:51:15.201848+0000","last_unstale":"2024-03-07T06:51:16.190674+0000","last_undegraded":"2024-03-07T06:51:16.190674+0000","last_fullsized":"2024-03-07T06:51:16.190674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:41:54.645832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205980+0000","last_change":"2024-03-07T06:51:15.188301+0000","last_active":"2024-03-07T06:51:16.205980+0000","last_peered":"2024-03-07T06:51:16.205980+0000","last_clean":"2024-03-07T06:51:16.205980+0000","last_became_active":"2024-03-07T06:51:15.188021+0000","last_became_peered":"2024-03-07T06:51:15.188021+0000","last_unstale":"2024-03-07T06:51:16.205980+0000","last_undegraded":"2024-03-07T06:51:16.205980+0000","last_fullsized":"2024-03-07T06:51:16.205980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412687+0000","last_change":"2024-03-07T06:51:15.188296+0000","last_active":"2024-03-07T06:51:17.412687+0000","last_peered":"2024-03-07T06:51:17.412687+0000","last_clean":"2024-03-07T06:51:17.412687+0000","last_became_active":"2024-03-07T06:51:15.187954+0000","last_became_peered":"2024-03-07T06:51:15.187954+0000","last_unstale":"2024-03-07T06:51:17.412687+0000","last_undegraded":"2024-03-07T06:51:17.412687+0000","last_fullsized":"2024-03-07T06:51:17.412687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414882+0000","last_change":"2024-03-07T06:51:15.200894+0000","last_active":"2024-03-07T06:51:17.414882+0000","last_peered":"2024-03-07T06:51:17.414882+0000","last_clean":"2024-03-07T06:51:17.414882+0000","last_became_active":"2024-03-07T06:51:15.200617+0000","last_became_peered":"2024-03-07T06:51:15.200617+0000","last_unstale":"2024-03-07T06:51:17.414882+0000","last_undegraded":"2024-03-07T06:51:17.414882+0000","last_fullsized":"2024-03-07T06:51:17.414882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190864+0000","last_change":"2024-03-07T06:51:15.189383+0000","last_active":"2024-03-07T06:51:16.190864+0000","last_peered":"2024-03-07T06:51:16.190864+0000","last_clean":"2024-03-07T06:51:16.190864+0000","last_became_active":"2024-03-07T06:51:15.188192+0000","last_became_peered":"2024-03-07T06:51:15.188192+0000","last_unstale":"2024-03-07T06:51:16.190864+0000","last_undegraded":"2024-03-07T06:51:16.190864+0000","last_fullsized":"2024-03-07T06:51:16.190864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:07:16.800545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414731+0000","last_change":"2024-03-07T06:51:15.195882+0000","last_active":"2024-03-07T06:51:17.414731+0000","last_peered":"2024-03-07T06:51:17.414731+0000","last_clean":"2024-03-07T06:51:17.414731+0000","last_became_active":"2024-03-07T06:51:15.195572+0000","last_became_peered":"2024-03-07T06:51:15.195572+0000","last_unstale":"2024-03-07T06:51:17.414731+0000","last_undegraded":"2024-03-07T06:51:17.414731+0000","last_fullsized":"2024-03-07T06:51:17.414731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189735+0000","last_change":"2024-03-07T06:51:15.190861+0000","last_active":"2024-03-07T06:51:16.189735+0000","last_peered":"2024-03-07T06:51:16.189735+0000","last_clean":"2024-03-07T06:51:16.189735+0000","last_became_active":"2024-03-07T06:51:15.190546+0000","last_became_peered":"2024-03-07T06:51:15.190546+0000","last_unstale":"2024-03-07T06:51:16.189735+0000","last_undegraded":"2024-03-07T06:51:16.189735+0000","last_fullsized":"2024-03-07T06:51:16.189735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:44:23.301530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415109+0000","last_change":"2024-03-07T06:51:15.202655+0000","last_active":"2024-03-07T06:51:17.415109+0000","last_peered":"2024-03-07T06:51:17.415109+0000","last_clean":"2024-03-07T06:51:17.415109+0000","last_became_active":"2024-03-07T06:51:15.197943+0000","last_became_peered":"2024-03-07T06:51:15.197943+0000","last_unstale":"2024-03-07T06:51:17.415109+0000","last_undegraded":"2024-03-07T06:51:17.415109+0000","last_fullsized":"2024-03-07T06:51:17.415109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190628+0000","last_change":"2024-03-07T06:51:15.209333+0000","last_active":"2024-03-07T06:51:16.190628+0000","last_peered":"2024-03-07T06:51:16.190628+0000","last_clean":"2024-03-07T06:51:16.190628+0000","last_became_active":"2024-03-07T06:51:15.209034+0000","last_became_peered":"2024-03-07T06:51:15.209034+0000","last_unstale":"2024-03-07T06:51:16.190628+0000","last_undegraded":"2024-03-07T06:51:16.190628+0000","last_fullsized":"2024-03-07T06:51:16.190628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:31:37.292017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193770+0000","last_change":"2024-03-07T06:51:15.196320+0000","last_active":"2024-03-07T06:51:16.193770+0000","last_peered":"2024-03-07T06:51:16.193770+0000","last_clean":"2024-03-07T06:51:16.193770+0000","last_became_active":"2024-03-07T06:51:15.196029+0000","last_became_peered":"2024-03-07T06:51:15.196029+0000","last_unstale":"2024-03-07T06:51:16.193770+0000","last_undegraded":"2024-03-07T06:51:16.193770+0000","last_fullsized":"2024-03-07T06:51:16.193770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192627+0000","last_change":"2024-03-07T06:51:15.212042+0000","last_active":"2024-03-07T06:51:16.192627+0000","last_peered":"2024-03-07T06:51:16.192627+0000","last_clean":"2024-03-07T06:51:16.192627+0000","last_became_active":"2024-03-07T06:51:15.211737+0000","last_became_peered":"2024-03-07T06:51:15.211737+0000","last_unstale":"2024-03-07T06:51:16.192627+0000","last_undegraded":"2024-03-07T06:51:16.192627+0000","last_fullsized":"2024-03-07T06:51:16.192627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192585+0000","last_change":"2024-03-07T06:51:15.200183+0000","last_active":"2024-03-07T06:51:16.192585+0000","last_peered":"2024-03-07T06:51:16.192585+0000","last_clean":"2024-03-07T06:51:16.192585+0000","last_became_active":"2024-03-07T06:51:15.199322+0000","last_became_peered":"2024-03-07T06:51:15.199322+0000","last_unstale":"2024-03-07T06:51:16.192585+0000","last_undegraded":"2024-03-07T06:51:16.192585+0000","last_fullsized":"2024-03-07T06:51:16.192585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:11.295266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415528+0000","last_change":"2024-03-07T06:51:15.201141+0000","last_active":"2024-03-07T06:51:17.415528+0000","last_peered":"2024-03-07T06:51:17.415528+0000","last_clean":"2024-03-07T06:51:17.415528+0000","last_became_active":"2024-03-07T06:51:15.200965+0000","last_became_peered":"2024-03-07T06:51:15.200965+0000","last_unstale":"2024-03-07T06:51:17.415528+0000","last_undegraded":"2024-03-07T06:51:17.415528+0000","last_fullsized":"2024-03-07T06:51:17.415528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412986+0000","last_change":"2024-03-07T06:51:15.191043+0000","last_active":"2024-03-07T06:51:17.412986+0000","last_peered":"2024-03-07T06:51:17.412986+0000","last_clean":"2024-03-07T06:51:17.412986+0000","last_became_active":"2024-03-07T06:51:15.190777+0000","last_became_peered":"2024-03-07T06:51:15.190777+0000","last_unstale":"2024-03-07T06:51:17.412986+0000","last_undegraded":"2024-03-07T06:51:17.412986+0000","last_fullsized":"2024-03-07T06:51:17.412986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205186+0000","last_change":"2024-03-07T06:51:15.178908+0000","last_active":"2024-03-07T06:51:16.205186+0000","last_peered":"2024-03-07T06:51:16.205186+0000","last_clean":"2024-03-07T06:51:16.205186+0000","last_became_active":"2024-03-07T06:51:15.178387+0000","last_became_peered":"2024-03-07T06:51:15.178387+0000","last_unstale":"2024-03-07T06:51:16.205186+0000","last_undegraded":"2024-03-07T06:51:16.205186+0000","last_fullsized":"2024-03-07T06:51:16.205186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414392+0000","last_change":"2024-03-07T06:51:15.195007+0000","last_active":"2024-03-07T06:51:17.414392+0000","last_peered":"2024-03-07T06:51:17.414392+0000","last_clean":"2024-03-07T06:51:17.414392+0000","last_became_active":"2024-03-07T06:51:15.194624+0000","last_became_peered":"2024-03-07T06:51:15.194624+0000","last_unstale":"2024-03-07T06:51:17.414392+0000","last_undegraded":"2024-03-07T06:51:17.414392+0000","last_fullsized":"2024-03-07T06:51:17.414392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.416108+0000","last_change":"2024-03-07T06:51:15.206405+0000","last_active":"2024-03-07T06:51:17.416108+0000","last_peered":"2024-03-07T06:51:17.416108+0000","last_clean":"2024-03-07T06:51:17.416108+0000","last_became_active":"2024-03-07T06:51:15.205493+0000","last_became_peered":"2024-03-07T06:51:15.205493+0000","last_unstale":"2024-03-07T06:51:17.416108+0000","last_undegraded":"2024-03-07T06:51:17.416108+0000","last_fullsized":"2024-03-07T06:51:17.416108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:43:58.927044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413141+0000","last_change":"2024-03-07T06:51:15.195307+0000","last_active":"2024-03-07T06:51:17.413141+0000","last_peered":"2024-03-07T06:51:17.413141+0000","last_clean":"2024-03-07T06:51:17.413141+0000","last_became_active":"2024-03-07T06:51:15.194990+0000","last_became_peered":"2024-03-07T06:51:15.194990+0000","last_unstale":"2024-03-07T06:51:17.413141+0000","last_undegraded":"2024-03-07T06:51:17.413141+0000","last_fullsized":"2024-03-07T06:51:17.413141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:11.295266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189287+0000","last_change":"2024-03-07T06:51:15.197914+0000","last_active":"2024-03-07T06:51:16.189287+0000","last_peered":"2024-03-07T06:51:16.189287+0000","last_clean":"2024-03-07T06:51:16.189287+0000","last_became_active":"2024-03-07T06:51:15.196938+0000","last_became_peered":"2024-03-07T06:51:15.196938+0000","last_unstale":"2024-03-07T06:51:16.189287+0000","last_undegraded":"2024-03-07T06:51:16.189287+0000","last_fullsized":"2024-03-07T06:51:16.189287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:56.465649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192765+0000","last_change":"2024-03-07T06:51:15.199775+0000","last_active":"2024-03-07T06:51:16.192765+0000","last_peered":"2024-03-07T06:51:16.192765+0000","last_clean":"2024-03-07T06:51:16.192765+0000","last_became_active":"2024-03-07T06:51:15.199447+0000","last_became_peered":"2024-03-07T06:51:15.199447+0000","last_unstale":"2024-03-07T06:51:16.192765+0000","last_undegraded":"2024-03-07T06:51:16.192765+0000","last_fullsized":"2024-03-07T06:51:16.192765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413641+0000","last_change":"2024-03-07T06:51:15.191968+0000","last_active":"2024-03-07T06:51:17.413641+0000","last_peered":"2024-03-07T06:51:17.413641+0000","last_clean":"2024-03-07T06:51:17.413641+0000","last_became_active":"2024-03-07T06:51:15.191587+0000","last_became_peered":"2024-03-07T06:51:15.191587+0000","last_unstale":"2024-03-07T06:51:17.413641+0000","last_undegraded":"2024-03-07T06:51:17.413641+0000","last_fullsized":"2024-03-07T06:51:17.413641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415294+0000","last_change":"2024-03-07T06:51:15.206834+0000","last_active":"2024-03-07T06:51:17.415294+0000","last_peered":"2024-03-07T06:51:17.415294+0000","last_clean":"2024-03-07T06:51:17.415294+0000","last_became_active":"2024-03-07T06:51:15.198815+0000","last_became_peered":"2024-03-07T06:51:15.198815+0000","last_unstale":"2024-03-07T06:51:17.415294+0000","last_undegraded":"2024-03-07T06:51:17.415294+0000","last_fullsized":"2024-03-07T06:51:17.415294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415387+0000","last_change":"2024-03-07T06:51:15.203302+0000","last_active":"2024-03-07T06:51:17.415387+0000","last_peered":"2024-03-07T06:51:17.415387+0000","last_clean":"2024-03-07T06:51:17.415387+0000","last_became_active":"2024-03-07T06:51:15.203041+0000","last_became_peered":"2024-03-07T06:51:15.203041+0000","last_unstale":"2024-03-07T06:51:17.415387+0000","last_undegraded":"2024-03-07T06:51:17.415387+0000","last_fullsized":"2024-03-07T06:51:17.415387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414478+0000","last_change":"2024-03-07T06:51:15.195865+0000","last_active":"2024-03-07T06:51:17.414478+0000","last_peered":"2024-03-07T06:51:17.414478+0000","last_clean":"2024-03-07T06:51:17.414478+0000","last_became_active":"2024-03-07T06:51:15.194043+0000","last_became_peered":"2024-03-07T06:51:15.194043+0000","last_unstale":"2024-03-07T06:51:17.414478+0000","last_undegraded":"2024-03-07T06:51:17.414478+0000","last_fullsized":"2024-03-07T06:51:17.414478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415922+0000","last_change":"2024-03-07T06:51:15.203857+0000","last_active":"2024-03-07T06:51:17.415922+0000","last_peered":"2024-03-07T06:51:17.415922+0000","last_clean":"2024-03-07T06:51:17.415922+0000","last_became_active":"2024-03-07T06:51:15.203562+0000","last_became_peered":"2024-03-07T06:51:15.203562+0000","last_unstale":"2024-03-07T06:51:17.415922+0000","last_undegraded":"2024-03-07T06:51:17.415922+0000","last_fullsized":"2024-03-07T06:51:17.415922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:34:40.365080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193687+0000","last_change":"2024-03-07T06:51:15.210400+0000","last_active":"2024-03-07T06:51:16.193687+0000","last_peered":"2024-03-07T06:51:16.193687+0000","last_clean":"2024-03-07T06:51:16.193687+0000","last_became_active":"2024-03-07T06:51:15.209692+0000","last_became_peered":"2024-03-07T06:51:15.209692+0000","last_unstale":"2024-03-07T06:51:16.193687+0000","last_undegraded":"2024-03-07T06:51:16.193687+0000","last_fullsized":"2024-03-07T06:51:16.193687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205792+0000","last_change":"2024-03-07T06:51:15.190106+0000","last_active":"2024-03-07T06:51:16.205792+0000","last_peered":"2024-03-07T06:51:16.205792+0000","last_clean":"2024-03-07T06:51:16.205792+0000","last_became_active":"2024-03-07T06:51:15.189498+0000","last_became_peered":"2024-03-07T06:51:15.189498+0000","last_unstale":"2024-03-07T06:51:16.205792+0000","last_undegraded":"2024-03-07T06:51:16.205792+0000","last_fullsized":"2024-03-07T06:51:16.205792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415671+0000","last_change":"2024-03-07T06:51:15.203374+0000","last_active":"2024-03-07T06:51:17.415671+0000","last_peered":"2024-03-07T06:51:17.415671+0000","last_clean":"2024-03-07T06:51:17.415671+0000","last_became_active":"2024-03-07T06:51:15.202849+0000","last_became_peered":"2024-03-07T06:51:15.202849+0000","last_unstale":"2024-03-07T06:51:17.415671+0000","last_undegraded":"2024-03-07T06:51:17.415671+0000","last_fullsized":"2024-03-07T06:51:17.415671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413542+0000","last_change":"2024-03-07T06:51:15.200244+0000","last_active":"2024-03-07T06:51:17.413542+0000","last_peered":"2024-03-07T06:51:17.413542+0000","last_clean":"2024-03-07T06:51:17.413542+0000","last_became_active":"2024-03-07T06:51:15.198483+0000","last_became_peered":"2024-03-07T06:51:15.198483+0000","last_unstale":"2024-03-07T06:51:17.413542+0000","last_undegraded":"2024-03-07T06:51:17.413542+0000","last_fullsized":"2024-03-07T06:51:17.413542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415577+0000","last_change":"2024-03-07T06:51:15.195170+0000","last_active":"2024-03-07T06:51:17.415577+0000","last_peered":"2024-03-07T06:51:17.415577+0000","last_clean":"2024-03-07T06:51:17.415577+0000","last_became_active":"2024-03-07T06:51:15.194900+0000","last_became_peered":"2024-03-07T06:51:15.194900+0000","last_unstale":"2024-03-07T06:51:17.415577+0000","last_undegraded":"2024-03-07T06:51:17.415577+0000","last_fullsized":"2024-03-07T06:51:17.415577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:44:23.301530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414305+0000","last_change":"2024-03-07T06:51:15.197411+0000","last_active":"2024-03-07T06:51:17.414305+0000","last_peered":"2024-03-07T06:51:17.414305+0000","last_clean":"2024-03-07T06:51:17.414305+0000","last_became_active":"2024-03-07T06:51:15.194699+0000","last_became_peered":"2024-03-07T06:51:15.194699+0000","last_unstale":"2024-03-07T06:51:17.414305+0000","last_undegraded":"2024-03-07T06:51:17.414305+0000","last_fullsized":"2024-03-07T06:51:17.414305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:43.574428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.195220+0000","last_change":"2024-03-07T06:51:15.201331+0000","last_active":"2024-03-07T06:51:16.195220+0000","last_peered":"2024-03-07T06:51:16.195220+0000","last_clean":"2024-03-07T06:51:16.195220+0000","last_became_active":"2024-03-07T06:51:15.200758+0000","last_became_peered":"2024-03-07T06:51:15.200758+0000","last_unstale":"2024-03-07T06:51:16.195220+0000","last_undegraded":"2024-03-07T06:51:16.195220+0000","last_fullsized":"2024-03-07T06:51:16.195220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192348+0000","last_change":"2024-03-07T06:51:15.180977+0000","last_active":"2024-03-07T06:51:16.192348+0000","last_peered":"2024-03-07T06:51:16.192348+0000","last_clean":"2024-03-07T06:51:16.192348+0000","last_became_active":"2024-03-07T06:51:15.180359+0000","last_became_peered":"2024-03-07T06:51:15.180359+0000","last_unstale":"2024-03-07T06:51:16.192348+0000","last_undegraded":"2024-03-07T06:51:16.192348+0000","last_fullsized":"2024-03-07T06:51:16.192348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:10.776380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415597+0000","last_change":"2024-03-07T06:51:14.171537+0000","last_active":"2024-03-07T06:51:17.415597+0000","last_peered":"2024-03-07T06:51:17.415597+0000","last_clean":"2024-03-07T06:51:17.415597+0000","last_became_active":"2024-03-07T06:51:14.171105+0000","last_became_peered":"2024-03-07T06:51:14.171105+0000","last_unstale":"2024-03-07T06:51:17.415597+0000","last_undegraded":"2024-03-07T06:51:17.415597+0000","last_fullsized":"2024-03-07T06:51:17.415597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413743+0000","last_change":"2024-03-07T06:51:15.198729+0000","last_active":"2024-03-07T06:51:17.413743+0000","last_peered":"2024-03-07T06:51:17.413743+0000","last_clean":"2024-03-07T06:51:17.413743+0000","last_became_active":"2024-03-07T06:51:15.198412+0000","last_became_peered":"2024-03-07T06:51:15.198412+0000","last_unstale":"2024-03-07T06:51:17.413743+0000","last_undegraded":"2024-03-07T06:51:17.413743+0000","last_fullsized":"2024-03-07T06:51:17.413743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192432+0000","last_change":"2024-03-07T06:51:14.172967+0000","last_active":"2024-03-07T06:51:16.192432+0000","last_peered":"2024-03-07T06:51:16.192432+0000","last_clean":"2024-03-07T06:51:16.192432+0000","last_became_active":"2024-03-07T06:51:14.172706+0000","last_became_peered":"2024-03-07T06:51:14.172706+0000","last_unstale":"2024-03-07T06:51:16.192432+0000","last_undegraded":"2024-03-07T06:51:16.192432+0000","last_fullsized":"2024-03-07T06:51:16.192432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415155+0000","last_change":"2024-03-07T06:51:14.167651+0000","last_active":"2024-03-07T06:51:17.415155+0000","last_peered":"2024-03-07T06:51:17.415155+0000","last_clean":"2024-03-07T06:51:17.415155+0000","last_became_active":"2024-03-07T06:51:14.166984+0000","last_became_peered":"2024-03-07T06:51:14.166984+0000","last_unstale":"2024-03-07T06:51:17.415155+0000","last_undegraded":"2024-03-07T06:51:17.415155+0000","last_fullsized":"2024-03-07T06:51:17.415155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415067+0000","last_change":"2024-03-07T06:51:15.202906+0000","last_active":"2024-03-07T06:51:17.415067+0000","last_peered":"2024-03-07T06:51:17.415067+0000","last_clean":"2024-03-07T06:51:17.415067+0000","last_became_active":"2024-03-07T06:51:15.200219+0000","last_became_peered":"2024-03-07T06:51:15.200219+0000","last_unstale":"2024-03-07T06:51:17.415067+0000","last_undegraded":"2024-03-07T06:51:17.415067+0000","last_fullsized":"2024-03-07T06:51:17.415067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414528+0000","last_change":"2024-03-07T06:51:14.175653+0000","last_active":"2024-03-07T06:51:17.414528+0000","last_peered":"2024-03-07T06:51:17.414528+0000","last_clean":"2024-03-07T06:51:17.414528+0000","last_became_active":"2024-03-07T06:51:14.175295+0000","last_became_peered":"2024-03-07T06:51:14.175295+0000","last_unstale":"2024-03-07T06:51:17.414528+0000","last_undegraded":"2024-03-07T06:51:17.414528+0000","last_fullsized":"2024-03-07T06:51:17.414528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205033+0000","last_change":"2024-03-07T06:51:15.190580+0000","last_active":"2024-03-07T06:51:16.205033+0000","last_peered":"2024-03-07T06:51:16.205033+0000","last_clean":"2024-03-07T06:51:16.205033+0000","last_became_active":"2024-03-07T06:51:15.189190+0000","last_became_peered":"2024-03-07T06:51:15.189190+0000","last_unstale":"2024-03-07T06:51:16.205033+0000","last_undegraded":"2024-03-07T06:51:16.205033+0000","last_fullsized":"2024-03-07T06:51:16.205033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193565+0000","last_change":"2024-03-07T06:51:14.173746+0000","last_active":"2024-03-07T06:51:16.193565+0000","last_peered":"2024-03-07T06:51:16.193565+0000","last_clean":"2024-03-07T06:51:16.193565+0000","last_became_active":"2024-03-07T06:51:14.173285+0000","last_became_peered":"2024-03-07T06:51:14.173285+0000","last_unstale":"2024-03-07T06:51:16.193565+0000","last_undegraded":"2024-03-07T06:51:16.193565+0000","last_fullsized":"2024-03-07T06:51:16.193565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415198+0000","last_change":"2024-03-07T06:51:15.215525+0000","last_active":"2024-03-07T06:51:17.415198+0000","last_peered":"2024-03-07T06:51:17.415198+0000","last_clean":"2024-03-07T06:51:17.415198+0000","last_became_active":"2024-03-07T06:51:15.215050+0000","last_became_peered":"2024-03-07T06:51:15.215050+0000","last_unstale":"2024-03-07T06:51:17.415198+0000","last_undegraded":"2024-03-07T06:51:17.415198+0000","last_fullsized":"2024-03-07T06:51:17.415198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:22:38.973224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414959+0000","last_change":"2024-03-07T06:51:14.168633+0000","last_active":"2024-03-07T06:51:17.414959+0000","last_peered":"2024-03-07T06:51:17.414959+0000","last_clean":"2024-03-07T06:51:17.414959+0000","last_became_active":"2024-03-07T06:51:14.168293+0000","last_became_peered":"2024-03-07T06:51:14.168293+0000","last_unstale":"2024-03-07T06:51:17.414959+0000","last_undegraded":"2024-03-07T06:51:17.414959+0000","last_fullsized":"2024-03-07T06:51:17.414959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412761+0000","last_change":"2024-03-07T06:51:15.208732+0000","last_active":"2024-03-07T06:51:17.412761+0000","last_peered":"2024-03-07T06:51:17.412761+0000","last_clean":"2024-03-07T06:51:17.412761+0000","last_became_active":"2024-03-07T06:51:15.208472+0000","last_became_peered":"2024-03-07T06:51:15.208472+0000","last_unstale":"2024-03-07T06:51:17.412761+0000","last_undegraded":"2024-03-07T06:51:17.412761+0000","last_fullsized":"2024-03-07T06:51:17.412761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205600+0000","last_change":"2024-03-07T06:51:14.171631+0000","last_active":"2024-03-07T06:51:16.205600+0000","last_peered":"2024-03-07T06:51:16.205600+0000","last_clean":"2024-03-07T06:51:16.205600+0000","last_became_active":"2024-03-07T06:51:14.171058+0000","last_became_peered":"2024-03-07T06:51:14.171058+0000","last_unstale":"2024-03-07T06:51:16.205600+0000","last_undegraded":"2024-03-07T06:51:16.205600+0000","last_fullsized":"2024-03-07T06:51:16.205600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193446+0000","last_change":"2024-03-07T06:51:15.195194+0000","last_active":"2024-03-07T06:51:16.193446+0000","last_peered":"2024-03-07T06:51:16.193446+0000","last_clean":"2024-03-07T06:51:16.193446+0000","last_became_active":"2024-03-07T06:51:15.191103+0000","last_became_peered":"2024-03-07T06:51:15.191103+0000","last_unstale":"2024-03-07T06:51:16.193446+0000","last_undegraded":"2024-03-07T06:51:16.193446+0000","last_fullsized":"2024-03-07T06:51:16.193446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:40:51.935140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192876+0000","last_change":"2024-03-07T06:51:14.166640+0000","last_active":"2024-03-07T06:51:16.192876+0000","last_peered":"2024-03-07T06:51:16.192876+0000","last_clean":"2024-03-07T06:51:16.192876+0000","last_became_active":"2024-03-07T06:51:14.166329+0000","last_became_peered":"2024-03-07T06:51:14.166329+0000","last_unstale":"2024-03-07T06:51:16.192876+0000","last_undegraded":"2024-03-07T06:51:16.192876+0000","last_fullsized":"2024-03-07T06:51:16.192876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188809+0000","last_change":"2024-03-07T06:51:15.193638+0000","last_active":"2024-03-07T06:51:16.188809+0000","last_peered":"2024-03-07T06:51:16.188809+0000","last_clean":"2024-03-07T06:51:16.188809+0000","last_became_active":"2024-03-07T06:51:15.193354+0000","last_became_peered":"2024-03-07T06:51:15.193354+0000","last_unstale":"2024-03-07T06:51:16.188809+0000","last_undegraded":"2024-03-07T06:51:16.188809+0000","last_fullsized":"2024-03-07T06:51:16.188809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413807+0000","last_change":"2024-03-07T06:51:14.167158+0000","last_active":"2024-03-07T06:51:17.413807+0000","last_peered":"2024-03-07T06:51:17.413807+0000","last_clean":"2024-03-07T06:51:17.413807+0000","last_became_active":"2024-03-07T06:51:14.166835+0000","last_became_peered":"2024-03-07T06:51:14.166835+0000","last_unstale":"2024-03-07T06:51:17.413807+0000","last_undegraded":"2024-03-07T06:51:17.413807+0000","last_fullsized":"2024-03-07T06:51:17.413807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:38:27.101103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413891+0000","last_change":"2024-03-07T06:51:15.199375+0000","last_active":"2024-03-07T06:51:17.413891+0000","last_peered":"2024-03-07T06:51:17.413891+0000","last_clean":"2024-03-07T06:51:17.413891+0000","last_became_active":"2024-03-07T06:51:15.199060+0000","last_became_peered":"2024-03-07T06:51:15.199060+0000","last_unstale":"2024-03-07T06:51:17.413891+0000","last_undegraded":"2024-03-07T06:51:17.413891+0000","last_fullsized":"2024-03-07T06:51:17.413891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415423+0000","last_change":"2024-03-07T06:51:14.170514+0000","last_active":"2024-03-07T06:51:17.415423+0000","last_peered":"2024-03-07T06:51:17.415423+0000","last_clean":"2024-03-07T06:51:17.415423+0000","last_became_active":"2024-03-07T06:51:14.169542+0000","last_became_peered":"2024-03-07T06:51:14.169542+0000","last_unstale":"2024-03-07T06:51:17.415423+0000","last_undegraded":"2024-03-07T06:51:17.415423+0000","last_fullsized":"2024-03-07T06:51:17.415423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191234+0000","last_change":"2024-03-07T06:51:15.208081+0000","last_active":"2024-03-07T06:51:16.191234+0000","last_peered":"2024-03-07T06:51:16.191234+0000","last_clean":"2024-03-07T06:51:16.191234+0000","last_became_active":"2024-03-07T06:51:15.207758+0000","last_became_peered":"2024-03-07T06:51:15.207758+0000","last_unstale":"2024-03-07T06:51:16.191234+0000","last_undegraded":"2024-03-07T06:51:16.191234+0000","last_fullsized":"2024-03-07T06:51:16.191234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:43:16.081728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191798+0000","last_change":"2024-03-07T06:51:14.168470+0000","last_active":"2024-03-07T06:51:16.191798+0000","last_peered":"2024-03-07T06:51:16.191798+0000","last_clean":"2024-03-07T06:51:16.191798+0000","last_became_active":"2024-03-07T06:51:14.168190+0000","last_became_peered":"2024-03-07T06:51:14.168190+0000","last_unstale":"2024-03-07T06:51:16.191798+0000","last_undegraded":"2024-03-07T06:51:16.191798+0000","last_fullsized":"2024-03-07T06:51:16.191798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192778+0000","last_change":"2024-03-07T06:51:15.201105+0000","last_active":"2024-03-07T06:51:16.192778+0000","last_peered":"2024-03-07T06:51:16.192778+0000","last_clean":"2024-03-07T06:51:16.192778+0000","last_became_active":"2024-03-07T06:51:15.200404+0000","last_became_peered":"2024-03-07T06:51:15.200404+0000","last_unstale":"2024-03-07T06:51:16.192778+0000","last_undegraded":"2024-03-07T06:51:16.192778+0000","last_fullsized":"2024-03-07T06:51:16.192778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412017+0000","last_change":"2024-03-07T06:51:14.154100+0000","last_active":"2024-03-07T06:51:17.412017+0000","last_peered":"2024-03-07T06:51:17.412017+0000","last_clean":"2024-03-07T06:51:17.412017+0000","last_became_active":"2024-03-07T06:51:14.152648+0000","last_became_peered":"2024-03-07T06:51:14.152648+0000","last_unstale":"2024-03-07T06:51:17.412017+0000","last_undegraded":"2024-03-07T06:51:17.412017+0000","last_fullsized":"2024-03-07T06:51:17.412017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189886+0000","last_change":"2024-03-07T06:51:15.203942+0000","last_active":"2024-03-07T06:51:16.189886+0000","last_peered":"2024-03-07T06:51:16.189886+0000","last_clean":"2024-03-07T06:51:16.189886+0000","last_became_active":"2024-03-07T06:51:15.203671+0000","last_became_peered":"2024-03-07T06:51:15.203671+0000","last_unstale":"2024-03-07T06:51:16.189886+0000","last_undegraded":"2024-03-07T06:51:16.189886+0000","last_fullsized":"2024-03-07T06:51:16.189886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:02:55.966834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193630+0000","last_change":"2024-03-07T06:51:14.164166+0000","last_active":"2024-03-07T06:51:16.193630+0000","last_peered":"2024-03-07T06:51:16.193630+0000","last_clean":"2024-03-07T06:51:16.193630+0000","last_became_active":"2024-03-07T06:51:14.163821+0000","last_became_peered":"2024-03-07T06:51:14.163821+0000","last_unstale":"2024-03-07T06:51:16.193630+0000","last_undegraded":"2024-03-07T06:51:16.193630+0000","last_fullsized":"2024-03-07T06:51:16.193630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:38:27.101103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190408+0000","last_change":"2024-03-07T06:51:15.207392+0000","last_active":"2024-03-07T06:51:16.190408+0000","last_peered":"2024-03-07T06:51:16.190408+0000","last_clean":"2024-03-07T06:51:16.190408+0000","last_became_active":"2024-03-07T06:51:15.203049+0000","last_became_peered":"2024-03-07T06:51:15.203049+0000","last_unstale":"2024-03-07T06:51:16.190408+0000","last_undegraded":"2024-03-07T06:51:16.190408+0000","last_fullsized":"2024-03-07T06:51:16.190408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:43:58.927044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413191+0000","last_change":"2024-03-07T06:51:14.162546+0000","last_active":"2024-03-07T06:51:17.413191+0000","last_peered":"2024-03-07T06:51:17.413191+0000","last_clean":"2024-03-07T06:51:17.413191+0000","last_became_active":"2024-03-07T06:51:14.161082+0000","last_became_peered":"2024-03-07T06:51:14.161082+0000","last_unstale":"2024-03-07T06:51:17.413191+0000","last_undegraded":"2024-03-07T06:51:17.413191+0000","last_fullsized":"2024-03-07T06:51:17.413191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190725+0000","last_change":"2024-03-07T06:51:15.208036+0000","last_active":"2024-03-07T06:51:16.190725+0000","last_peered":"2024-03-07T06:51:16.190725+0000","last_clean":"2024-03-07T06:51:16.190725+0000","last_became_active":"2024-03-07T06:51:15.206837+0000","last_became_peered":"2024-03-07T06:51:15.206837+0000","last_unstale":"2024-03-07T06:51:16.190725+0000","last_undegraded":"2024-03-07T06:51:16.190725+0000","last_fullsized":"2024-03-07T06:51:16.190725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:55:33.816984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205890+0000","last_change":"2024-03-07T06:51:14.168546+0000","last_active":"2024-03-07T06:51:16.205890+0000","last_peered":"2024-03-07T06:51:16.205890+0000","last_clean":"2024-03-07T06:51:16.205890+0000","last_became_active":"2024-03-07T06:51:14.168238+0000","last_became_peered":"2024-03-07T06:51:14.168238+0000","last_unstale":"2024-03-07T06:51:16.205890+0000","last_undegraded":"2024-03-07T06:51:16.205890+0000","last_fullsized":"2024-03-07T06:51:16.205890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415347+0000","last_change":"2024-03-07T06:51:15.204629+0000","last_active":"2024-03-07T06:51:17.415347+0000","last_peered":"2024-03-07T06:51:17.415347+0000","last_clean":"2024-03-07T06:51:17.415347+0000","last_became_active":"2024-03-07T06:51:15.203615+0000","last_became_peered":"2024-03-07T06:51:15.203615+0000","last_unstale":"2024-03-07T06:51:17.415347+0000","last_undegraded":"2024-03-07T06:51:17.415347+0000","last_fullsized":"2024-03-07T06:51:17.415347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:38:28.416311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412767+0000","last_change":"2024-03-07T06:51:14.170179+0000","last_active":"2024-03-07T06:51:17.412767+0000","last_peered":"2024-03-07T06:51:17.412767+0000","last_clean":"2024-03-07T06:51:17.412767+0000","last_became_active":"2024-03-07T06:51:14.169797+0000","last_became_peered":"2024-03-07T06:51:14.169797+0000","last_unstale":"2024-03-07T06:51:17.412767+0000","last_undegraded":"2024-03-07T06:51:17.412767+0000","last_fullsized":"2024-03-07T06:51:17.412767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188955+0000","last_change":"2024-03-07T06:51:15.212961+0000","last_active":"2024-03-07T06:51:16.188955+0000","last_peered":"2024-03-07T06:51:16.188955+0000","last_clean":"2024-03-07T06:51:16.188955+0000","last_became_active":"2024-03-07T06:51:15.212691+0000","last_became_peered":"2024-03-07T06:51:15.212691+0000","last_unstale":"2024-03-07T06:51:16.188955+0000","last_undegraded":"2024-03-07T06:51:16.188955+0000","last_fullsized":"2024-03-07T06:51:16.188955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:38:28.416311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188951+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:51:16.188951+0000","last_peered":"2024-03-07T06:51:16.188951+0000","last_clean":"2024-03-07T06:51:16.188951+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:51:16.188951+0000","last_undegraded":"2024-03-07T06:51:16.188951+0000","last_fullsized":"2024-03-07T06:51:16.188951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414355+0000","last_change":"2024-03-07T06:51:14.172051+0000","last_active":"2024-03-07T06:51:17.414355+0000","last_peered":"2024-03-07T06:51:17.414355+0000","last_clean":"2024-03-07T06:51:17.414355+0000","last_became_active":"2024-03-07T06:51:14.170736+0000","last_became_peered":"2024-03-07T06:51:14.170736+0000","last_unstale":"2024-03-07T06:51:17.414355+0000","last_undegraded":"2024-03-07T06:51:17.414355+0000","last_fullsized":"2024-03-07T06:51:17.414355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415537+0000","last_change":"2024-03-07T06:51:15.208661+0000","last_active":"2024-03-07T06:51:17.415537+0000","last_peered":"2024-03-07T06:51:17.415537+0000","last_clean":"2024-03-07T06:51:17.415537+0000","last_became_active":"2024-03-07T06:51:15.208394+0000","last_became_peered":"2024-03-07T06:51:15.208394+0000","last_unstale":"2024-03-07T06:51:17.415537+0000","last_undegraded":"2024-03-07T06:51:17.415537+0000","last_fullsized":"2024-03-07T06:51:17.415537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188640+0000","last_change":"2024-03-07T06:51:14.163185+0000","last_active":"2024-03-07T06:51:16.188640+0000","last_peered":"2024-03-07T06:51:16.188640+0000","last_clean":"2024-03-07T06:51:16.188640+0000","last_became_active":"2024-03-07T06:51:14.162494+0000","last_became_peered":"2024-03-07T06:51:14.162494+0000","last_unstale":"2024-03-07T06:51:16.188640+0000","last_undegraded":"2024-03-07T06:51:16.188640+0000","last_fullsized":"2024-03-07T06:51:16.188640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412672+0000","last_change":"2024-03-07T06:51:15.193638+0000","last_active":"2024-03-07T06:51:17.412672+0000","last_peered":"2024-03-07T06:51:17.412672+0000","last_clean":"2024-03-07T06:51:17.412672+0000","last_became_active":"2024-03-07T06:51:15.193379+0000","last_became_peered":"2024-03-07T06:51:15.193379+0000","last_unstale":"2024-03-07T06:51:17.412672+0000","last_undegraded":"2024-03-07T06:51:17.412672+0000","last_fullsized":"2024-03-07T06:51:17.412672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:16:10.967500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412927+0000","last_change":"2024-03-07T06:51:14.163793+0000","last_active":"2024-03-07T06:51:17.412927+0000","last_peered":"2024-03-07T06:51:17.412927+0000","last_clean":"2024-03-07T06:51:17.412927+0000","last_became_active":"2024-03-07T06:51:14.162059+0000","last_became_peered":"2024-03-07T06:51:14.162059+0000","last_unstale":"2024-03-07T06:51:17.412927+0000","last_undegraded":"2024-03-07T06:51:17.412927+0000","last_fullsized":"2024-03-07T06:51:17.412927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205257+0000","last_change":"2024-03-07T06:51:15.187522+0000","last_active":"2024-03-07T06:51:16.205257+0000","last_peered":"2024-03-07T06:51:16.205257+0000","last_clean":"2024-03-07T06:51:16.205257+0000","last_became_active":"2024-03-07T06:51:15.179882+0000","last_became_peered":"2024-03-07T06:51:15.179882+0000","last_unstale":"2024-03-07T06:51:16.205257+0000","last_undegraded":"2024-03-07T06:51:16.205257+0000","last_fullsized":"2024-03-07T06:51:16.205257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415629+0000","last_change":"2024-03-07T06:51:14.164766+0000","last_active":"2024-03-07T06:51:17.415629+0000","last_peered":"2024-03-07T06:51:17.415629+0000","last_clean":"2024-03-07T06:51:17.415629+0000","last_became_active":"2024-03-07T06:51:14.164397+0000","last_became_peered":"2024-03-07T06:51:14.164397+0000","last_unstale":"2024-03-07T06:51:17.415629+0000","last_undegraded":"2024-03-07T06:51:17.415629+0000","last_fullsized":"2024-03-07T06:51:17.415629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192589+0000","last_change":"2024-03-07T06:51:15.190608+0000","last_active":"2024-03-07T06:51:16.192589+0000","last_peered":"2024-03-07T06:51:16.192589+0000","last_clean":"2024-03-07T06:51:16.192589+0000","last_became_active":"2024-03-07T06:51:15.190326+0000","last_became_peered":"2024-03-07T06:51:15.190326+0000","last_unstale":"2024-03-07T06:51:16.192589+0000","last_undegraded":"2024-03-07T06:51:16.192589+0000","last_fullsized":"2024-03-07T06:51:16.192589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:43.574428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205896+0000","last_change":"2024-03-07T06:51:14.164402+0000","last_active":"2024-03-07T06:51:16.205896+0000","last_peered":"2024-03-07T06:51:16.205896+0000","last_clean":"2024-03-07T06:51:16.205896+0000","last_became_active":"2024-03-07T06:51:14.164081+0000","last_became_peered":"2024-03-07T06:51:14.164081+0000","last_unstale":"2024-03-07T06:51:16.205896+0000","last_undegraded":"2024-03-07T06:51:16.205896+0000","last_fullsized":"2024-03-07T06:51:16.205896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415903+0000","last_change":"2024-03-07T06:51:15.208063+0000","last_active":"2024-03-07T06:51:17.415903+0000","last_peered":"2024-03-07T06:51:17.415903+0000","last_clean":"2024-03-07T06:51:17.415903+0000","last_became_active":"2024-03-07T06:51:15.207782+0000","last_became_peered":"2024-03-07T06:51:15.207782+0000","last_unstale":"2024-03-07T06:51:17.415903+0000","last_undegraded":"2024-03-07T06:51:17.415903+0000","last_fullsized":"2024-03-07T06:51:17.415903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:02:55.966834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189333+0000","last_change":"2024-03-07T06:51:14.156560+0000","last_active":"2024-03-07T06:51:16.189333+0000","last_peered":"2024-03-07T06:51:16.189333+0000","last_clean":"2024-03-07T06:51:16.189333+0000","last_became_active":"2024-03-07T06:51:14.156040+0000","last_became_peered":"2024-03-07T06:51:14.156040+0000","last_unstale":"2024-03-07T06:51:16.189333+0000","last_undegraded":"2024-03-07T06:51:16.189333+0000","last_fullsized":"2024-03-07T06:51:16.189333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:07:44.222048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205750+0000","last_change":"2024-03-07T06:51:15.180806+0000","last_active":"2024-03-07T06:51:16.205750+0000","last_peered":"2024-03-07T06:51:16.205750+0000","last_clean":"2024-03-07T06:51:16.205750+0000","last_became_active":"2024-03-07T06:51:15.180402+0000","last_became_peered":"2024-03-07T06:51:15.180402+0000","last_unstale":"2024-03-07T06:51:16.205750+0000","last_undegraded":"2024-03-07T06:51:16.205750+0000","last_fullsized":"2024-03-07T06:51:16.205750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205976+0000","last_change":"2024-03-07T06:51:14.168110+0000","last_active":"2024-03-07T06:51:16.205976+0000","last_peered":"2024-03-07T06:51:16.205976+0000","last_clean":"2024-03-07T06:51:16.205976+0000","last_became_active":"2024-03-07T06:51:14.167759+0000","last_became_peered":"2024-03-07T06:51:14.167759+0000","last_unstale":"2024-03-07T06:51:16.205976+0000","last_undegraded":"2024-03-07T06:51:16.205976+0000","last_fullsized":"2024-03-07T06:51:16.205976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.416085+0000","last_change":"2024-03-07T06:51:15.199832+0000","last_active":"2024-03-07T06:51:17.416085+0000","last_peered":"2024-03-07T06:51:17.416085+0000","last_clean":"2024-03-07T06:51:17.416085+0000","last_became_active":"2024-03-07T06:51:15.199572+0000","last_became_peered":"2024-03-07T06:51:15.199572+0000","last_unstale":"2024-03-07T06:51:17.416085+0000","last_undegraded":"2024-03-07T06:51:17.416085+0000","last_fullsized":"2024-03-07T06:51:17.416085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193650+0000","last_change":"2024-03-07T06:51:14.173549+0000","last_active":"2024-03-07T06:51:16.193650+0000","last_peered":"2024-03-07T06:51:16.193650+0000","last_clean":"2024-03-07T06:51:16.193650+0000","last_became_active":"2024-03-07T06:51:14.173251+0000","last_became_peered":"2024-03-07T06:51:14.173251+0000","last_unstale":"2024-03-07T06:51:16.193650+0000","last_undegraded":"2024-03-07T06:51:16.193650+0000","last_fullsized":"2024-03-07T06:51:16.193650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413748+0000","last_change":"2024-03-07T06:51:15.195178+0000","last_active":"2024-03-07T06:51:17.413748+0000","last_peered":"2024-03-07T06:51:17.413748+0000","last_clean":"2024-03-07T06:51:17.413748+0000","last_became_active":"2024-03-07T06:51:15.194907+0000","last_became_peered":"2024-03-07T06:51:15.194907+0000","last_unstale":"2024-03-07T06:51:17.413748+0000","last_undegraded":"2024-03-07T06:51:17.413748+0000","last_fullsized":"2024-03-07T06:51:17.413748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188590+0000","last_change":"2024-03-07T06:51:14.156566+0000","last_active":"2024-03-07T06:51:16.188590+0000","last_peered":"2024-03-07T06:51:16.188590+0000","last_clean":"2024-03-07T06:51:16.188590+0000","last_became_active":"2024-03-07T06:51:14.156261+0000","last_became_peered":"2024-03-07T06:51:14.156261+0000","last_unstale":"2024-03-07T06:51:16.188590+0000","last_undegraded":"2024-03-07T06:51:16.188590+0000","last_fullsized":"2024-03-07T06:51:16.188590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193424+0000","last_change":"2024-03-07T06:51:15.183065+0000","last_active":"2024-03-07T06:51:16.193424+0000","last_peered":"2024-03-07T06:51:16.193424+0000","last_clean":"2024-03-07T06:51:16.193424+0000","last_became_active":"2024-03-07T06:51:15.182189+0000","last_became_peered":"2024-03-07T06:51:15.182189+0000","last_unstale":"2024-03-07T06:51:16.193424+0000","last_undegraded":"2024-03-07T06:51:16.193424+0000","last_fullsized":"2024-03-07T06:51:16.193424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:05:21.883987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205102+0000","last_change":"2024-03-07T06:51:14.169705+0000","last_active":"2024-03-07T06:51:16.205102+0000","last_peered":"2024-03-07T06:51:16.205102+0000","last_clean":"2024-03-07T06:51:16.205102+0000","last_became_active":"2024-03-07T06:51:14.169444+0000","last_became_peered":"2024-03-07T06:51:14.169444+0000","last_unstale":"2024-03-07T06:51:16.205102+0000","last_undegraded":"2024-03-07T06:51:16.205102+0000","last_fullsized":"2024-03-07T06:51:16.205102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415220+0000","last_change":"2024-03-07T06:51:15.197549+0000","last_active":"2024-03-07T06:51:17.415220+0000","last_peered":"2024-03-07T06:51:17.415220+0000","last_clean":"2024-03-07T06:51:17.415220+0000","last_became_active":"2024-03-07T06:51:15.195020+0000","last_became_peered":"2024-03-07T06:51:15.195020+0000","last_unstale":"2024-03-07T06:51:17.415220+0000","last_undegraded":"2024-03-07T06:51:17.415220+0000","last_fullsized":"2024-03-07T06:51:17.415220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205415+0000","last_change":"2024-03-07T06:51:14.170410+0000","last_active":"2024-03-07T06:51:16.205415+0000","last_peered":"2024-03-07T06:51:16.205415+0000","last_clean":"2024-03-07T06:51:16.205415+0000","last_became_active":"2024-03-07T06:51:14.170043+0000","last_became_peered":"2024-03-07T06:51:14.170043+0000","last_unstale":"2024-03-07T06:51:16.205415+0000","last_undegraded":"2024-03-07T06:51:16.205415+0000","last_fullsized":"2024-03-07T06:51:16.205415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415636+0000","last_change":"2024-03-07T06:51:15.193076+0000","last_active":"2024-03-07T06:51:17.415636+0000","last_peered":"2024-03-07T06:51:17.415636+0000","last_clean":"2024-03-07T06:51:17.415636+0000","last_became_active":"2024-03-07T06:51:15.192700+0000","last_became_peered":"2024-03-07T06:51:15.192700+0000","last_unstale":"2024-03-07T06:51:17.415636+0000","last_undegraded":"2024-03-07T06:51:17.415636+0000","last_fullsized":"2024-03-07T06:51:17.415636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191169+0000","last_change":"2024-03-07T06:51:14.173220+0000","last_active":"2024-03-07T06:51:16.191169+0000","last_peered":"2024-03-07T06:51:16.191169+0000","last_clean":"2024-03-07T06:51:16.191169+0000","last_became_active":"2024-03-07T06:51:14.169687+0000","last_became_peered":"2024-03-07T06:51:14.169687+0000","last_unstale":"2024-03-07T06:51:16.191169+0000","last_undegraded":"2024-03-07T06:51:16.191169+0000","last_fullsized":"2024-03-07T06:51:16.191169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:02:40.568300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192059+0000","last_change":"2024-03-07T06:51:15.205150+0000","last_active":"2024-03-07T06:51:16.192059+0000","last_peered":"2024-03-07T06:51:16.192059+0000","last_clean":"2024-03-07T06:51:16.192059+0000","last_became_active":"2024-03-07T06:51:15.204435+0000","last_became_peered":"2024-03-07T06:51:15.204435+0000","last_unstale":"2024-03-07T06:51:16.192059+0000","last_undegraded":"2024-03-07T06:51:16.192059+0000","last_fullsized":"2024-03-07T06:51:16.192059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415532+0000","last_change":"2024-03-07T06:51:15.214214+0000","last_active":"2024-03-07T06:51:17.415532+0000","last_peered":"2024-03-07T06:51:17.415532+0000","last_clean":"2024-03-07T06:51:17.415532+0000","last_became_active":"2024-03-07T06:51:15.209505+0000","last_became_peered":"2024-03-07T06:51:15.209505+0000","last_unstale":"2024-03-07T06:51:17.415532+0000","last_undegraded":"2024-03-07T06:51:17.415532+0000","last_fullsized":"2024-03-07T06:51:17.415532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:56.465649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188962+0000","last_change":"2024-03-07T06:51:14.172650+0000","last_active":"2024-03-07T06:51:16.188962+0000","last_peered":"2024-03-07T06:51:16.188962+0000","last_clean":"2024-03-07T06:51:16.188962+0000","last_became_active":"2024-03-07T06:51:14.170441+0000","last_became_peered":"2024-03-07T06:51:14.170441+0000","last_unstale":"2024-03-07T06:51:16.188962+0000","last_undegraded":"2024-03-07T06:51:16.188962+0000","last_fullsized":"2024-03-07T06:51:16.188962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:05:20.871811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415022+0000","last_change":"2024-03-07T06:51:15.204124+0000","last_active":"2024-03-07T06:51:17.415022+0000","last_peered":"2024-03-07T06:51:17.415022+0000","last_clean":"2024-03-07T06:51:17.415022+0000","last_became_active":"2024-03-07T06:51:15.203713+0000","last_became_peered":"2024-03-07T06:51:15.203713+0000","last_unstale":"2024-03-07T06:51:17.415022+0000","last_undegraded":"2024-03-07T06:51:17.415022+0000","last_fullsized":"2024-03-07T06:51:17.415022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:34:40.365080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189015+0000","last_change":"2024-03-07T06:51:14.167422+0000","last_active":"2024-03-07T06:51:16.189015+0000","last_peered":"2024-03-07T06:51:16.189015+0000","last_clean":"2024-03-07T06:51:16.189015+0000","last_became_active":"2024-03-07T06:51:14.167152+0000","last_became_peered":"2024-03-07T06:51:14.167152+0000","last_unstale":"2024-03-07T06:51:16.189015+0000","last_undegraded":"2024-03-07T06:51:16.189015+0000","last_fullsized":"2024-03-07T06:51:16.189015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:40:50.922964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412609+0000","last_change":"2024-03-07T06:51:15.210848+0000","last_active":"2024-03-07T06:51:17.412609+0000","last_peered":"2024-03-07T06:51:17.412609+0000","last_clean":"2024-03-07T06:51:17.412609+0000","last_became_active":"2024-03-07T06:51:15.210526+0000","last_became_peered":"2024-03-07T06:51:15.210526+0000","last_unstale":"2024-03-07T06:51:17.412609+0000","last_undegraded":"2024-03-07T06:51:17.412609+0000","last_fullsized":"2024-03-07T06:51:17.412609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:40:57.934235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415818+0000","last_change":"2024-03-07T06:51:14.163311+0000","last_active":"2024-03-07T06:51:17.415818+0000","last_peered":"2024-03-07T06:51:17.415818+0000","last_clean":"2024-03-07T06:51:17.415818+0000","last_became_active":"2024-03-07T06:51:14.161332+0000","last_became_peered":"2024-03-07T06:51:14.161332+0000","last_unstale":"2024-03-07T06:51:17.415818+0000","last_undegraded":"2024-03-07T06:51:17.415818+0000","last_fullsized":"2024-03-07T06:51:17.415818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415995+0000","last_change":"2024-03-07T06:51:15.200486+0000","last_active":"2024-03-07T06:51:17.415995+0000","last_peered":"2024-03-07T06:51:17.415995+0000","last_clean":"2024-03-07T06:51:17.415995+0000","last_became_active":"2024-03-07T06:51:15.200171+0000","last_became_peered":"2024-03-07T06:51:15.200171+0000","last_unstale":"2024-03-07T06:51:17.415995+0000","last_undegraded":"2024-03-07T06:51:17.415995+0000","last_fullsized":"2024-03-07T06:51:17.415995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:22:38.973224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412810+0000","last_change":"2024-03-07T06:51:14.169164+0000","last_active":"2024-03-07T06:51:17.412810+0000","last_peered":"2024-03-07T06:51:17.412810+0000","last_clean":"2024-03-07T06:51:17.412810+0000","last_became_active":"2024-03-07T06:51:14.168662+0000","last_became_peered":"2024-03-07T06:51:14.168662+0000","last_unstale":"2024-03-07T06:51:17.412810+0000","last_undegraded":"2024-03-07T06:51:17.412810+0000","last_fullsized":"2024-03-07T06:51:17.412810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:51:25.137 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-07T06:51:25.137 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-07T06:51:25.137 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-07T06:51:25.138 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-07T06:51:25.138 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-07T06:51:25.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.388+0000 7fe909000640 1 Processor -- start 2024-03-07T06:51:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.389+0000 7fe909000640 1 -- start start 2024-03-07T06:51:25.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.390+0000 7fe909000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe90415bd30 0x7fe90406e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:25.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.390+0000 7fe909000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe9040cf5d0 0x7fe904072830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:25.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.390+0000 7fe909000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe904160760 con 0x7fe904072d70 2024-03-07T06:51:25.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.390+0000 7fe909000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9040731e0 con 0x7fe9040cf5d0 2024-03-07T06:51:25.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.390+0000 7fe909000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90415f9a0 con 0x7fe90415bd30 2024-03-07T06:51:25.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.391+0000 7fe903e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe904072d70 0x7fe90406cf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44098/0 (socket says 172.21.15.133:44098) 2024-03-07T06:51:25.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.391+0000 7fe903e00640 1 -- 172.21.15.133:0/177033157 learned_addr learned my addr 172.21.15.133:0/177033157 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:25.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.391+0000 7fe903400640 1 --2- 172.21.15.133:0/177033157 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe90415bd30 0x7fe90406e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:25.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.391+0000 7fe902a00640 1 --2- 172.21.15.133:0/177033157 >> v2:172.21.15.161:3300/0 conn(0x7fe9040cf5d0 0x7fe904072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:25.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.391+0000 7fe902000640 1 -- 172.21.15.133:0/177033157 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33622458 0 0) 0x7fe904160760 con 0x7fe904072d70 2024-03-07T06:51:25.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.391+0000 7fe902000640 1 -- 172.21.15.133:0/177033157 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e8003520 con 0x7fe904072d70 2024-03-07T06:51:25.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.391+0000 7fe903400640 1 -- 172.21.15.133:0/177033157 >> v2:172.21.15.161:3300/0 conn(0x7fe9040cf5d0 msgr2=0x7fe904072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:25.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.391+0000 7fe903400640 1 --2- 172.21.15.133:0/177033157 >> v2:172.21.15.161:3300/0 conn(0x7fe9040cf5d0 0x7fe904072830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.391+0000 7fe903400640 1 -- 172.21.15.133:0/177033157 >> v1:172.21.15.133:6789/0 conn(0x7fe904072d70 legacy=0x7fe90406cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:25.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.391+0000 7fe902000640 1 -- 172.21.15.133:0/177033157 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3952232936 0 0) 0x7fe8e8003520 con 0x7fe904072d70 2024-03-07T06:51:25.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.391+0000 7fe903400640 1 -- 172.21.15.133:0/177033157 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90415fc80 con 0x7fe90415bd30 2024-03-07T06:51:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.393+0000 7fe903400640 1 --2- 172.21.15.133:0/177033157 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe90415bd30 0x7fe90406e8e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40097a0 tx=0x7fe8f402b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21cc9349f481d5b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.393+0000 7fe902000640 1 -- 172.21.15.133:0/177033157 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4037070 con 0x7fe90415bd30 2024-03-07T06:51:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.393+0000 7fe902000640 1 -- 172.21.15.133:0/177033157 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f4030510 con 0x7fe90415bd30 2024-03-07T06:51:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.393+0000 7fe902000640 1 -- 172.21.15.133:0/177033157 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0019f0 con 0x7fe90415bd30 2024-03-07T06:51:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.394+0000 7fe909000640 1 -- 172.21.15.133:0/177033157 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe90415bd30 msgr2=0x7fe90406e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.394+0000 7fe909000640 1 --2- 172.21.15.133:0/177033157 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe90415bd30 0x7fe90406e8e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40097a0 tx=0x7fe8f402b790 comp rx=0 tx=0).stop 2024-03-07T06:51:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.394+0000 7fe909000640 1 -- 172.21.15.133:0/177033157 shutdown_connections 2024-03-07T06:51:25.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.394+0000 7fe909000640 1 --2- 172.21.15.133:0/177033157 >> v2:172.21.15.161:3300/0 conn(0x7fe9040cf5d0 0x7fe904072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.394+0000 7fe909000640 1 --2- 172.21.15.133:0/177033157 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe90415bd30 0x7fe90406e8e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.394+0000 7fe909000640 1 -- 172.21.15.133:0/177033157 >> 172.21.15.133:0/177033157 conn(0x7fe9040b2e90 msgr2=0x7fe9040b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:25.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.394+0000 7fe909000640 1 -- 172.21.15.133:0/177033157 shutdown_connections 2024-03-07T06:51:25.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.394+0000 7fe909000640 1 -- 172.21.15.133:0/177033157 wait complete. 2024-03-07T06:51:25.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.396+0000 7fe909000640 1 Processor -- start 2024-03-07T06:51:25.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.396+0000 7fe909000640 1 -- start start 2024-03-07T06:51:25.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe909000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe904072d70 0x7fe904153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:25.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe909000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9040cf5d0 0x7fe9040cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:25.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe909000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe90415bd30 0x7fe90415a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:25.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe909000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90406ee20 con 0x7fe90415bd30 2024-03-07T06:51:25.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe909000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe904160240 con 0x7fe9040cf5d0 2024-03-07T06:51:25.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe909000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90406f130 con 0x7fe904072d70 2024-03-07T06:51:25.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe903e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe90415bd30 0x7fe90415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:25.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe903e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe90415bd30 0x7fe90415a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38458/0 (socket says 172.21.15.133:38458) 2024-03-07T06:51:25.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe903e00640 1 -- 172.21.15.133:0/3938945951 learned_addr learned my addr 172.21.15.133:0/3938945951 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:25.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe902a00640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9040cf5d0 0x7fe9040cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:25.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe903400640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe904072d70 0x7fe904153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:25.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe903e00640 1 -- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe904072d70 msgr2=0x7fe904153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:25.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe903e00640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe904072d70 0x7fe904153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe903e00640 1 -- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9040cf5d0 msgr2=0x7fe9040cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:25.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe903e00640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9040cf5d0 0x7fe9040cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.397+0000 7fe903e00640 1 -- 172.21.15.133:0/3938945951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9040cd290 con 0x7fe90415bd30 2024-03-07T06:51:25.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.398+0000 7fe903400640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe904072d70 0x7fe904153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:25.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.398+0000 7fe903e00640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe90415bd30 0x7fe90415a370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00ccc0 tx=0x7fe8fc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:25.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.398+0000 7fe8fb400640 1 -- 172.21.15.133:0/3938945951 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc012400 con 0x7fe90415bd30 2024-03-07T06:51:25.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.398+0000 7fe8fb400640 1 -- 172.21.15.133:0/3938945951 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc0125c0 con 0x7fe90415bd30 2024-03-07T06:51:25.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.398+0000 7fe8fb400640 1 -- 172.21.15.133:0/3938945951 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0129b0 con 0x7fe90415bd30 2024-03-07T06:51:25.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.398+0000 7fe909000640 1 -- 172.21.15.133:0/3938945951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9040cd570 con 0x7fe90415bd30 2024-03-07T06:51:25.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.398+0000 7fe909000640 1 -- 172.21.15.133:0/3938945951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9040cdb30 con 0x7fe90415bd30 2024-03-07T06:51:25.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.399+0000 7fe909000640 1 -- 172.21.15.133:0/3938945951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe904062c90 con 0x7fe90415bd30 2024-03-07T06:51:25.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.400+0000 7fe8fb400640 1 -- 172.21.15.133:0/3938945951 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe8fc012b50 con 0x7fe90415bd30 2024-03-07T06:51:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.401+0000 7fe8fb400640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.401+0000 7fe8fb400640 1 -- 172.21.15.133:0/3938945951 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8fc08aa60 con 0x7fe90415bd30 2024-03-07T06:51:25.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.402+0000 7fe903400640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:25.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.403+0000 7fe903400640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe8e406b4d0 0x7fe8e406d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4034000 tx=0x7fe8f402f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:25.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.405+0000 7fe8fb400640 1 -- 172.21.15.133:0/3938945951 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe8fc0182b0 con 0x7fe90415bd30 2024-03-07T06:51:25.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.647+0000 7fe909000640 1 -- 172.21.15.133:0/3938945951 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe9040734a0 con 0x7fe90415bd30 2024-03-07T06:51:25.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.653+0000 7fe8fb400640 1 -- 172.21.15.133:0/3938945951 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171785 (secure 0 0 0) 0x7fe8fc0598d0 con 0x7fe90415bd30 2024-03-07T06:51:25.656 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:25.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.657+0000 7fe909000640 1 -- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe8e406b4d0 msgr2=0x7fe8e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.657+0000 7fe909000640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe8e406b4d0 0x7fe8e406d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4034000 tx=0x7fe8f402f000 comp rx=0 tx=0).stop 2024-03-07T06:51:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.657+0000 7fe909000640 1 -- 172.21.15.133:0/3938945951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe90415bd30 msgr2=0x7fe90415a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.657+0000 7fe909000640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe90415bd30 0x7fe90415a370 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00ccc0 tx=0x7fe8fc00a430 comp rx=0 tx=0).stop 2024-03-07T06:51:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.657+0000 7fe909000640 1 -- 172.21.15.133:0/3938945951 shutdown_connections 2024-03-07T06:51:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.657+0000 7fe909000640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe90415bd30 0x7fe90415a370 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.657+0000 7fe909000640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.657+0000 7fe909000640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe9040cf5d0 0x7fe9040cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.657+0000 7fe909000640 1 --2- 172.21.15.133:0/3938945951 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe904072d70 0x7fe904153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.657+0000 7fe909000640 1 -- 172.21.15.133:0/3938945951 >> 172.21.15.133:0/3938945951 conn(0x7fe9040b2e90 msgr2=0x7fe9040af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:25.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.657+0000 7fe909000640 1 -- 172.21.15.133:0/3938945951 shutdown_connections 2024-03-07T06:51:25.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.657+0000 7fe909000640 1 -- 172.21.15.133:0/3938945951 wait complete. 2024-03-07T06:51:25.674 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":187879951},{"type":"v1","addr":"172.21.15.161:6801","nonce":187879951}]},"active_addr":"172.21.15.161:6801/187879951","active_change":"2024-03-07T06:50:54.448413+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":130564255}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":508400880}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":2035506333}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":2950484297}]}]} 2024-03-07T06:51:25.678 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-07T06:51:25.678 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-07T06:51:25.678 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:25.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.877+0000 7f11eca00640 1 Processor -- start 2024-03-07T06:51:25.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.877+0000 7f11eca00640 1 -- start start 2024-03-07T06:51:25.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.878+0000 7f11eca00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f11e815bd30 0x7f11e806e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:25.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.878+0000 7f11eca00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11e80cf5d0 0x7f11e8072830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:25.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.879+0000 7f11eca00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e8160760 con 0x7f11e8072d70 2024-03-07T06:51:25.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.879+0000 7f11eca00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e80731e0 con 0x7f11e815bd30 2024-03-07T06:51:25.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.879+0000 7f11eca00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e815f9a0 con 0x7f11e80cf5d0 2024-03-07T06:51:25.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.879+0000 7f11e7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f11e815bd30 0x7f11e806e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:25.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.879+0000 7f11dfe00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11e80cf5d0 0x7f11e8072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:25.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.879+0000 7f11e7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f11e815bd30 0x7f11e806e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58704/0 (socket says 172.21.15.133:58704) 2024-03-07T06:51:25.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.879+0000 7f11e7400640 1 -- 172.21.15.133:0/807593873 learned_addr learned my addr 172.21.15.133:0/807593873 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.880+0000 7f11dfe00640 1 -- 172.21.15.133:0/807593873 >> v2:172.21.15.161:3300/0 conn(0x7f11e815bd30 msgr2=0x7f11e806e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.880+0000 7f11dfe00640 1 --2- 172.21.15.133:0/807593873 >> v2:172.21.15.161:3300/0 conn(0x7f11e815bd30 0x7f11e806e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.880+0000 7f11dfe00640 1 -- 172.21.15.133:0/807593873 >> v1:172.21.15.161:6789/0 conn(0x7f11e8072d70 legacy=0x7f11e806cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.880+0000 7f11dfe00640 1 -- 172.21.15.133:0/807593873 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e815fc80 con 0x7f11e80cf5d0 2024-03-07T06:51:25.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.881+0000 7f11dfe00640 1 --2- 172.21.15.133:0/807593873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11e80cf5d0 0x7f11e8072830 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f11d00097a0 tx=0x7f11d002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd0f7dd893e44171 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.882+0000 7f11e6a00640 1 -- 172.21.15.133:0/807593873 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d0035020 con 0x7f11e80cf5d0 2024-03-07T06:51:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.882+0000 7f11e6a00640 1 -- 172.21.15.133:0/807593873 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d0031810 con 0x7f11e80cf5d0 2024-03-07T06:51:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.882+0000 7f11e6a00640 1 -- 172.21.15.133:0/807593873 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d0031be0 con 0x7f11e80cf5d0 2024-03-07T06:51:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.882+0000 7f11eca00640 1 -- 172.21.15.133:0/807593873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11e80cf5d0 msgr2=0x7f11e8072830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.882+0000 7f11eca00640 1 --2- 172.21.15.133:0/807593873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11e80cf5d0 0x7f11e8072830 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f11d00097a0 tx=0x7f11d002b8b0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.882+0000 7f11eca00640 1 -- 172.21.15.133:0/807593873 shutdown_connections 2024-03-07T06:51:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.882+0000 7f11eca00640 1 --2- 172.21.15.133:0/807593873 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11e80cf5d0 0x7f11e8072830 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.882+0000 7f11eca00640 1 --2- 172.21.15.133:0/807593873 >> v2:172.21.15.161:3300/0 conn(0x7f11e815bd30 0x7f11e806e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.882+0000 7f11eca00640 1 -- 172.21.15.133:0/807593873 >> 172.21.15.133:0/807593873 conn(0x7f11e80b2e90 msgr2=0x7f11e80b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:25.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.882+0000 7f11eca00640 1 -- 172.21.15.133:0/807593873 shutdown_connections 2024-03-07T06:51:25.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.882+0000 7f11eca00640 1 -- 172.21.15.133:0/807593873 wait complete. 2024-03-07T06:51:25.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.884+0000 7f11eca00640 1 Processor -- start 2024-03-07T06:51:25.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.884+0000 7f11eca00640 1 -- start start 2024-03-07T06:51:25.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11eca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e8072d70 0x7f11e81e68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11eca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11e80cf5d0 0x7f11e81e61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11eca00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11e815bd30 0x7f11e81465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11eca00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e806ee20 con 0x7f11e8072d70 2024-03-07T06:51:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11eca00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e8160240 con 0x7f11e80cf5d0 2024-03-07T06:51:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11eca00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e806f130 con 0x7f11e815bd30 2024-03-07T06:51:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11e7400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e8072d70 0x7f11e81e68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11e7400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e8072d70 0x7f11e81e68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38468/0 (socket says 172.21.15.133:38468) 2024-03-07T06:51:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11e7400640 1 -- 172.21.15.133:0/2741949888 learned_addr learned my addr 172.21.15.133:0/2741949888 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11dfe00640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11e80cf5d0 0x7f11e81e61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11e7400640 1 -- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11e815bd30 msgr2=0x7f11e81465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11e7e00640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11e815bd30 0x7f11e81465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11e7400640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11e815bd30 0x7f11e81465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11e7400640 1 -- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11e80cf5d0 msgr2=0x7f11e81e61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11e7400640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11e80cf5d0 0x7f11e81e61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.885+0000 7f11e7400640 1 -- 172.21.15.133:0/2741949888 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e81e6fb0 con 0x7f11e8072d70 2024-03-07T06:51:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.886+0000 7f11dfe00640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11e80cf5d0 0x7f11e81e61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.886+0000 7f11e7400640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e8072d70 0x7f11e81e68b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f11d800e840 tx=0x7f11d800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.886+0000 7f11df400640 1 -- 172.21.15.133:0/2741949888 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d8017070 con 0x7f11e8072d70 2024-03-07T06:51:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.886+0000 7f11df400640 1 -- 172.21.15.133:0/2741949888 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d800c8e0 con 0x7f11e8072d70 2024-03-07T06:51:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.886+0000 7f11eca00640 1 -- 172.21.15.133:0/2741949888 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e81e7290 con 0x7f11e8072d70 2024-03-07T06:51:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.886+0000 7f11df400640 1 -- 172.21.15.133:0/2741949888 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d800ccd0 con 0x7f11e8072d70 2024-03-07T06:51:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.886+0000 7f11eca00640 1 -- 172.21.15.133:0/2741949888 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e81e7850 con 0x7f11e8072d70 2024-03-07T06:51:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.887+0000 7f11eca00640 1 -- 172.21.15.133:0/2741949888 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e8062c90 con 0x7f11e8072d70 2024-03-07T06:51:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.888+0000 7f11df400640 1 -- 172.21.15.133:0/2741949888 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f11d801a070 con 0x7f11e8072d70 2024-03-07T06:51:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.889+0000 7f11df400640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f11c406b4d0 0x7f11c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.889+0000 7f11df400640 1 -- 172.21.15.133:0/2741949888 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11d808a440 con 0x7f11e8072d70 2024-03-07T06:51:25.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.889+0000 7f11dfe00640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f11c406b4d0 0x7f11c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:25.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.890+0000 7f11dfe00640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f11c406b4d0 0x7f11c406d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11d0036000 tx=0x7f11d0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:25.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:25.893+0000 7f11df400640 1 -- 172.21.15.133:0/2741949888 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f11d80592b0 con 0x7f11e8072d70 2024-03-07T06:51:26.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.089+0000 7f11eca00640 1 -- 172.21.15.133:0/2741949888 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11e80734a0 con 0x7f11e8072d70 2024-03-07T06:51:26.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.092+0000 7f11df400640 1 -- 172.21.15.133:0/2741949888 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f11d8058bb0 con 0x7f11e8072d70 2024-03-07T06:51:26.091 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:26.091 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":21,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:16.174650+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:26.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.094+0000 7f11eca00640 1 -- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f11c406b4d0 msgr2=0x7f11c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.094+0000 7f11eca00640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f11c406b4d0 0x7f11c406d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f11d0036000 tx=0x7f11d0008030 comp rx=0 tx=0).stop 2024-03-07T06:51:26.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.094+0000 7f11eca00640 1 -- 172.21.15.133:0/2741949888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e8072d70 msgr2=0x7f11e81e68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.094+0000 7f11eca00640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e8072d70 0x7f11e81e68b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f11d800e840 tx=0x7f11d800ed00 comp rx=0 tx=0).stop 2024-03-07T06:51:26.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.094+0000 7f11eca00640 1 -- 172.21.15.133:0/2741949888 shutdown_connections 2024-03-07T06:51:26.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.094+0000 7f11eca00640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f11e815bd30 0x7f11e81465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.094+0000 7f11eca00640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f11c406b4d0 0x7f11c406d990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.094+0000 7f11eca00640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f11e80cf5d0 0x7f11e81e61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.094+0000 7f11eca00640 1 --2- 172.21.15.133:0/2741949888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e8072d70 0x7f11e81e68b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.094+0000 7f11eca00640 1 -- 172.21.15.133:0/2741949888 >> 172.21.15.133:0/2741949888 conn(0x7f11e80b2e90 msgr2=0x7f11e806ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.095+0000 7f11eca00640 1 -- 172.21.15.133:0/2741949888 shutdown_connections 2024-03-07T06:51:26.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.095+0000 7f11eca00640 1 -- 172.21.15.133:0/2741949888 wait complete. 2024-03-07T06:51:26.112 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-07T06:51:26.112 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:26.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.318+0000 7f68bac00640 1 Processor -- start 2024-03-07T06:51:26.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.318+0000 7f68bac00640 1 -- start start 2024-03-07T06:51:26.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.320+0000 7f68bac00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68b415fbb0 0x7f68b40cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.320+0000 7f68bac00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f68b40cdee0 0x7f68b415c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.320+0000 7f68bac00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b40731b0 con 0x7f68b4072950 2024-03-07T06:51:26.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.320+0000 7f68bac00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b4072f60 con 0x7f68b40cdee0 2024-03-07T06:51:26.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.320+0000 7f68bac00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b4073490 con 0x7f68b415fbb0 2024-03-07T06:51:26.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.321+0000 7f68b3e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68b415fbb0 0x7f68b40cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.321+0000 7f68b8c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f68b40cdee0 0x7f68b415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.321+0000 7f68b8c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f68b40cdee0 0x7f68b415c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58720/0 (socket says 172.21.15.133:58720) 2024-03-07T06:51:26.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.321+0000 7f68b8c00640 1 -- 172.21.15.133:0/754549809 learned_addr learned my addr 172.21.15.133:0/754549809 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:26.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.321+0000 7f68b3e00640 1 -- 172.21.15.133:0/754549809 >> v2:172.21.15.161:3300/0 conn(0x7f68b40cdee0 msgr2=0x7f68b415c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:26.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.321+0000 7f68b3e00640 1 --2- 172.21.15.133:0/754549809 >> v2:172.21.15.161:3300/0 conn(0x7f68b40cdee0 0x7f68b415c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.321+0000 7f68b3e00640 1 -- 172.21.15.133:0/754549809 >> v1:172.21.15.161:6789/0 conn(0x7f68b4072950 legacy=0x7f68b415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.321+0000 7f68b3e00640 1 -- 172.21.15.133:0/754549809 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b4072150 con 0x7f68b415fbb0 2024-03-07T06:51:26.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.321+0000 7f68b8c00640 1 --2- 172.21.15.133:0/754549809 >> v2:172.21.15.161:3300/0 conn(0x7f68b40cdee0 0x7f68b415c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T06:51:26.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.323+0000 7f68b3e00640 1 --2- 172.21.15.133:0/754549809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68b415fbb0 0x7f68b40cf7d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f68b40730e0 tx=0x7f68a402be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96632ddc39257d0b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.323+0000 7f68b3400640 1 -- 172.21.15.133:0/754549809 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4039070 con 0x7f68b415fbb0 2024-03-07T06:51:26.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.323+0000 7f68b3400640 1 -- 172.21.15.133:0/754549809 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a402cbb0 con 0x7f68b415fbb0 2024-03-07T06:51:26.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.323+0000 7f68b3400640 1 -- 172.21.15.133:0/754549809 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4034440 con 0x7f68b415fbb0 2024-03-07T06:51:26.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.324+0000 7f68bac00640 1 -- 172.21.15.133:0/754549809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68b415fbb0 msgr2=0x7f68b40cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.324+0000 7f68bac00640 1 --2- 172.21.15.133:0/754549809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68b415fbb0 0x7f68b40cf7d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f68b40730e0 tx=0x7f68a402be00 comp rx=0 tx=0).stop 2024-03-07T06:51:26.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.324+0000 7f68bac00640 1 -- 172.21.15.133:0/754549809 shutdown_connections 2024-03-07T06:51:26.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.324+0000 7f68bac00640 1 --2- 172.21.15.133:0/754549809 >> v2:172.21.15.161:3300/0 conn(0x7f68b40cdee0 0x7f68b415c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.324+0000 7f68bac00640 1 --2- 172.21.15.133:0/754549809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68b415fbb0 0x7f68b40cf7d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.324+0000 7f68bac00640 1 -- 172.21.15.133:0/754549809 >> 172.21.15.133:0/754549809 conn(0x7f68b40b3120 msgr2=0x7f68b40b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.324+0000 7f68bac00640 1 -- 172.21.15.133:0/754549809 shutdown_connections 2024-03-07T06:51:26.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.324+0000 7f68bac00640 1 -- 172.21.15.133:0/754549809 wait complete. 2024-03-07T06:51:26.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.326+0000 7f68bac00640 1 Processor -- start 2024-03-07T06:51:26.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.326+0000 7f68bac00640 1 -- start start 2024-03-07T06:51:26.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68bac00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b40cdee0 0x7f68b40aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68bac00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68b415fbb0 0x7f68b4144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68bac00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68b40b0b70 0x7f68b40b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68bac00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b4160980 con 0x7f68b40cdee0 2024-03-07T06:51:26.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68bac00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40ce2f0 con 0x7f68b40b0b70 2024-03-07T06:51:26.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68bac00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40b1a00 con 0x7f68b415fbb0 2024-03-07T06:51:26.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68b3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b40cdee0 0x7f68b40aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68b3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b40cdee0 0x7f68b40aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38476/0 (socket says 172.21.15.133:38476) 2024-03-07T06:51:26.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68b3e00640 1 -- 172.21.15.133:0/2164693264 learned_addr learned my addr 172.21.15.133:0/2164693264 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:26.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68ab400640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68b415fbb0 0x7f68b4144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68b8c00640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68b40b0b70 0x7f68b40b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68b3e00640 1 -- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68b415fbb0 msgr2=0x7f68b4144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68b3e00640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68b415fbb0 0x7f68b4144870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68b3e00640 1 -- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68b40b0b70 msgr2=0x7f68b40b5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68b3e00640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68b40b0b70 0x7f68b40b5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68b3e00640 1 -- 172.21.15.133:0/2164693264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b4151e50 con 0x7f68b40cdee0 2024-03-07T06:51:26.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.328+0000 7f68b8c00640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68b40b0b70 0x7f68b40b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:26.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.329+0000 7f68b3e00640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b40cdee0 0x7f68b40aff40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009e80 tx=0x7f68a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.329+0000 7f68b0c00640 1 -- 172.21.15.133:0/2164693264 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4039050 con 0x7f68b40cdee0 2024-03-07T06:51:26.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.329+0000 7f68b0c00640 1 -- 172.21.15.133:0/2164693264 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a4045070 con 0x7f68b40cdee0 2024-03-07T06:51:26.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.329+0000 7f68bac00640 1 -- 172.21.15.133:0/2164693264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b40aebc0 con 0x7f68b40cdee0 2024-03-07T06:51:26.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.329+0000 7f68b0c00640 1 -- 172.21.15.133:0/2164693264 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4009450 con 0x7f68b40cdee0 2024-03-07T06:51:26.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.329+0000 7f68bac00640 1 -- 172.21.15.133:0/2164693264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b40af180 con 0x7f68b40cdee0 2024-03-07T06:51:26.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.330+0000 7f68bac00640 1 -- 172.21.15.133:0/2164693264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b4062c90 con 0x7f68b40cdee0 2024-03-07T06:51:26.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.331+0000 7f68b0c00640 1 -- 172.21.15.133:0/2164693264 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f68a403e870 con 0x7f68b40cdee0 2024-03-07T06:51:26.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.332+0000 7f68b0c00640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f689006b4d0 0x7f689006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.332+0000 7f68b0c00640 1 -- 172.21.15.133:0/2164693264 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68a40ace20 con 0x7f68b40cdee0 2024-03-07T06:51:26.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.332+0000 7f68ab400640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f689006b4d0 0x7f689006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.333+0000 7f68ab400640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f689006b4d0 0x7f689006d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f68b4156f90 tx=0x7f689c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.336+0000 7f68b0c00640 1 -- 172.21.15.133:0/2164693264 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f68a404f020 con 0x7f68b40cdee0 2024-03-07T06:51:26.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.519+0000 7f68bac00640 1 -- 172.21.15.133:0/2164693264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68b40008d0 con 0x7f68b40cdee0 2024-03-07T06:51:26.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.521+0000 7f68b0c00640 1 -- 172.21.15.133:0/2164693264 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f68a4032e20 con 0x7f68b40cdee0 2024-03-07T06:51:26.521 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:26.521 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":21,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:16.174650+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:26.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.523+0000 7f68bac00640 1 -- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f689006b4d0 msgr2=0x7f689006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.523+0000 7f68bac00640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f689006b4d0 0x7f689006d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f68b4156f90 tx=0x7f689c009040 comp rx=0 tx=0).stop 2024-03-07T06:51:26.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.523+0000 7f68bac00640 1 -- 172.21.15.133:0/2164693264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b40cdee0 msgr2=0x7f68b40aff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.523+0000 7f68bac00640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b40cdee0 0x7f68b40aff40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009e80 tx=0x7f68a4004a40 comp rx=0 tx=0).stop 2024-03-07T06:51:26.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.523+0000 7f68bac00640 1 -- 172.21.15.133:0/2164693264 shutdown_connections 2024-03-07T06:51:26.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.523+0000 7f68bac00640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f689006b4d0 0x7f689006d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.523+0000 7f68bac00640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f68b40b0b70 0x7f68b40b5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.523+0000 7f68bac00640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f68b415fbb0 0x7f68b4144870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.523+0000 7f68bac00640 1 --2- 172.21.15.133:0/2164693264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b40cdee0 0x7f68b40aff40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.523+0000 7f68bac00640 1 -- 172.21.15.133:0/2164693264 >> 172.21.15.133:0/2164693264 conn(0x7f68b40b3120 msgr2=0x7f68b40740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.524+0000 7f68bac00640 1 -- 172.21.15.133:0/2164693264 shutdown_connections 2024-03-07T06:51:26.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.524+0000 7f68bac00640 1 -- 172.21.15.133:0/2164693264 wait complete. 2024-03-07T06:51:26.542 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-07T06:51:26.542 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-07T06:51:26.543 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-07T06:51:26.543 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-07T06:51:26.543 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-07T06:51:26.544 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-07T06:51:26.544 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-07T06:51:26.544 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-07T06:51:26.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.829+0000 7ff7aa400640 1 Processor -- start 2024-03-07T06:51:26.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.830+0000 7f860e600640 1 Processor -- start 2024-03-07T06:51:26.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.829+0000 7ff7aa400640 1 -- start start 2024-03-07T06:51:26.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.831+0000 7f860e600640 1 -- start start 2024-03-07T06:51:26.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.831+0000 7ff7aa400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff79c0e3160 0x7ff79c0e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.831+0000 7ff7aa400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff79c053cb0 0x7ff79c054120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.831+0000 7ff7aa400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff79c054660 con 0x7ff79c03c0a0 2024-03-07T06:51:26.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.831+0000 7ff7aa400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79c007b70 con 0x7ff79c053cb0 2024-03-07T06:51:26.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.831+0000 7ff7aa400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c058810 con 0x7ff79c0e3160 2024-03-07T06:51:26.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.832+0000 7ff7a9000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff79c0e3160 0x7ff79c0e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.832+0000 7ff7a9000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff79c0e3160 0x7ff79c0e7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:41894/0 (socket says 172.21.15.133:41894) 2024-03-07T06:51:26.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.832+0000 7ff7a9000640 1 -- 172.21.15.133:0/1855736143 learned_addr learned my addr 172.21.15.133:0/1855736143 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:26.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.833+0000 7ff7a9a00640 1 --2- 172.21.15.133:0/1855736143 >> v2:172.21.15.161:3300/0 conn(0x7ff79c053cb0 0x7ff79c054120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.833+0000 7ff7a9000640 1 -- 172.21.15.133:0/1855736143 >> v2:172.21.15.161:3300/0 conn(0x7ff79c053cb0 msgr2=0x7ff79c054120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:26.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.833+0000 7f860e600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f86081551f0 0x7f8608161d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.834+0000 7ff7a9000640 1 --2- 172.21.15.133:0/1855736143 >> v2:172.21.15.161:3300/0 conn(0x7ff79c053cb0 0x7ff79c054120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.834+0000 7f860e600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86080b43f0 con 0x7f8608154e10 2024-03-07T06:51:26.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.834+0000 7f860e600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86080b4690 con 0x7f86080d39e0 2024-03-07T06:51:26.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.834+0000 7f860e600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86080b4970 con 0x7f86081551f0 2024-03-07T06:51:26.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.834+0000 7ff7a9000640 1 -- 172.21.15.133:0/1855736143 >> v1:172.21.15.161:6789/0 conn(0x7ff79c03c0a0 legacy=0x7ff79c058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.834+0000 7ff7a9000640 1 -- 172.21.15.133:0/1855736143 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c058af0 con 0x7ff79c0e3160 2024-03-07T06:51:26.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.834+0000 7ff7a9a00640 1 --2- 172.21.15.133:0/1855736143 >> v2:172.21.15.161:3300/0 conn(0x7ff79c053cb0 0x7ff79c054120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:26.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.834+0000 7f860d200640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8608154e10 0x7f86080d32d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44104/0 (socket says 172.21.15.133:44104) 2024-03-07T06:51:26.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.835+0000 7f860d200640 1 -- 172.21.15.133:0/2962945834 learned_addr learned my addr 172.21.15.133:0/2962945834 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:26.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.834+0000 7f8607e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f86081551f0 0x7f8608161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.836+0000 7f8607400640 1 -- 172.21.15.133:0/2962945834 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204218732 0 0) 0x7f86080b4690 con 0x7f86080d39e0 2024-03-07T06:51:26.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.836+0000 7f8607400640 1 -- 172.21.15.133:0/2962945834 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85ec003520 con 0x7f86080d39e0 2024-03-07T06:51:26.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.836+0000 7f8607400640 1 -- 172.21.15.133:0/2962945834 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211045820 0 0) 0x7f86080b43f0 con 0x7f8608154e10 2024-03-07T06:51:26.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.836+0000 7f8607e00640 1 -- 172.21.15.133:0/2962945834 >> v1:172.21.15.161:6789/0 conn(0x7f86080d39e0 legacy=0x7f8608162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.836+0000 7f8607e00640 1 -- 172.21.15.133:0/2962945834 >> v1:172.21.15.133:6789/0 conn(0x7f8608154e10 legacy=0x7f86080d32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.836+0000 7f8607e00640 1 -- 172.21.15.133:0/2962945834 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86080b4c50 con 0x7f86081551f0 2024-03-07T06:51:26.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.837+0000 7f0157000640 1 Processor -- start 2024-03-07T06:51:26.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.837+0000 7ff7a9000640 1 --2- 172.21.15.133:0/1855736143 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff79c0e3160 0x7ff79c0e7780 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4069c20 tx=0x7ff7a40a6a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ed3acf31adf542 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.837+0000 7f0157000640 1 -- start start 2024-03-07T06:51:26.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.837+0000 7f8607e00640 1 --2- 172.21.15.133:0/2962945834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86081551f0 0x7f8608161d50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85f80097a0 tx=0x7f85f802b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f89743724340e7ba server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.837+0000 7ff7a3400640 1 -- 172.21.15.133:0/1855736143 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40aa4c0 con 0x7ff79c0e3160 2024-03-07T06:51:26.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.838+0000 7ff7a3400640 1 -- 172.21.15.133:0/1855736143 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a40aa660 con 0x7ff79c0e3160 2024-03-07T06:51:26.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.838+0000 7ff7a3400640 1 -- 172.21.15.133:0/1855736143 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40aaa90 con 0x7ff79c0e3160 2024-03-07T06:51:26.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.838+0000 7f8607400640 1 -- 172.21.15.133:0/2962945834 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f802f810 con 0x7f86081551f0 2024-03-07T06:51:26.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.838+0000 7f8607400640 1 -- 172.21.15.133:0/2962945834 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f802f9b0 con 0x7f86081551f0 2024-03-07T06:51:26.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.838+0000 7f8607400640 1 -- 172.21.15.133:0/2962945834 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f802fd60 con 0x7f86081551f0 2024-03-07T06:51:26.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.839+0000 7ff7aa400640 1 -- 172.21.15.133:0/1855736143 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff79c0e3160 msgr2=0x7ff79c0e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.839+0000 7ff7aa400640 1 --2- 172.21.15.133:0/1855736143 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff79c0e3160 0x7ff79c0e7780 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4069c20 tx=0x7ff7a40a6a70 comp rx=0 tx=0).stop 2024-03-07T06:51:26.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.839+0000 7f860e600640 1 -- 172.21.15.133:0/2962945834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86081551f0 msgr2=0x7f8608161d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.839+0000 7f860e600640 1 --2- 172.21.15.133:0/2962945834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86081551f0 0x7f8608161d50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f85f80097a0 tx=0x7f85f802b9f0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.839+0000 7f0157000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f01500b70d0 0x7f01500b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.839+0000 7f0157000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0150143810 con 0x7f01500d1af0 2024-03-07T06:51:26.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.839+0000 7f0157000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0150143af0 con 0x7f0150140cb0 2024-03-07T06:51:26.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.839+0000 7f0157000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01501435c0 con 0x7f01500b70d0 2024-03-07T06:51:26.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.839+0000 7f860e600640 1 -- 172.21.15.133:0/2962945834 shutdown_connections 2024-03-07T06:51:26.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.839+0000 7f860e600640 1 --2- 172.21.15.133:0/2962945834 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86081551f0 0x7f8608161d50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.839+0000 7f860e600640 1 -- 172.21.15.133:0/2962945834 >> 172.21.15.133:0/2962945834 conn(0x7f8608070400 msgr2=0x7f86080d2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.839+0000 7f860e600640 1 -- 172.21.15.133:0/2962945834 shutdown_connections 2024-03-07T06:51:26.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.839+0000 7f860e600640 1 -- 172.21.15.133:0/2962945834 wait complete. 2024-03-07T06:51:26.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.840+0000 7f0155200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f01500b70d0 0x7f01500b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.840+0000 7f0156600640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f01500d1af0 0x7f01500d1f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44108/0 (socket says 172.21.15.133:44108) 2024-03-07T06:51:26.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.840+0000 7f0156600640 1 -- 172.21.15.133:0/869018641 learned_addr learned my addr 172.21.15.133:0/869018641 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:26.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.841+0000 7ff7aa400640 1 -- 172.21.15.133:0/1855736143 shutdown_connections 2024-03-07T06:51:26.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.841+0000 7ff7aa400640 1 --2- 172.21.15.133:0/1855736143 >> v2:172.21.15.161:3300/0 conn(0x7ff79c053cb0 0x7ff79c054120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.841+0000 7ff7aa400640 1 --2- 172.21.15.133:0/1855736143 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff79c0e3160 0x7ff79c0e7780 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.841+0000 7ff7aa400640 1 -- 172.21.15.133:0/1855736143 >> 172.21.15.133:0/1855736143 conn(0x7ff79c058ee0 msgr2=0x7ff79c0592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.841+0000 7f0155200640 1 -- 172.21.15.133:0/869018641 >> v1:172.21.15.161:6789/0 conn(0x7f0150140cb0 legacy=0x7f0150162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.841+0000 7f0155200640 1 -- 172.21.15.133:0/869018641 >> v1:172.21.15.133:6789/0 conn(0x7f01500d1af0 legacy=0x7f01500d1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.841+0000 7f0155200640 1 -- 172.21.15.133:0/869018641 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0150143dd0 con 0x7f01500b70d0 2024-03-07T06:51:26.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.842+0000 7ff7aa400640 1 -- 172.21.15.133:0/1855736143 shutdown_connections 2024-03-07T06:51:26.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.842+0000 7ff7aa400640 1 -- 172.21.15.133:0/1855736143 wait complete. 2024-03-07T06:51:26.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.843+0000 7f860e600640 1 Processor -- start 2024-03-07T06:51:26.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.843+0000 7f860e600640 1 -- start start 2024-03-07T06:51:26.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.843+0000 7f0155200640 1 --2- 172.21.15.133:0/869018641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01500b70d0 0x7f01500b7560 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f013c0097a0 tx=0x7f013c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bbe0929c39f8240 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.844+0000 7f014fe00640 1 -- 172.21.15.133:0/869018641 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c037070 con 0x7f01500b70d0 2024-03-07T06:51:26.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.844+0000 7f014fe00640 1 -- 172.21.15.133:0/869018641 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c030710 con 0x7f01500b70d0 2024-03-07T06:51:26.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.844+0000 7f014fe00640 1 -- 172.21.15.133:0/869018641 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c030ae0 con 0x7f01500b70d0 2024-03-07T06:51:26.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.844+0000 7ff7aa400640 1 Processor -- start 2024-03-07T06:51:26.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.844+0000 7f8dd8200640 1 Processor -- start 2024-03-07T06:51:26.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.845+0000 7f8dd8200640 1 -- start start 2024-03-07T06:51:26.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.845+0000 7ff7aa400640 1 -- start start 2024-03-07T06:51:26.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.844+0000 7f860e600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86080d39e0 0x7f8608137510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.845+0000 7f860e600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8608154e10 0x7f860814e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.845+0000 7f860e600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86081551f0 0x7f860814e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.845+0000 7f0157000640 1 -- 172.21.15.133:0/869018641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01500b70d0 msgr2=0x7f01500b7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.845+0000 7f860e600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8608168d80 con 0x7f86080d39e0 2024-03-07T06:51:26.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.845+0000 7f860e600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8608168c00 con 0x7f8608154e10 2024-03-07T06:51:26.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.845+0000 7f860e600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8608168f00 con 0x7f86081551f0 2024-03-07T06:51:26.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.845+0000 7f0157000640 1 --2- 172.21.15.133:0/869018641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01500b70d0 0x7f01500b7560 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f013c0097a0 tx=0x7f013c02b790 comp rx=0 tx=0).stop 2024-03-07T06:51:26.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.845+0000 7f0157000640 1 -- 172.21.15.133:0/869018641 shutdown_connections 2024-03-07T06:51:26.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.845+0000 7f0157000640 1 --2- 172.21.15.133:0/869018641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01500b70d0 0x7f01500b7560 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.845+0000 7f0157000640 1 -- 172.21.15.133:0/869018641 >> 172.21.15.133:0/869018641 conn(0x7f01500674e0 msgr2=0x7f01500d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.845+0000 7f0157000640 1 -- 172.21.15.133:0/869018641 shutdown_connections 2024-03-07T06:51:26.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.846+0000 7f860d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86080d39e0 0x7f8608137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.846+0000 7f860d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86080d39e0 0x7f8608137510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38492/0 (socket says 172.21.15.133:38492) 2024-03-07T06:51:26.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.846+0000 7f860d200640 1 -- 172.21.15.133:0/2698880532 learned_addr learned my addr 172.21.15.133:0/2698880532 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:26.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.846+0000 7f8607e00640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8608154e10 0x7f860814e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.846+0000 7ff7aa400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff79c03c0a0 0x7ff79c0d53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.846+0000 7ff7aa400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c053cb0 0x7ff79c16cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.846+0000 7ff7aa400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff79c0e3160 0x7ff79c04d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.846+0000 7ff7aa400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c0e3540 con 0x7ff79c03c0a0 2024-03-07T06:51:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.846+0000 7ff7aa400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c006640 con 0x7ff79c053cb0 2024-03-07T06:51:26.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.846+0000 7ff7aa400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c006b60 con 0x7ff79c0e3160 2024-03-07T06:51:26.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.846+0000 7f0157000640 1 -- 172.21.15.133:0/869018641 wait complete. 2024-03-07T06:51:26.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.846+0000 7ff5c8400640 1 Processor -- start 2024-03-07T06:51:26.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7ff7a9000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff79c03c0a0 0x7ff79c0d53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.846+0000 7f860dc00640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86081551f0 0x7f860814e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7ff7a9000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff79c03c0a0 0x7ff79c0d53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38502/0 (socket says 172.21.15.133:38502) 2024-03-07T06:51:26.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7ff7a9000640 1 -- 172.21.15.133:0/2042785242 learned_addr learned my addr 172.21.15.133:0/2042785242 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:26.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7f8dd8200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8dd0143d10 0x7f8dd00cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7f8dd8200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd00b1110 0x7f8dd0072400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7f8dd8200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd00afe20 con 0x7f8dd00ce420 2024-03-07T06:51:26.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7f8dd8200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd00ce880 con 0x7f8dd0143d10 2024-03-07T06:51:26.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7f8dd8200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd0071920 con 0x7f8dd00b1110 2024-03-07T06:51:26.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7ff7a3e00640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c053cb0 0x7ff79c16cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7ff7a9a00640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff79c0e3160 0x7ff79c04d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7ff5c8400640 1 -- start start 2024-03-07T06:51:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7f8607e00640 1 -- 172.21.15.133:0/2698880532 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86081551f0 msgr2=0x7f860814e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7f8607e00640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86081551f0 0x7f860814e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7f8607e00640 1 -- 172.21.15.133:0/2698880532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86080d39e0 msgr2=0x7f8608137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7f8607e00640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86080d39e0 0x7f8608137510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7f8607e00640 1 -- 172.21.15.133:0/2698880532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8608136470 con 0x7f8608154e10 2024-03-07T06:51:26.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7ff7a9000640 1 -- 172.21.15.133:0/2042785242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff79c0e3160 msgr2=0x7ff79c04d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7ff7a9000640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff79c0e3160 0x7ff79c04d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7ff7a9000640 1 -- 172.21.15.133:0/2042785242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c053cb0 msgr2=0x7ff79c16cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7ff7a9000640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff79c053cb0 0x7ff79c16cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.847+0000 7ff7a9000640 1 -- 172.21.15.133:0/2042785242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c16d530 con 0x7ff79c03c0a0 2024-03-07T06:51:26.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.848+0000 7ff7a9a00640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff79c0e3160 0x7ff79c04d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:51:26.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.848+0000 7ff7a9000640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff79c03c0a0 0x7ff79c0d53d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4050ba0 tx=0x7ff7a40738f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.848+0000 7f0157000640 1 Processor -- start 2024-03-07T06:51:26.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.848+0000 7f2f65c00640 1 Processor -- start 2024-03-07T06:51:26.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.848+0000 7f8607e00640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8608154e10 0x7f860814e160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8608071e50 tx=0x7f85f80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.848+0000 7ff7a0c00640 1 -- 172.21.15.133:0/2042785242 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40c3070 con 0x7ff79c03c0a0 2024-03-07T06:51:26.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.848+0000 7ff7a0c00640 1 -- 172.21.15.133:0/2042785242 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a40aabe0 con 0x7ff79c03c0a0 2024-03-07T06:51:26.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.848+0000 7ff7a0c00640 1 -- 172.21.15.133:0/2042785242 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40b2400 con 0x7ff79c03c0a0 2024-03-07T06:51:26.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.848+0000 7f0157000640 1 -- start start 2024-03-07T06:51:26.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.849+0000 7ff7aa400640 1 -- 172.21.15.133:0/2042785242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79c16d810 con 0x7ff79c03c0a0 2024-03-07T06:51:26.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.849+0000 7ff7aa400640 1 -- 172.21.15.133:0/2042785242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79c16ddd0 con 0x7ff79c03c0a0 2024-03-07T06:51:26.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.849+0000 7f2f65c00640 1 -- start start 2024-03-07T06:51:26.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.849+0000 7f8604c00640 1 -- 172.21.15.133:0/2698880532 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8047070 con 0x7f8608154e10 2024-03-07T06:51:26.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.849+0000 7f8604c00640 1 -- 172.21.15.133:0/2698880532 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f8004250 con 0x7f8608154e10 2024-03-07T06:51:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.849+0000 7f8604c00640 1 -- 172.21.15.133:0/2698880532 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8003610 con 0x7f8608154e10 2024-03-07T06:51:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.849+0000 7f860e600640 1 -- 172.21.15.133:0/2698880532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8608136750 con 0x7f8608154e10 2024-03-07T06:51:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.849+0000 7f860d200640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86080d39e0 0x7f8608137510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T06:51:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.849+0000 7f860e600640 1 -- 172.21.15.133:0/2698880532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8608136d10 con 0x7f8608154e10 2024-03-07T06:51:26.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.849+0000 7f860dc00640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86081551f0 0x7f860814e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:51:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.850+0000 7f0157000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01500b70d0 0x7f015014e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.850+0000 7f0157000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01500d1af0 0x7f015013f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.850+0000 7f0157000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0150140cb0 0x7f01500b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7f0157000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0150161bb0 con 0x7f01500b70d0 2024-03-07T06:51:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7f0157000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0150144420 con 0x7f0150140cb0 2024-03-07T06:51:26.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7f0157000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0150161d30 con 0x7f01500d1af0 2024-03-07T06:51:26.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.850+0000 7f1f52000640 1 Processor -- start 2024-03-07T06:51:26.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.850+0000 7ff5c8400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5c00b52c0 0x7ff5c0162c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.848+0000 7f8dd5800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8dd0143d10 0x7f8dd00cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.848+0000 7f8dd6200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd00b1110 0x7f8dd0072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.848+0000 7f8dd5800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8dd0143d10 0x7f8dd00cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38518/0 (socket says 172.21.15.133:38518) 2024-03-07T06:51:26.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.848+0000 7f8dd5800640 1 -- 172.21.15.133:0/2170232721 learned_addr learned my addr 172.21.15.133:0/2170232721 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:26.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7ff5c8400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff5c0071b20 0x7ff5c006e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7ff5c8400640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c0163740 con 0x7ff5c0163190 2024-03-07T06:51:26.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7ff5c8400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c0163e10 con 0x7ff5c0071b20 2024-03-07T06:51:26.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7ff5c8400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c0163a20 con 0x7ff5c00b52c0 2024-03-07T06:51:26.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7ff5c5000640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff5c0163190 0x7ff5c0071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44126/0 (socket says 172.21.15.133:44126) 2024-03-07T06:51:26.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7ff5c5000640 1 -- 172.21.15.133:0/2861972310 learned_addr learned my addr 172.21.15.133:0/2861972310 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:26.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.850+0000 7f2f65c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2f601625d0 0x7f2f600b71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7ff7aa400640 1 -- 172.21.15.133:0/2042785242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff79c16e1b0 con 0x7ff79c03c0a0 2024-03-07T06:51:26.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.850+0000 7f2f65c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2f601629b0 0x7f2f6006b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.850+0000 7f2f65c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f600d21c0 con 0x7f2f600b15d0 2024-03-07T06:51:26.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.850+0000 7f2f65c00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f600d2dd0 con 0x7f2f601629b0 2024-03-07T06:51:26.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.850+0000 7f2f65c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f600d24a0 con 0x7f2f601625d0 2024-03-07T06:51:26.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.852+0000 7f2f5f400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f2f600b15d0 0x7f2f601663b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.133:60152/0 (socket says 172.21.15.133:60152) 2024-03-07T06:51:26.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.852+0000 7f2f5f400640 1 -- 172.21.15.133:0/557278497 learned_addr learned my addr 172.21.15.133:0/557278497 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:26.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.852+0000 7f0155200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01500d1af0 0x7f015013f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.852+0000 7f2f5ea00640 1 --2- 172.21.15.133:0/557278497 >> v2:172.21.15.161:3300/0 conn(0x7f2f601625d0 0x7f2f600b71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7f8dd5800640 1 -- 172.21.15.133:0/2170232721 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd00b1110 msgr2=0x7f8dd0072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:26.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.852+0000 7f0155c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01500b70d0 0x7f015014e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.852+0000 7ff5bfe00640 1 -- 172.21.15.133:0/2861972310 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119214694 0 0) 0x7ff5c0163740 con 0x7ff5c0163190 2024-03-07T06:51:26.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7f8dd5800640 1 --2- 172.21.15.133:0/2170232721 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd00b1110 0x7f8dd0072400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7f8dd5800640 1 -- 172.21.15.133:0/2170232721 >> v1:172.21.15.133:6789/0 conn(0x7f8dd00ce420 legacy=0x7f8dd00b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.852+0000 7ff5bfe00640 1 -- 172.21.15.133:0/2861972310 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5a8003520 con 0x7ff5c0163190 2024-03-07T06:51:26.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.852+0000 7f0155c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01500b70d0 0x7f015014e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38528/0 (socket says 172.21.15.133:38528) 2024-03-07T06:51:26.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.852+0000 7f2f5e000640 1 --2- 172.21.15.133:0/557278497 >> v2:172.21.15.133:3300/0 conn(0x7f2f601629b0 0x7f2f6006b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.852+0000 7f0155c00640 1 -- 172.21.15.133:0/1487482422 learned_addr learned my addr 172.21.15.133:0/1487482422 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:26.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7f8dd5800640 1 -- 172.21.15.133:0/2170232721 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd00716b0 con 0x7f8dd0143d10 2024-03-07T06:51:26.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7f2f5d600640 1 -- 172.21.15.133:0/557278497 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872498249 0 0) 0x7f2f600d21c0 con 0x7f2f600b15d0 2024-03-07T06:51:26.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7f2f5d600640 1 -- 172.21.15.133:0/557278497 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f44003520 con 0x7f2f600b15d0 2024-03-07T06:51:26.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7f2f5ea00640 1 -- 172.21.15.133:0/557278497 >> v2:172.21.15.133:3300/0 conn(0x7f2f601629b0 msgr2=0x7f2f6006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7f2f5ea00640 1 --2- 172.21.15.133:0/557278497 >> v2:172.21.15.133:3300/0 conn(0x7f2f601629b0 0x7f2f6006b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7f2f5ea00640 1 -- 172.21.15.133:0/557278497 >> v1:172.21.15.161:6789/0 conn(0x7f2f600b15d0 legacy=0x7f2f601663b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7f2f5ea00640 1 -- 172.21.15.133:0/557278497 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f600d2780 con 0x7f2f601625d0 2024-03-07T06:51:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7f860e600640 1 -- 172.21.15.133:0/2698880532 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8608062c90 con 0x7f8608154e10 2024-03-07T06:51:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7f8604c00640 1 -- 172.21.15.133:0/2698880532 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f85f80037b0 con 0x7f8608154e10 2024-03-07T06:51:26.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7f0156600640 1 --2- 172.21.15.133:0/1487482422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0150140cb0 0x7f01500b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7ff5c6400640 1 --2- 172.21.15.133:0/2861972310 >> v2:172.21.15.161:3300/0 conn(0x7ff5c0071b20 0x7ff5c006e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7ff5c5a00640 1 --2- 172.21.15.133:0/2861972310 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5c00b52c0 0x7ff5c0162c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.851+0000 7f1f52000640 1 -- start start 2024-03-07T06:51:26.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7f1f52000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1f4c15a3e0 0x7f1f4c072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7ff5bfe00640 1 -- 172.21.15.133:0/2861972310 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 310689744 0 0) 0x7ff5a8003520 con 0x7ff5c0163190 2024-03-07T06:51:26.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff5bfe00640 1 -- 172.21.15.133:0/2861972310 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5c00b52c0 msgr2=0x7ff5c0162c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:26.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff5bfe00640 1 --2- 172.21.15.133:0/2861972310 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5c00b52c0 0x7ff5c0162c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff5bfe00640 1 -- 172.21.15.133:0/2861972310 >> v2:172.21.15.161:3300/0 conn(0x7ff5c0071b20 msgr2=0x7ff5c006e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:26.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff5bfe00640 1 --2- 172.21.15.133:0/2861972310 >> v2:172.21.15.161:3300/0 conn(0x7ff5c0071b20 0x7ff5c006e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff5bfe00640 1 -- 172.21.15.133:0/2861972310 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c0164060 con 0x7ff5c0163190 2024-03-07T06:51:26.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7f0155c00640 1 -- 172.21.15.133:0/1487482422 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01500d1af0 msgr2=0x7f015013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff5bfe00640 1 -- 172.21.15.133:0/2861972310 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7ff5ac001670 con 0x7ff5c0163190 2024-03-07T06:51:26.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff5bfe00640 1 -- 172.21.15.133:0/2861972310 >> v1:172.21.15.133:6789/0 conn(0x7ff5c0163190 legacy=0x7ff5c0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7f0155c00640 1 --2- 172.21.15.133:0/1487482422 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01500d1af0 0x7f015013f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7f0155c00640 1 -- 172.21.15.133:0/1487482422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0150140cb0 msgr2=0x7f01500b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7f0155c00640 1 --2- 172.21.15.133:0/1487482422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0150140cb0 0x7f01500b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7f0155c00640 1 -- 172.21.15.133:0/1487482422 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015014f840 con 0x7f01500b70d0 2024-03-07T06:51:26.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7f1f52000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1f4c06d2a0 0x7f1f4c06d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7f1f52000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4c072c50 con 0x7f1f4c0acd00 2024-03-07T06:51:26.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7f1f52000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f4c0aea50 con 0x7f1f4c06d2a0 2024-03-07T06:51:26.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.853+0000 7f1f52000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0ad2a0 con 0x7f1f4c15a3e0 2024-03-07T06:51:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff5bfe00640 1 --2- 172.21.15.133:0/2861972310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5a80039b0 0x7ff5a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff5bfe00640 1 --2- 172.21.15.133:0/2861972310 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5c00b52c0 0x7ff5a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff5bfe00640 1 --2- 172.21.15.133:0/2861972310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5a8004d40 0x7ff5a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff5bfe00640 1 -- 172.21.15.133:0/2861972310 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c0163e10 con 0x7ff5a80039b0 2024-03-07T06:51:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff5bfe00640 1 -- 172.21.15.133:0/2861972310 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c0163a20 con 0x7ff5a8004d40 2024-03-07T06:51:26.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff5bfe00640 1 -- 172.21.15.133:0/2861972310 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a800bb50 con 0x7ff5c00b52c0 2024-03-07T06:51:26.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff7a0c00640 1 -- 172.21.15.133:0/2042785242 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff7a40b25a0 con 0x7ff79c03c0a0 2024-03-07T06:51:26.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7ff5c6400640 1 --2- 172.21.15.133:0/2861972310 >> v2:172.21.15.161:3300/0 conn(0x7ff5c0071b20 0x7ff5c006e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:26.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.854+0000 7f8dd5800640 1 --2- 172.21.15.133:0/2170232721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd0143d10 0x7f8dd00cdee0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009870 tx=0x7f8dbc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864ed9c02c3a86cf server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7f8604c00640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f85e806b520 0x7f85e806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7f8604c00640 1 -- 172.21.15.133:0/2698880532 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85f80ab8a0 con 0x7f8608154e10 2024-03-07T06:51:26.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7f2f5ea00640 1 --2- 172.21.15.133:0/557278497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f601625d0 0x7f2f600b71d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f6006c130 tx=0x7f2f5802be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97797637709cf976 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7ff5c5000640 1 --2- 172.21.15.133:0/2861972310 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5c00b52c0 0x7ff5a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7ff5c6400640 1 --2- 172.21.15.133:0/2861972310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5a8004d40 0x7ff5a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7f8604c00640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.133:6814/316026862,v1:172.21.15.133:6819/316026862] conn(0x7f85e8073720 0x7f85e8075b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7f8604c00640 1 -- 172.21.15.133:0/2698880532 --> [v2:172.21.15.133:6814/316026862,v1:172.21.15.133:6819/316026862] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85f8008eb0 con 0x7f85e8073720 2024-03-07T06:51:26.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7f0155c00640 1 --2- 172.21.15.133:0/1487482422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01500b70d0 0x7f015014e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f014400c780 tx=0x7f014400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7ff5c5a00640 1 --2- 172.21.15.133:0/2861972310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5a80039b0 0x7ff5a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7f860d200640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f85e806b520 0x7f85e806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7ff7a0c00640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff78406b4d0 0x7ff78406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7f8dcfe00640 1 -- 172.21.15.133:0/2170232721 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc034070 con 0x7f8dd0143d10 2024-03-07T06:51:26.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7ff7a0c00640 1 -- 172.21.15.133:0/2042785242 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7a4126a90 con 0x7ff79c03c0a0 2024-03-07T06:51:26.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7f1f4aa00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1f4c0acd00 0x7f1f4c0ae340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44134/0 (socket says 172.21.15.133:44134) 2024-03-07T06:51:26.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7f1f4be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1f4c06d2a0 0x7f1f4c06d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7f1f4b400640 1 --2- 172.21.15.133:0/3827871970 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1f4c15a3e0 0x7f1f4c072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7f8dcfe00640 1 -- 172.21.15.133:0/2170232721 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc030840 con 0x7f8dd0143d10 2024-03-07T06:51:26.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7ff7a0c00640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.133:6800/1149391957,v1:172.21.15.133:6801/1149391957] conn(0x7ff7840736f0 0x7ff784075b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7ff7a0c00640 1 -- 172.21.15.133:0/2042785242 --> [v2:172.21.15.133:6800/1149391957,v1:172.21.15.133:6801/1149391957] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7840761c0 con 0x7ff7840736f0 2024-03-07T06:51:26.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7ff7a0c00640 1 -- 172.21.15.133:0/2042785242 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7a412b2a0 con 0x7ff79c03c0a0 2024-03-07T06:51:26.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7f2f5d600640 1 -- 172.21.15.133:0/557278497 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58039070 con 0x7f2f601625d0 2024-03-07T06:51:26.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.855+0000 7f1f4aa00640 1 -- 172.21.15.133:0/3827871970 learned_addr learned my addr 172.21.15.133:0/3827871970 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:26.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7f2f5d600640 1 -- 172.21.15.133:0/557278497 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5802cbe0 con 0x7f2f601625d0 2024-03-07T06:51:26.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7f2f5d600640 1 -- 172.21.15.133:0/557278497 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58034440 con 0x7f2f601625d0 2024-03-07T06:51:26.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7ff7a3e00640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff78406b4d0 0x7ff78406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7ff7a9a00640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.133:6800/1149391957,v1:172.21.15.133:6801/1149391957] conn(0x7ff7840736f0 0x7ff784075b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7f8dcfe00640 1 -- 172.21.15.133:0/2170232721 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc030c10 con 0x7f8dd0143d10 2024-03-07T06:51:26.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7f5eeec00640 1 Processor -- start 2024-03-07T06:51:26.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7f8604c00640 1 -- 172.21.15.133:0/2698880532 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85f80b2050 con 0x7f8608154e10 2024-03-07T06:51:26.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7f1f4b400640 1 -- 172.21.15.133:0/3827871970 >> v2:172.21.15.161:3300/0 conn(0x7f1f4c06d2a0 msgr2=0x7f1f4c06d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:26.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7f1f4b400640 1 --2- 172.21.15.133:0/3827871970 >> v2:172.21.15.161:3300/0 conn(0x7f1f4c06d2a0 0x7f1f4c06d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7f1f4b400640 1 -- 172.21.15.133:0/3827871970 >> v1:172.21.15.133:6789/0 conn(0x7f1f4c0acd00 legacy=0x7f1f4c0ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7f1f4b400640 1 -- 172.21.15.133:0/3827871970 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0ad580 con 0x7f1f4c15a3e0 2024-03-07T06:51:26.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.857+0000 7f860dc00640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.133:6814/316026862,v1:172.21.15.133:6819/316026862] conn(0x7f85e8073720 0x7f85e8075b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.857+0000 7f014d600640 1 -- 172.21.15.133:0/1487482422 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144016020 con 0x7f01500b70d0 2024-03-07T06:51:26.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.857+0000 7f014d600640 1 -- 172.21.15.133:0/1487482422 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01440050d0 con 0x7f01500b70d0 2024-03-07T06:51:26.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.857+0000 7f0157000640 1 -- 172.21.15.133:0/1487482422 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01501581e0 con 0x7f01500b70d0 2024-03-07T06:51:26.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7ff5c5a00640 1 -- 172.21.15.133:0/2861972310 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5c00b52c0 msgr2=0x7ff5a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:26.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7ff5c5a00640 1 --2- 172.21.15.133:0/2861972310 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5c00b52c0 0x7ff5a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7ff5c5a00640 1 -- 172.21.15.133:0/2861972310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5a8004d40 msgr2=0x7ff5a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:26.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7ff5c5a00640 1 --2- 172.21.15.133:0/2861972310 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5a8004d40 0x7ff5a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.856+0000 7ff5c5a00640 1 -- 172.21.15.133:0/2861972310 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c0164060 con 0x7ff5a80039b0 2024-03-07T06:51:26.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.857+0000 7f5eeec00640 1 -- start start 2024-03-07T06:51:26.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.857+0000 7ff7a3e00640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff78406b4d0 0x7ff78406d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff79c052970 tx=0x7ff790009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.857+0000 7ff7a9a00640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.133:6800/1149391957,v1:172.21.15.133:6801/1149391957] conn(0x7ff7840736f0 0x7ff784075b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f8dd8200640 1 -- 172.21.15.133:0/2170232721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd0143d10 msgr2=0x7f8dd00cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f2f65c00640 1 -- 172.21.15.133:0/557278497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f601625d0 msgr2=0x7f2f600b71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f2f65c00640 1 --2- 172.21.15.133:0/557278497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f601625d0 0x7f2f600b71d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f6006c130 tx=0x7f2f5802be30 comp rx=0 tx=0).stop 2024-03-07T06:51:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f8dd8200640 1 --2- 172.21.15.133:0/2170232721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd0143d10 0x7f8dd00cdee0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009870 tx=0x7f8dbc02b980 comp rx=0 tx=0).stop 2024-03-07T06:51:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.857+0000 7f0157000640 1 -- 172.21.15.133:0/1487482422 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01500b4e70 con 0x7f01500b70d0 2024-03-07T06:51:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.857+0000 7f014d600640 1 -- 172.21.15.133:0/1487482422 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144007020 con 0x7f01500b70d0 2024-03-07T06:51:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f0157000640 1 -- 172.21.15.133:0/1487482422 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f012c001030 con 0x7f01500b70d0 2024-03-07T06:51:26.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f2f65c00640 1 -- 172.21.15.133:0/557278497 shutdown_connections 2024-03-07T06:51:26.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f2f65c00640 1 --2- 172.21.15.133:0/557278497 >> v2:172.21.15.133:3300/0 conn(0x7f2f601629b0 0x7f2f6006b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f2f65c00640 1 --2- 172.21.15.133:0/557278497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f601625d0 0x7f2f600b71d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f2f65c00640 1 -- 172.21.15.133:0/557278497 >> 172.21.15.133:0/557278497 conn(0x7f2f6006d4c0 msgr2=0x7f2f60066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7ff5c5a00640 1 --2- 172.21.15.133:0/2861972310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5a80039b0 0x7ff5a8003dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400ba30 tx=0x7ff5a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b4638a0115c059 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f860d200640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f85e806b520 0x7f85e806d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8600005c00 tx=0x7f8600009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f8dd8200640 1 -- 172.21.15.133:0/2170232721 shutdown_connections 2024-03-07T06:51:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f2f65c00640 1 -- 172.21.15.133:0/557278497 shutdown_connections 2024-03-07T06:51:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f2f65c00640 1 -- 172.21.15.133:0/557278497 wait complete. 2024-03-07T06:51:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f8dd8200640 1 --2- 172.21.15.133:0/2170232721 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd00b1110 0x7f8dd0072400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f8dd8200640 1 --2- 172.21.15.133:0/2170232721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd0143d10 0x7f8dd00cdee0 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009870 tx=0x7f8dbc02b980 comp rx=0 tx=0).stop 2024-03-07T06:51:26.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f8dd8200640 1 -- 172.21.15.133:0/2170232721 >> 172.21.15.133:0/2170232721 conn(0x7f8dd00b3120 msgr2=0x7f8dd00b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.859+0000 7f860dc00640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.133:6814/316026862,v1:172.21.15.133:6819/316026862] conn(0x7f85e8073720 0x7f85e8075b40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.859+0000 7f8dd8200640 1 -- 172.21.15.133:0/2170232721 shutdown_connections 2024-03-07T06:51:26.909 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.859+0000 7ff5bfe00640 1 -- 172.21.15.133:0/2861972310 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400b600 con 0x7ff5a80039b0 2024-03-07T06:51:26.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.859+0000 7ff5bfe00640 1 -- 172.21.15.133:0/2861972310 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b4038070 con 0x7ff5a80039b0 2024-03-07T06:51:26.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.859+0000 7ff5bfe00640 1 -- 172.21.15.133:0/2861972310 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4033880 con 0x7ff5a80039b0 2024-03-07T06:51:26.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.858+0000 7f1f4b400640 1 --2- 172.21.15.133:0/3827871970 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1f4c15a3e0 0x7f1f4c072710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00dbd0 tx=0x7f1f3c02fcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebc3426e91738369 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.860+0000 7f8dd8200640 1 -- 172.21.15.133:0/2170232721 wait complete. 2024-03-07T06:51:26.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.860+0000 7f1f4a000640 1 -- 172.21.15.133:0/3827871970 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c036070 con 0x7f1f4c15a3e0 2024-03-07T06:51:26.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.860+0000 7f1f4a000640 1 -- 172.21.15.133:0/3827871970 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c030a80 con 0x7f1f4c15a3e0 2024-03-07T06:51:26.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.860+0000 7f1f4a000640 1 -- 172.21.15.133:0/3827871970 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c030e50 con 0x7f1f4c15a3e0 2024-03-07T06:51:26.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.860+0000 7f8604c00640 1 -- 172.21.15.133:0/2698880532 <== osd.0 v2:172.21.15.133:6814/316026862 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f85f8008eb0 con 0x7f85e8073720 2024-03-07T06:51:26.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.860+0000 7ff5c8400640 1 -- 172.21.15.133:0/2861972310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5a80039b0 msgr2=0x7ff5a8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.860+0000 7ff5c8400640 1 --2- 172.21.15.133:0/2861972310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5a80039b0 0x7ff5a8003dd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400ba30 tx=0x7ff5a8005830 comp rx=0 tx=0).stop 2024-03-07T06:51:26.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.861+0000 7f2f65c00640 1 Processor -- start 2024-03-07T06:51:26.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.861+0000 7ff5c6400640 1 -- 172.21.15.133:0/2861972310 reap_dead start 2024-03-07T06:51:26.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.861+0000 7f2f65c00640 1 -- start start 2024-03-07T06:51:26.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.861+0000 7ff5c8400640 1 -- 172.21.15.133:0/2861972310 shutdown_connections 2024-03-07T06:51:26.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.861+0000 7ff5c8400640 1 -- 172.21.15.133:0/2861972310 >> 172.21.15.133:0/2861972310 conn(0x7ff5c00d0e70 msgr2=0x7ff5c00d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.861+0000 7ff7a0c00640 1 -- 172.21.15.133:0/2042785242 <== osd.1 v2:172.21.15.133:6800/1149391957 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff7840761c0 con 0x7ff7840736f0 2024-03-07T06:51:26.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.861+0000 7f5eeec00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5ee80acd00 0x7f5ee80ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.861+0000 7f5eeec00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5ee80ae880 0x7f5ee806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.861+0000 7f5eeec00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee80ad2a0 con 0x7f5ee815a3e0 2024-03-07T06:51:26.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.862+0000 7f5eeec00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee80ad580 con 0x7f5ee80ae880 2024-03-07T06:51:26.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.862+0000 7f5eeec00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee80ad860 con 0x7f5ee80acd00 2024-03-07T06:51:26.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.862+0000 7f5ee7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5ee80acd00 0x7f5ee80ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.862+0000 7f5ee7400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5ee80acd00 0x7f5ee80ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58834/0 (socket says 172.21.15.133:58834) 2024-03-07T06:51:26.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.862+0000 7f5ee7400640 1 -- 172.21.15.133:0/2188258191 learned_addr learned my addr 172.21.15.133:0/2188258191 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:26.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.862+0000 7f2f65c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f600b15d0 0x7f2f60148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.862+0000 7f2f65c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f601625d0 0x7f2f600b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.862+0000 7f2f65c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f601629b0 0x7f2f600b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.862+0000 7f8dd8200640 1 Processor -- start 2024-03-07T06:51:26.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.862+0000 7f014d600640 1 -- 172.21.15.133:0/1487482422 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f01440071c0 con 0x7f01500b70d0 2024-03-07T06:51:26.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.862+0000 7f2f65c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f600731d0 con 0x7f2f601629b0 2024-03-07T06:51:26.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.862+0000 7f2f65c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f600cea40 con 0x7f2f600b15d0 2024-03-07T06:51:26.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.862+0000 7f2f65c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f600d1e70 con 0x7f2f601625d0 2024-03-07T06:51:26.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f2f5f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f601629b0 0x7f2f600b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.861+0000 7f5ee7e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5ee815a3e0 0x7f5ee8072710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44138/0 (socket says 172.21.15.133:44138) 2024-03-07T06:51:26.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f5eecc00640 1 --2- 172.21.15.133:0/2188258191 >> v2:172.21.15.133:3300/0 conn(0x7f5ee80ae880 0x7f5ee806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f5ee7400640 1 -- 172.21.15.133:0/2188258191 >> v2:172.21.15.133:3300/0 conn(0x7f5ee80ae880 msgr2=0x7f5ee806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:26.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f8dd8200640 1 -- start start 2024-03-07T06:51:26.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f5ee7400640 1 --2- 172.21.15.133:0/2188258191 >> v2:172.21.15.133:3300/0 conn(0x7f5ee80ae880 0x7f5ee806d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f5ee7400640 1 -- 172.21.15.133:0/2188258191 >> v1:172.21.15.133:6789/0 conn(0x7f5ee815a3e0 legacy=0x7f5ee8072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f1f52000640 1 -- 172.21.15.133:0/3827871970 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1f4c15a3e0 msgr2=0x7f1f4c072710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f1f52000640 1 --2- 172.21.15.133:0/3827871970 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1f4c15a3e0 0x7f1f4c072710 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00dbd0 tx=0x7f1f3c02fcd0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f5ee7400640 1 -- 172.21.15.133:0/2188258191 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee80adb40 con 0x7f5ee80acd00 2024-03-07T06:51:26.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f2f5f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f601629b0 0x7f2f600b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38548/0 (socket says 172.21.15.133:38548) 2024-03-07T06:51:26.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f2f5e000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f601625d0 0x7f2f600b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f2f5f400640 1 -- 172.21.15.133:0/3746106822 learned_addr learned my addr 172.21.15.133:0/3746106822 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:26.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f2f5e000640 1 --2- 172.21.15.133:0/3746106822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f601625d0 0x7f2f600b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:41984/0 (socket says 172.21.15.133:41984) 2024-03-07T06:51:26.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f2f5ea00640 1 --2- 172.21.15.133:0/3746106822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f600b15d0 0x7f2f60148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f014d600640 1 --2- 172.21.15.133:0/1487482422 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f013006b5a0 0x7f013006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f014d600640 1 -- 172.21.15.133:0/1487482422 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f014408a780 con 0x7f01500b70d0 2024-03-07T06:51:26.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f014d600640 1 --2- 172.21.15.133:0/1487482422 >> [v2:172.21.15.133:6808/3182454988,v1:172.21.15.133:6809/3182454988] conn(0x7f01300737c0 0x7f0130075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f014d600640 1 -- 172.21.15.133:0/1487482422 --> [v2:172.21.15.133:6808/3182454988,v1:172.21.15.133:6809/3182454988] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0130076290 con 0x7f01300737c0 2024-03-07T06:51:26.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f014d600640 1 -- 172.21.15.133:0/1487482422 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f014408ab70 con 0x7f01500b70d0 2024-03-07T06:51:26.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.863+0000 7f5eecc00640 1 --2- 172.21.15.133:0/2188258191 >> v2:172.21.15.133:3300/0 conn(0x7f5ee80ae880 0x7f5ee806d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:26.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7ff5c8400640 1 -- 172.21.15.133:0/2861972310 shutdown_connections 2024-03-07T06:51:26.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f8dd8200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd00b1110 0x7f8dd0158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.920 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7ff5c8400640 1 -- 172.21.15.133:0/2861972310 wait complete. 2024-03-07T06:51:26.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f0155200640 1 --2- 172.21.15.133:0/1487482422 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f013006b5a0 0x7f013006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f8dd8200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd00ce420 0x7f8dd00b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f8dd8200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd01e3d60 0x7f8dd01e4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f8dd8200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd015ba40 con 0x7f8dd00b1110 2024-03-07T06:51:26.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f8dd8200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd0071ee0 con 0x7f8dd00ce420 2024-03-07T06:51:26.921 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f8dd8200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd00b1610 con 0x7f8dd01e3d60 2024-03-07T06:51:26.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f2f5f400640 1 -- 172.21.15.133:0/3746106822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f601625d0 msgr2=0x7f2f600b21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f2f5f400640 1 --2- 172.21.15.133:0/3746106822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f601625d0 0x7f2f600b21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f2f5f400640 1 -- 172.21.15.133:0/3746106822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f600b15d0 msgr2=0x7f2f60148550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f2f5f400640 1 --2- 172.21.15.133:0/3746106822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f600b15d0 0x7f2f60148550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.864+0000 7f2f5f400640 1 -- 172.21.15.133:0/3746106822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6015f390 con 0x7f2f601629b0 2024-03-07T06:51:26.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.865+0000 7f8dd5800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd00b1110 0x7f8dd0158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.865+0000 7f8dd5800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd00b1110 0x7f8dd0158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38556/0 (socket says 172.21.15.133:38556) 2024-03-07T06:51:26.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.865+0000 7f8dd5800640 1 -- 172.21.15.133:0/1885915878 learned_addr learned my addr 172.21.15.133:0/1885915878 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:26.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.865+0000 7f2f5f400640 1 --2- 172.21.15.133:0/3746106822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f601629b0 0x7f2f600b26e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000c410 tx=0x7f2f5000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.865+0000 7f2f56a00640 1 -- 172.21.15.133:0/3746106822 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f500075a0 con 0x7f2f601629b0 2024-03-07T06:51:26.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.865+0000 7f2f65c00640 1 -- 172.21.15.133:0/3746106822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f6015f670 con 0x7f2f601629b0 2024-03-07T06:51:26.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.865+0000 7f2f65c00640 1 -- 172.21.15.133:0/3746106822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f60159480 con 0x7f2f601629b0 2024-03-07T06:51:26.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.865+0000 7f2f56a00640 1 -- 172.21.15.133:0/3746106822 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f500047e0 con 0x7f2f601629b0 2024-03-07T06:51:26.923 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.865+0000 7f2f56a00640 1 -- 172.21.15.133:0/3746106822 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f50004bb0 con 0x7f2f601629b0 2024-03-07T06:51:26.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f5ee7400640 1 --2- 172.21.15.133:0/2188258191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee80acd00 0x7f5ee80ae340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8068820 tx=0x7f5ed402dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46076cc68ad0534f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f0156600640 1 --2- 172.21.15.133:0/1487482422 >> [v2:172.21.15.133:6808/3182454988,v1:172.21.15.133:6809/3182454988] conn(0x7f01300737c0 0x7f0130075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f0155200640 1 --2- 172.21.15.133:0/1487482422 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f013006b5a0 0x7f013006da60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f013c034000 tx=0x7f013c02f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f5ee6a00640 1 -- 172.21.15.133:0/2188258191 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4008df0 con 0x7f5ee80acd00 2024-03-07T06:51:26.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f5ee6a00640 1 -- 172.21.15.133:0/2188258191 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed400a0d0 con 0x7f5ee80acd00 2024-03-07T06:51:26.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f5ee6a00640 1 -- 172.21.15.133:0/2188258191 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed400a4a0 con 0x7f5ee80acd00 2024-03-07T06:51:26.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f8dd5800640 1 -- 172.21.15.133:0/1885915878 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd01e3d60 msgr2=0x7f8dd01e4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f8dd5800640 1 --2- 172.21.15.133:0/1885915878 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd01e3d60 0x7f8dd01e4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f8dd5800640 1 -- 172.21.15.133:0/1885915878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd00ce420 msgr2=0x7f8dd00b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f8dd5800640 1 --2- 172.21.15.133:0/1885915878 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd00ce420 0x7f8dd00b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f8dd5800640 1 -- 172.21.15.133:0/1885915878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd00b4da0 con 0x7f8dd00b1110 2024-03-07T06:51:26.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f1f52000640 1 -- 172.21.15.133:0/3827871970 shutdown_connections 2024-03-07T06:51:26.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f1f52000640 1 --2- 172.21.15.133:0/3827871970 >> v2:172.21.15.161:3300/0 conn(0x7f1f4c06d2a0 0x7f1f4c06d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f1f52000640 1 --2- 172.21.15.133:0/3827871970 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1f4c15a3e0 0x7f1f4c072710 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f1f52000640 1 -- 172.21.15.133:0/3827871970 >> 172.21.15.133:0/3827871970 conn(0x7f1f4c0b3120 msgr2=0x7f1f4c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f8dd5800640 1 --2- 172.21.15.133:0/1885915878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd00b1110 0x7f8dd0158e20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc000c00 tx=0x7f8dbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.866+0000 7f2f65c00640 1 -- 172.21.15.133:0/3746106822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2f60062c90 con 0x7f2f601629b0 2024-03-07T06:51:26.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.867+0000 7f5eeec00640 1 -- 172.21.15.133:0/2188258191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee80acd00 msgr2=0x7f5ee80ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.867+0000 7f5eeec00640 1 --2- 172.21.15.133:0/2188258191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee80acd00 0x7f5ee80ae340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8068820 tx=0x7f5ed402dbe0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.867+0000 7f8dcd600640 1 -- 172.21.15.133:0/1885915878 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc048070 con 0x7f8dd00b1110 2024-03-07T06:51:26.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.867+0000 7f8dcd600640 1 -- 172.21.15.133:0/1885915878 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc030640 con 0x7f8dd00b1110 2024-03-07T06:51:26.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.867+0000 7f5eeec00640 1 -- 172.21.15.133:0/2188258191 shutdown_connections 2024-03-07T06:51:26.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.867+0000 7f5eeec00640 1 --2- 172.21.15.133:0/2188258191 >> v2:172.21.15.133:3300/0 conn(0x7f5ee80ae880 0x7f5ee806d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.867+0000 7f5eeec00640 1 --2- 172.21.15.133:0/2188258191 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee80acd00 0x7f5ee80ae340 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.867+0000 7f5eeec00640 1 -- 172.21.15.133:0/2188258191 >> 172.21.15.133:0/2188258191 conn(0x7f5ee80b3120 msgr2=0x7f5ee80b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.867+0000 7f1f52000640 1 -- 172.21.15.133:0/3827871970 shutdown_connections 2024-03-07T06:51:26.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.867+0000 7f1f52000640 1 -- 172.21.15.133:0/3827871970 wait complete. 2024-03-07T06:51:26.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.867+0000 7f8dd8200640 1 -- 172.21.15.133:0/1885915878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd00b5080 con 0x7f8dd00b1110 2024-03-07T06:51:26.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.868+0000 7f8dcd600640 1 -- 172.21.15.133:0/1885915878 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc034040 con 0x7f8dd00b1110 2024-03-07T06:51:26.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.868+0000 7f5eeec00640 1 -- 172.21.15.133:0/2188258191 shutdown_connections 2024-03-07T06:51:26.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.868+0000 7f5eeec00640 1 -- 172.21.15.133:0/2188258191 wait complete. 2024-03-07T06:51:26.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.868+0000 7ff5c8400640 1 Processor -- start 2024-03-07T06:51:26.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.868+0000 7ff5c8400640 1 -- start start 2024-03-07T06:51:26.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.868+0000 7f8dd8200640 1 -- 172.21.15.133:0/1885915878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd00b5640 con 0x7f8dd00b1110 2024-03-07T06:51:26.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.869+0000 7f8dd8200640 1 -- 172.21.15.133:0/1885915878 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8dd0159360 con 0x7f8dd00b1110 2024-03-07T06:51:26.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.870+0000 7ff5c8400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5a80039b0 0x7ff5c015d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.870+0000 7ff5c8400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c014ede0 0x7ff5c015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.870+0000 7ff5c8400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c015f200 0x7ff5c01ec5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.870+0000 7ff5c8400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c00b56a0 con 0x7ff5c015f200 2024-03-07T06:51:26.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.870+0000 7ff5c8400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c00b7010 con 0x7ff5c014ede0 2024-03-07T06:51:26.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.870+0000 7ff5c8400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c00b5820 con 0x7ff5a80039b0 2024-03-07T06:51:26.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.870+0000 7f5eeec00640 1 Processor -- start 2024-03-07T06:51:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.870+0000 7ff5c5000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c014ede0 0x7ff5c015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.870+0000 7ff5c5000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c014ede0 0x7ff5c015ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58858/0 (socket says 172.21.15.133:58858) 2024-03-07T06:51:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.870+0000 7ff5c5000640 1 -- 172.21.15.133:0/4031216755 learned_addr learned my addr 172.21.15.133:0/4031216755 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.871+0000 7ff5c6400640 1 --2- 172.21.15.133:0/4031216755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c015f200 0x7ff5c01ec5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.871+0000 7ff5c5000640 1 -- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5a80039b0 msgr2=0x7ff5c015d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.871+0000 7ff5c5000640 1 --2- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5a80039b0 0x7ff5c015d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.871+0000 7ff5c5000640 1 -- 172.21.15.133:0/4031216755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c015f200 msgr2=0x7ff5c01ec5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.871+0000 7ff5c5000640 1 --2- 172.21.15.133:0/4031216755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5c015f200 0x7ff5c01ec5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.871+0000 7ff5c5000640 1 -- 172.21.15.133:0/4031216755 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c015d850 con 0x7ff5c014ede0 2024-03-07T06:51:26.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.872+0000 7f1f52000640 1 Processor -- start 2024-03-07T06:51:26.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.872+0000 7f1f52000640 1 -- start start 2024-03-07T06:51:26.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.869+0000 7f0156600640 1 --2- 172.21.15.133:0/1487482422 >> [v2:172.21.15.133:6808/3182454988,v1:172.21.15.133:6809/3182454988] conn(0x7f01300737c0 0x7f0130075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.872+0000 7f2f56a00640 1 -- 172.21.15.133:0/3746106822 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2f50011070 con 0x7f2f601629b0 2024-03-07T06:51:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.871+0000 7ff5c5000640 1 --2- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c014ede0 0x7ff5c015ecc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac001a70 tx=0x7ff5ac0077f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.872+0000 7ff5bd600640 1 -- 172.21.15.133:0/4031216755 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac00f9f0 con 0x7ff5c014ede0 2024-03-07T06:51:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.872+0000 7ff5bd600640 1 -- 172.21.15.133:0/4031216755 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ac018070 con 0x7ff5c014ede0 2024-03-07T06:51:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.872+0000 7ff5bd600640 1 -- 172.21.15.133:0/4031216755 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac00fdf0 con 0x7ff5c014ede0 2024-03-07T06:51:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.872+0000 7ff5c8400640 1 -- 172.21.15.133:0/4031216755 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c01605e0 con 0x7ff5c014ede0 2024-03-07T06:51:26.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.873+0000 7ff5c8400640 1 -- 172.21.15.133:0/4031216755 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c015fb00 con 0x7ff5c014ede0 2024-03-07T06:51:26.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.873+0000 7f014d600640 1 -- 172.21.15.133:0/1487482422 <== osd.2 v2:172.21.15.133:6808/3182454988 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0130076290 con 0x7f01300737c0 2024-03-07T06:51:26.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.873+0000 7f2f56a00640 1 --2- 172.21.15.133:0/3746106822 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f2f3c06b4d0 0x7f2f3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.873+0000 7f2f56a00640 1 -- 172.21.15.133:0/3746106822 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f5008a610 con 0x7f2f601629b0 2024-03-07T06:51:26.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.873+0000 7f8dcd600640 1 -- 172.21.15.133:0/1885915878 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8dbc04b070 con 0x7f8dd00b1110 2024-03-07T06:51:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.874+0000 7ff5c8400640 1 -- 172.21.15.133:0/4031216755 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5c01601c0 con 0x7ff5c014ede0 2024-03-07T06:51:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.874+0000 7f1f52000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f4c06d2a0 0x7f1f4c1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.874+0000 7f2f56a00640 1 --2- 172.21.15.133:0/3746106822 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x7f2f3c0736f0 0x7f2f3c075b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.874+0000 7f2f56a00640 1 -- 172.21.15.133:0/3746106822 --> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f50009e50 con 0x7f2f3c0736f0 2024-03-07T06:51:26.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.874+0000 7f2f56a00640 1 -- 172.21.15.133:0/3746106822 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f5008aa00 con 0x7f2f601629b0 2024-03-07T06:51:26.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.874+0000 7f1f52000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f4c0acd00 0x7f1f4c15cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.874+0000 7f1f52000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1f4c15a3e0 0x7f1f4c15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.874+0000 7f1f4b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f4c06d2a0 0x7f1f4c1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.874+0000 7f1f4b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f4c06d2a0 0x7f1f4c1581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38564/0 (socket says 172.21.15.133:38564) 2024-03-07T06:51:26.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.874+0000 7f1f4aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f4c0acd00 0x7f1f4c15cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.874+0000 7f1f4aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f4c0acd00 0x7f1f4c15cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58872/0 (socket says 172.21.15.133:58872) 2024-03-07T06:51:26.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.874+0000 7f8dcd600640 1 --2- 172.21.15.133:0/1885915878 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8db006b520 0x7f8db006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.874+0000 7f2f5e000640 1 --2- 172.21.15.133:0/3746106822 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x7f2f3c0736f0 0x7f2f3c075b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.875+0000 7f2f5ea00640 1 --2- 172.21.15.133:0/3746106822 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f2f3c06b4d0 0x7f2f3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.875+0000 7f8dcd600640 1 -- 172.21.15.133:0/1885915878 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8dbc0abec0 con 0x7f8dd00b1110 2024-03-07T06:51:26.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.874+0000 7f5eeec00640 1 -- start start 2024-03-07T06:51:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.875+0000 7f8dcd600640 1 --2- 172.21.15.133:0/1885915878 >> [v2:172.21.15.161:6802/2969599292,v1:172.21.15.161:6803/2969599292] conn(0x7f8db0073740 0x7f8db0075b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.875+0000 7f8dcd600640 1 -- 172.21.15.133:0/1885915878 --> [v2:172.21.15.161:6802/2969599292,v1:172.21.15.161:6803/2969599292] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8dbc001d30 con 0x7f8db0073740 2024-03-07T06:51:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.875+0000 7f8dcd600640 1 -- 172.21.15.133:0/1885915878 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dbc0b1060 con 0x7f8dd00b1110 2024-03-07T06:51:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.875+0000 7f8dd4e00640 1 --2- 172.21.15.133:0/1885915878 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8db006b520 0x7f8db006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.875+0000 7f8dd6200640 1 --2- 172.21.15.133:0/1885915878 >> [v2:172.21.15.161:6802/2969599292,v1:172.21.15.161:6803/2969599292] conn(0x7f8db0073740 0x7f8db0075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.876+0000 7f5eeec00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ee80acd00 0x7f5ee81ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.876+0000 7f5eeec00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5ee815a3e0 0x7f5ee8157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.876+0000 7f5eeec00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee80b5620 0x7f5ee80afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.876+0000 7f5eeec00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee80b3d90 con 0x7f5ee80acd00 2024-03-07T06:51:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.876+0000 7f5eeec00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee80b3c10 con 0x7f5ee80b5620 2024-03-07T06:51:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.876+0000 7f5eeec00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee806d7e0 con 0x7f5ee815a3e0 2024-03-07T06:51:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.876+0000 7f8dd4e00640 1 --2- 172.21.15.133:0/1885915878 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8db006b520 0x7f8db006d9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4000f60 tx=0x7f8dc40079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.877+0000 7ff5bd600640 1 -- 172.21.15.133:0/4031216755 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff5ac01d460 con 0x7ff5c014ede0 2024-03-07T06:51:26.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.876+0000 7f8dd6200640 1 --2- 172.21.15.133:0/1885915878 >> [v2:172.21.15.161:6802/2969599292,v1:172.21.15.161:6803/2969599292] conn(0x7f8db0073740 0x7f8db0075b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.875+0000 7f2f5e000640 1 --2- 172.21.15.133:0/3746106822 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x7f2f3c0736f0 0x7f2f3c075b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.877+0000 7f5ee7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ee80acd00 0x7f5ee81ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.877+0000 7f5ee7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ee80acd00 0x7f5ee81ec5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38574/0 (socket says 172.21.15.133:38574) 2024-03-07T06:51:26.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.877+0000 7f5ee7e00640 1 -- 172.21.15.133:0/407290883 learned_addr learned my addr 172.21.15.133:0/407290883 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:26.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.877+0000 7f2f56a00640 1 -- 172.21.15.133:0/3746106822 <== osd.3 v2:172.21.15.133:6816/2661528260 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2f50009e50 con 0x7f2f3c0736f0 2024-03-07T06:51:26.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.877+0000 7f5eecc00640 1 --2- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee80b5620 0x7f5ee80afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.877+0000 7f2f5ea00640 1 --2- 172.21.15.133:0/3746106822 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f2f3c06b4d0 0x7f2f3c06d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f58036000 tx=0x7f2f58030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7f1f4b400640 1 -- 172.21.15.133:0/2943087741 learned_addr learned my addr 172.21.15.133:0/2943087741 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:26.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7f1f52000640 1 -- 172.21.15.133:0/2943087741 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0ade20 con 0x7f1f4c06d2a0 2024-03-07T06:51:26.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7f1f52000640 1 -- 172.21.15.133:0/2943087741 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0adb40 con 0x7f1f4c0acd00 2024-03-07T06:51:26.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7f1f52000640 1 -- 172.21.15.133:0/2943087741 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0b3c10 con 0x7f1f4c15a3e0 2024-03-07T06:51:26.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.877+0000 7f5ee7e00640 1 -- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5ee815a3e0 msgr2=0x7f5ee8157b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-07T06:51:26.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.877+0000 7f5ee7e00640 1 --2- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5ee815a3e0 0x7f5ee8157b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.877+0000 7f5ee7e00640 1 -- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee80b5620 msgr2=0x7f5ee80afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.877+0000 7f5ee7e00640 1 --2- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ee80b5620 0x7f5ee80afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.877+0000 7f5ee7e00640 1 -- 172.21.15.133:0/407290883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee81ecb30 con 0x7f5ee80acd00 2024-03-07T06:51:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7ff5bd600640 1 --2- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff5a006b4d0 0x7ff5a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7ff5bd600640 1 -- 172.21.15.133:0/4031216755 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5ac08ae20 con 0x7ff5c014ede0 2024-03-07T06:51:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7f5ee7e00640 1 --2- 172.21.15.133:0/407290883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ee80acd00 0x7f5ee81ec5f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800c3f0 tx=0x7f5ed800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7ff5bd600640 1 --2- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:6818/1085452100,v1:172.21.15.161:6819/1085452100] conn(0x7ff5a0073670 0x7ff5a0075a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7ff5bd600640 1 -- 172.21.15.133:0/4031216755 --> [v2:172.21.15.161:6818/1085452100,v1:172.21.15.161:6819/1085452100] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5ac008e50 con 0x7ff5a0073670 2024-03-07T06:51:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7ff5bd600640 1 -- 172.21.15.133:0/4031216755 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5ac08b2b0 con 0x7ff5c014ede0 2024-03-07T06:51:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7f1f4b400640 1 -- 172.21.15.133:0/2943087741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1f4c15a3e0 msgr2=0x7f1f4c15d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7f1f4b400640 1 --2- 172.21.15.133:0/2943087741 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1f4c15a3e0 0x7f1f4c15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7f1f4b400640 1 -- 172.21.15.133:0/2943087741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f4c0acd00 msgr2=0x7f1f4c15cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7f1f4b400640 1 --2- 172.21.15.133:0/2943087741 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f4c0acd00 0x7f1f4c15cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7f1f4b400640 1 -- 172.21.15.133:0/2943087741 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c15b180 con 0x7f1f4c06d2a0 2024-03-07T06:51:26.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.878+0000 7f5edfe00640 1 -- 172.21.15.133:0/407290883 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed80074e0 con 0x7f5ee80acd00 2024-03-07T06:51:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.879+0000 7f1f4b400640 1 --2- 172.21.15.133:0/2943087741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f4c06d2a0 0x7f1f4c1581e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c03c040 tx=0x7f1f3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.879+0000 7f5edfe00640 1 -- 172.21.15.133:0/407290883 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed8007680 con 0x7f5ee80acd00 2024-03-07T06:51:26.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.879+0000 7f5eeec00640 1 -- 172.21.15.133:0/407290883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee815c220 con 0x7f5ee80acd00 2024-03-07T06:51:26.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.879+0000 7f1f43400640 1 -- 172.21.15.133:0/2943087741 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c00d2c0 con 0x7f1f4c06d2a0 2024-03-07T06:51:26.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.879+0000 7f1f43400640 1 -- 172.21.15.133:0/2943087741 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c0395c0 con 0x7f1f4c06d2a0 2024-03-07T06:51:26.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.879+0000 7f1f52000640 1 -- 172.21.15.133:0/2943087741 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4c15b420 con 0x7f1f4c06d2a0 2024-03-07T06:51:26.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.879+0000 7f1f52000640 1 -- 172.21.15.133:0/2943087741 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4c15b9e0 con 0x7f1f4c06d2a0 2024-03-07T06:51:26.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.880+0000 7ff5c5a00640 1 --2- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff5a006b4d0 0x7ff5a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.880+0000 7f8dcd600640 1 -- 172.21.15.133:0/1885915878 <== osd.6 v2:172.21.15.161:6802/2969599292 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8dbc001d30 con 0x7f8db0073740 2024-03-07T06:51:26.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.880+0000 7ff5c6400640 1 --2- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:6818/1085452100,v1:172.21.15.161:6819/1085452100] conn(0x7ff5a0073670 0x7ff5a0075a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.879+0000 7f5edfe00640 1 -- 172.21.15.133:0/407290883 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8004a00 con 0x7f5ee80acd00 2024-03-07T06:51:26.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.879+0000 7f5eeec00640 1 -- 172.21.15.133:0/407290883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee815cac0 con 0x7f5ee80acd00 2024-03-07T06:51:26.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.879+0000 7f1f43400640 1 -- 172.21.15.133:0/2943087741 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0399b0 con 0x7f1f4c06d2a0 2024-03-07T06:51:26.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.881+0000 7ff5c6400640 1 --2- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:6818/1085452100,v1:172.21.15.161:6819/1085452100] conn(0x7ff5a0073670 0x7ff5a0075a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.881+0000 7ff5c5a00640 1 --2- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff5a006b4d0 0x7ff5a006d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400ba00 tx=0x7ff5b4032000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.882+0000 7f5eeec00640 1 -- 172.21.15.133:0/407290883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5ec0001030 con 0x7f5ee80acd00 2024-03-07T06:51:26.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.883+0000 7f1f52000640 1 -- 172.21.15.133:0/2943087741 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1f24001030 con 0x7f1f4c06d2a0 2024-03-07T06:51:26.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.883+0000 7f1f43400640 1 -- 172.21.15.133:0/2943087741 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1f3c04a060 con 0x7f1f4c06d2a0 2024-03-07T06:51:26.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.885+0000 7f1f43400640 1 --2- 172.21.15.133:0/2943087741 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1f2c06b5a0 0x7f1f2c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.885+0000 7f1f43400640 1 -- 172.21.15.133:0/2943087741 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f3c0ab8f0 con 0x7f1f4c06d2a0 2024-03-07T06:51:26.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.885+0000 7f1f43400640 1 --2- 172.21.15.133:0/2943087741 >> [v2:172.21.15.161:6826/3482024257,v1:172.21.15.161:6827/3482024257] conn(0x7f1f2c073740 0x7f1f2c075b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.885+0000 7f1f43400640 1 -- 172.21.15.133:0/2943087741 --> [v2:172.21.15.161:6826/3482024257,v1:172.21.15.161:6827/3482024257] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f3c007900 con 0x7f1f2c073740 2024-03-07T06:51:26.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.883+0000 7ff5bd600640 1 -- 172.21.15.133:0/4031216755 <== osd.5 v2:172.21.15.161:6818/1085452100 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff5ac008e50 con 0x7ff5a0073670 2024-03-07T06:51:26.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.883+0000 7f5edfe00640 1 -- 172.21.15.133:0/407290883 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5ed801a020 con 0x7f5ee80acd00 2024-03-07T06:51:26.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.886+0000 7f1f4aa00640 1 --2- 172.21.15.133:0/2943087741 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1f2c06b5a0 0x7f1f2c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.887+0000 7f1f43400640 1 -- 172.21.15.133:0/2943087741 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f3c0762f0 con 0x7f1f4c06d2a0 2024-03-07T06:51:26.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.887+0000 7f1f4be00640 1 --2- 172.21.15.133:0/2943087741 >> [v2:172.21.15.161:6826/3482024257,v1:172.21.15.161:6827/3482024257] conn(0x7f1f2c073740 0x7f1f2c075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.887+0000 7f5edfe00640 1 --2- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5ec806b5a0 0x7f5ec806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.887+0000 7f5edfe00640 1 -- 172.21.15.133:0/407290883 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ed80898a0 con 0x7f5ee80acd00 2024-03-07T06:51:26.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.887+0000 7f5edfe00640 1 --2- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:6810/3665789028,v1:172.21.15.161:6811/3665789028] conn(0x7f5ec80737c0 0x7f5ec8075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:26.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.887+0000 7f5edfe00640 1 -- 172.21.15.133:0/407290883 --> [v2:172.21.15.161:6810/3665789028,v1:172.21.15.161:6811/3665789028] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ec8076290 con 0x7f5ec80737c0 2024-03-07T06:51:26.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.888+0000 7f5edfe00640 1 -- 172.21.15.133:0/407290883 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ed808e2a0 con 0x7f5ee80acd00 2024-03-07T06:51:26.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.888+0000 7f5eecc00640 1 --2- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:6810/3665789028,v1:172.21.15.161:6811/3665789028] conn(0x7f5ec80737c0 0x7f5ec8075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.888+0000 7f5ee7400640 1 --2- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5ec806b5a0 0x7f5ec806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:26.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.889+0000 7f1f4be00640 1 --2- 172.21.15.133:0/2943087741 >> [v2:172.21.15.161:6826/3482024257,v1:172.21.15.161:6827/3482024257] conn(0x7f1f2c073740 0x7f1f2c075b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.889+0000 7f5eecc00640 1 --2- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:6810/3665789028,v1:172.21.15.161:6811/3665789028] conn(0x7f5ec80737c0 0x7f5ec8075be0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.891+0000 7f1f4aa00640 1 --2- 172.21.15.133:0/2943087741 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1f2c06b5a0 0x7f1f2c06da60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c1541e0 tx=0x7f1f38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.891+0000 7f5edfe00640 1 -- 172.21.15.133:0/407290883 <== osd.7 v2:172.21.15.161:6810/3665789028 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5ec8076290 con 0x7f5ec80737c0 2024-03-07T06:51:26.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.892+0000 7f1f43400640 1 -- 172.21.15.133:0/2943087741 <== osd.4 v2:172.21.15.161:6826/3482024257 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1f3c007900 con 0x7f1f2c073740 2024-03-07T06:51:26.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.896+0000 7f860e600640 1 -- 172.21.15.133:0/2698880532 --> [v2:172.21.15.133:6814/316026862,v1:172.21.15.133:6819/316026862] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8608137ca0 con 0x7f85e8073720 2024-03-07T06:51:26.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.897+0000 7f8604c00640 1 -- 172.21.15.133:0/2698880532 <== osd.0 v2:172.21.15.133:6814/316026862 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8608137ca0 con 0x7f85e8073720 2024-03-07T06:51:26.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.892+0000 7f5ee7400640 1 --2- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5ec806b5a0 0x7f5ec806da60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4037000 tx=0x7f5ed4032000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.897+0000 7f860e600640 1 -- 172.21.15.133:0/2698880532 >> [v2:172.21.15.133:6814/316026862,v1:172.21.15.133:6819/316026862] conn(0x7f85e8073720 msgr2=0x7f85e8075b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.897+0000 7f860e600640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.133:6814/316026862,v1:172.21.15.133:6819/316026862] conn(0x7f85e8073720 0x7f85e8075b40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.898+0000 7f860e600640 1 -- 172.21.15.133:0/2698880532 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f85e806b520 msgr2=0x7f85e806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.898+0000 7f860e600640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f85e806b520 0x7f85e806d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8600005c00 tx=0x7f8600009040 comp rx=0 tx=0).stop 2024-03-07T06:51:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.898+0000 7f860e600640 1 -- 172.21.15.133:0/2698880532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8608154e10 msgr2=0x7f860814e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.898+0000 7f860e600640 1 --2- 172.21.15.133:0/2698880532 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8608154e10 0x7f860814e160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8608071e50 tx=0x7f85f80046c0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.899+0000 7f860dc00640 1 -- 172.21.15.133:0/2698880532 reap_dead start 2024-03-07T06:51:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.899+0000 7f860e600640 1 -- 172.21.15.133:0/2698880532 shutdown_connections 2024-03-07T06:51:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.899+0000 7f860e600640 1 -- 172.21.15.133:0/2698880532 >> 172.21.15.133:0/2698880532 conn(0x7f8608070400 msgr2=0x7f86080d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.899+0000 7f860e600640 1 -- 172.21.15.133:0/2698880532 shutdown_connections 2024-03-07T06:51:26.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.899+0000 7f860e600640 1 -- 172.21.15.133:0/2698880532 wait complete. 2024-03-07T06:51:26.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.899+0000 7ff7aa400640 1 -- 172.21.15.133:0/2042785242 --> [v2:172.21.15.133:6800/1149391957,v1:172.21.15.133:6801/1149391957] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff79c005390 con 0x7ff7840736f0 2024-03-07T06:51:26.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.900+0000 7ff7a0c00640 1 -- 172.21.15.133:0/2042785242 <== osd.1 v2:172.21.15.133:6800/1149391957 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff79c005390 con 0x7ff7840736f0 2024-03-07T06:51:26.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.902+0000 7f2f65c00640 1 -- 172.21.15.133:0/3746106822 --> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2f600b5640 con 0x7f2f3c0736f0 2024-03-07T06:51:26.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.902+0000 7ff7aa400640 1 -- 172.21.15.133:0/2042785242 >> [v2:172.21.15.133:6800/1149391957,v1:172.21.15.133:6801/1149391957] conn(0x7ff7840736f0 msgr2=0x7ff784075b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.903+0000 7f2f56a00640 1 -- 172.21.15.133:0/3746106822 <== osd.3 v2:172.21.15.133:6816/2661528260 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2f600b5640 con 0x7f2f3c0736f0 2024-03-07T06:51:26.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.902+0000 7ff7aa400640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.133:6800/1149391957,v1:172.21.15.133:6801/1149391957] conn(0x7ff7840736f0 0x7ff784075b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.903+0000 7f2f65c00640 1 -- 172.21.15.133:0/3746106822 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x7f2f3c0736f0 msgr2=0x7f2f3c075b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.903+0000 7f2f65c00640 1 --2- 172.21.15.133:0/3746106822 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x7f2f3c0736f0 0x7f2f3c075b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.903+0000 7ff7aa400640 1 -- 172.21.15.133:0/2042785242 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff78406b4d0 msgr2=0x7ff78406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.903+0000 7ff7aa400640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff78406b4d0 0x7ff78406d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff79c052970 tx=0x7ff790009040 comp rx=0 tx=0).stop 2024-03-07T06:51:26.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.903+0000 7ff7aa400640 1 -- 172.21.15.133:0/2042785242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff79c03c0a0 msgr2=0x7ff79c0d53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.903+0000 7ff7aa400640 1 --2- 172.21.15.133:0/2042785242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff79c03c0a0 0x7ff79c0d53d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4050ba0 tx=0x7ff7a40738f0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.904+0000 7ff7a9a00640 1 -- 172.21.15.133:0/2042785242 reap_dead start 2024-03-07T06:51:26.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.904+0000 7ff7aa400640 1 -- 172.21.15.133:0/2042785242 shutdown_connections 2024-03-07T06:51:26.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.904+0000 7ff7aa400640 1 -- 172.21.15.133:0/2042785242 >> 172.21.15.133:0/2042785242 conn(0x7ff79c058ee0 msgr2=0x7ff79c039e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.904+0000 7ff7aa400640 1 -- 172.21.15.133:0/2042785242 shutdown_connections 2024-03-07T06:51:26.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.904+0000 7ff7aa400640 1 -- 172.21.15.133:0/2042785242 wait complete. 2024-03-07T06:51:26.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.905+0000 7f2f65c00640 1 -- 172.21.15.133:0/3746106822 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f2f3c06b4d0 msgr2=0x7f2f3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.905+0000 7f2f65c00640 1 --2- 172.21.15.133:0/3746106822 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f2f3c06b4d0 0x7f2f3c06d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2f58036000 tx=0x7f2f58030000 comp rx=0 tx=0).stop 2024-03-07T06:51:26.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.905+0000 7f2f65c00640 1 -- 172.21.15.133:0/3746106822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f601629b0 msgr2=0x7f2f600b26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.905+0000 7f2f65c00640 1 --2- 172.21.15.133:0/3746106822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f601629b0 0x7f2f600b26e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000c410 tx=0x7f2f5000c8d0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.905+0000 7f2f5f400640 1 -- 172.21.15.133:0/3746106822 reap_dead start 2024-03-07T06:51:26.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.907+0000 7f2f65c00640 1 -- 172.21.15.133:0/3746106822 shutdown_connections 2024-03-07T06:51:26.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.907+0000 7f2f65c00640 1 -- 172.21.15.133:0/3746106822 >> 172.21.15.133:0/3746106822 conn(0x7f2f6006d4c0 msgr2=0x7f2f600ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.907+0000 7f2f65c00640 1 -- 172.21.15.133:0/3746106822 shutdown_connections 2024-03-07T06:51:26.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.907+0000 7f2f65c00640 1 -- 172.21.15.133:0/3746106822 wait complete. 2024-03-07T06:51:26.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.908+0000 7f8dd8200640 1 -- 172.21.15.133:0/1885915878 --> [v2:172.21.15.161:6802/2969599292,v1:172.21.15.161:6803/2969599292] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8dd0062c90 con 0x7f8db0073740 2024-03-07T06:51:26.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.909+0000 7f8dcd600640 1 -- 172.21.15.133:0/1885915878 <== osd.6 v2:172.21.15.161:6802/2969599292 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8dd0062c90 con 0x7f8db0073740 2024-03-07T06:51:26.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.910+0000 7f8dd8200640 1 -- 172.21.15.133:0/1885915878 >> [v2:172.21.15.161:6802/2969599292,v1:172.21.15.161:6803/2969599292] conn(0x7f8db0073740 msgr2=0x7f8db0075b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.910+0000 7f8dd8200640 1 --2- 172.21.15.133:0/1885915878 >> [v2:172.21.15.161:6802/2969599292,v1:172.21.15.161:6803/2969599292] conn(0x7f8db0073740 0x7f8db0075b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.910+0000 7f8dd8200640 1 -- 172.21.15.133:0/1885915878 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8db006b520 msgr2=0x7f8db006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.911+0000 7f8dd8200640 1 --2- 172.21.15.133:0/1885915878 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8db006b520 0x7f8db006d9e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4000f60 tx=0x7f8dc40079d0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.911+0000 7f8dd8200640 1 -- 172.21.15.133:0/1885915878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd00b1110 msgr2=0x7f8dd0158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.911+0000 7f8dd8200640 1 --2- 172.21.15.133:0/1885915878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8dd00b1110 0x7f8dd0158e20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc000c00 tx=0x7f8dbc004730 comp rx=0 tx=0).stop 2024-03-07T06:51:26.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.911+0000 7f8dd6200640 1 -- 172.21.15.133:0/1885915878 reap_dead start 2024-03-07T06:51:26.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.911+0000 7f8dd8200640 1 -- 172.21.15.133:0/1885915878 shutdown_connections 2024-03-07T06:51:26.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.911+0000 7f8dd8200640 1 -- 172.21.15.133:0/1885915878 >> 172.21.15.133:0/1885915878 conn(0x7f8dd00b3120 msgr2=0x7f8dd006ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.911+0000 7f8dd8200640 1 -- 172.21.15.133:0/1885915878 shutdown_connections 2024-03-07T06:51:26.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.911+0000 7f0157000640 1 -- 172.21.15.133:0/1487482422 --> [v2:172.21.15.133:6808/3182454988,v1:172.21.15.133:6809/3182454988] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f012c002dd0 con 0x7f01300737c0 2024-03-07T06:51:26.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.912+0000 7f8dd8200640 1 -- 172.21.15.133:0/1885915878 wait complete. 2024-03-07T06:51:26.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.912+0000 7f014d600640 1 -- 172.21.15.133:0/1487482422 <== osd.2 v2:172.21.15.133:6808/3182454988 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f012c002dd0 con 0x7f01300737c0 2024-03-07T06:51:26.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.913+0000 7f0157000640 1 -- 172.21.15.133:0/1487482422 >> [v2:172.21.15.133:6808/3182454988,v1:172.21.15.133:6809/3182454988] conn(0x7f01300737c0 msgr2=0x7f0130075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.913+0000 7f0157000640 1 --2- 172.21.15.133:0/1487482422 >> [v2:172.21.15.133:6808/3182454988,v1:172.21.15.133:6809/3182454988] conn(0x7f01300737c0 0x7f0130075be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.914+0000 7f0157000640 1 -- 172.21.15.133:0/1487482422 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f013006b5a0 msgr2=0x7f013006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.914+0000 7f0157000640 1 --2- 172.21.15.133:0/1487482422 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f013006b5a0 0x7f013006da60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f013c034000 tx=0x7f013c02f000 comp rx=0 tx=0).stop 2024-03-07T06:51:26.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.914+0000 7f0157000640 1 -- 172.21.15.133:0/1487482422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01500b70d0 msgr2=0x7f015014e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.914+0000 7f0157000640 1 --2- 172.21.15.133:0/1487482422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01500b70d0 0x7f015014e9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f014400c780 tx=0x7f014400cc40 comp rx=0 tx=0).stop 2024-03-07T06:51:26.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.914+0000 7f0156600640 1 -- 172.21.15.133:0/1487482422 reap_dead start 2024-03-07T06:51:26.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.914+0000 7f0157000640 1 -- 172.21.15.133:0/1487482422 shutdown_connections 2024-03-07T06:51:26.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.914+0000 7f0157000640 1 -- 172.21.15.133:0/1487482422 >> 172.21.15.133:0/1487482422 conn(0x7f01500674e0 msgr2=0x7f0150071ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.914+0000 7f0157000640 1 -- 172.21.15.133:0/1487482422 shutdown_connections 2024-03-07T06:51:26.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.915+0000 7f0157000640 1 -- 172.21.15.133:0/1487482422 wait complete. 2024-03-07T06:51:26.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.916+0000 7ff5c8400640 1 -- 172.21.15.133:0/4031216755 --> [v2:172.21.15.161:6818/1085452100,v1:172.21.15.161:6819/1085452100] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff5c0062c90 con 0x7ff5a0073670 2024-03-07T06:51:26.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.917+0000 7ff5bd600640 1 -- 172.21.15.133:0/4031216755 <== osd.5 v2:172.21.15.161:6818/1085452100 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff5c0062c90 con 0x7ff5a0073670 2024-03-07T06:51:26.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.918+0000 7ff5c8400640 1 -- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:6818/1085452100,v1:172.21.15.161:6819/1085452100] conn(0x7ff5a0073670 msgr2=0x7ff5a0075a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.918+0000 7ff5c8400640 1 --2- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:6818/1085452100,v1:172.21.15.161:6819/1085452100] conn(0x7ff5a0073670 0x7ff5a0075a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.918+0000 7ff5c8400640 1 -- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff5a006b4d0 msgr2=0x7ff5a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.918+0000 7ff5c8400640 1 --2- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff5a006b4d0 0x7ff5a006d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400ba00 tx=0x7ff5b4032000 comp rx=0 tx=0).stop 2024-03-07T06:51:26.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.918+0000 7ff5c8400640 1 -- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c014ede0 msgr2=0x7ff5c015ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.918+0000 7ff5c8400640 1 --2- 172.21.15.133:0/4031216755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff5c014ede0 0x7ff5c015ecc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac001a70 tx=0x7ff5ac0077f0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.919+0000 7ff5c6400640 1 -- 172.21.15.133:0/4031216755 reap_dead start 2024-03-07T06:51:26.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.919+0000 7ff5c8400640 1 -- 172.21.15.133:0/4031216755 shutdown_connections 2024-03-07T06:51:26.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.919+0000 7ff5c8400640 1 -- 172.21.15.133:0/4031216755 >> 172.21.15.133:0/4031216755 conn(0x7ff5c00d0e70 msgr2=0x7ff5c00d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.919+0000 7ff5c8400640 1 -- 172.21.15.133:0/4031216755 shutdown_connections 2024-03-07T06:51:26.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.920+0000 7ff5c8400640 1 -- 172.21.15.133:0/4031216755 wait complete. 2024-03-07T06:51:26.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.922+0000 7f1f52000640 1 -- 172.21.15.133:0/2943087741 --> [v2:172.21.15.161:6826/3482024257,v1:172.21.15.161:6827/3482024257] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1f24002dd0 con 0x7f1f2c073740 2024-03-07T06:51:26.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.924+0000 7f1f43400640 1 -- 172.21.15.133:0/2943087741 <== osd.4 v2:172.21.15.161:6826/3482024257 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1f24002dd0 con 0x7f1f2c073740 2024-03-07T06:51:26.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.924+0000 7f1f52000640 1 -- 172.21.15.133:0/2943087741 >> [v2:172.21.15.161:6826/3482024257,v1:172.21.15.161:6827/3482024257] conn(0x7f1f2c073740 msgr2=0x7f1f2c075b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.924+0000 7f1f52000640 1 --2- 172.21.15.133:0/2943087741 >> [v2:172.21.15.161:6826/3482024257,v1:172.21.15.161:6827/3482024257] conn(0x7f1f2c073740 0x7f1f2c075b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.971 INFO:teuthology.orchestra.run.smithi133.stdout:55834574857 2024-03-07T06:51:26.971 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-07T06:51:26.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.925+0000 7f1f52000640 1 -- 172.21.15.133:0/2943087741 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1f2c06b5a0 msgr2=0x7f1f2c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.925+0000 7f1f52000640 1 --2- 172.21.15.133:0/2943087741 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1f2c06b5a0 0x7f1f2c06da60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c1541e0 tx=0x7f1f38009040 comp rx=0 tx=0).stop 2024-03-07T06:51:26.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.925+0000 7f1f52000640 1 -- 172.21.15.133:0/2943087741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f4c06d2a0 msgr2=0x7f1f4c1581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.925+0000 7f1f52000640 1 --2- 172.21.15.133:0/2943087741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f4c06d2a0 0x7f1f4c1581e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c03c040 tx=0x7f1f3c004730 comp rx=0 tx=0).stop 2024-03-07T06:51:26.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.925+0000 7f1f4be00640 1 -- 172.21.15.133:0/2943087741 reap_dead start 2024-03-07T06:51:26.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.925+0000 7f5eeec00640 1 -- 172.21.15.133:0/407290883 --> [v2:172.21.15.161:6810/3665789028,v1:172.21.15.161:6811/3665789028] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5ec0002dd0 con 0x7f5ec80737c0 2024-03-07T06:51:26.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.926+0000 7f1f52000640 1 -- 172.21.15.133:0/2943087741 shutdown_connections 2024-03-07T06:51:26.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.926+0000 7f1f52000640 1 -- 172.21.15.133:0/2943087741 >> 172.21.15.133:0/2943087741 conn(0x7f1f4c0b3120 msgr2=0x7f1f4c0cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.926+0000 7f1f52000640 1 -- 172.21.15.133:0/2943087741 shutdown_connections 2024-03-07T06:51:26.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.926+0000 7f1f52000640 1 -- 172.21.15.133:0/2943087741 wait complete. 2024-03-07T06:51:26.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.926+0000 7f5edfe00640 1 -- 172.21.15.133:0/407290883 <== osd.7 v2:172.21.15.161:6810/3665789028 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5ec0002dd0 con 0x7f5ec80737c0 2024-03-07T06:51:26.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.929+0000 7f5eeec00640 1 -- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:6810/3665789028,v1:172.21.15.161:6811/3665789028] conn(0x7f5ec80737c0 msgr2=0x7f5ec8075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.929+0000 7f5eeec00640 1 --2- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:6810/3665789028,v1:172.21.15.161:6811/3665789028] conn(0x7f5ec80737c0 0x7f5ec8075be0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.929+0000 7f5eeec00640 1 -- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5ec806b5a0 msgr2=0x7f5ec806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.929+0000 7f5eeec00640 1 --2- 172.21.15.133:0/407290883 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5ec806b5a0 0x7f5ec806da60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4037000 tx=0x7f5ed4032000 comp rx=0 tx=0).stop 2024-03-07T06:51:26.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.929+0000 7f5eeec00640 1 -- 172.21.15.133:0/407290883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ee80acd00 msgr2=0x7f5ee81ec5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:26.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.929+0000 7f5eeec00640 1 --2- 172.21.15.133:0/407290883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ee80acd00 0x7f5ee81ec5f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800c3f0 tx=0x7f5ed800c8b0 comp rx=0 tx=0).stop 2024-03-07T06:51:26.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.929+0000 7f5eecc00640 1 -- 172.21.15.133:0/407290883 reap_dead start 2024-03-07T06:51:26.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.929+0000 7f5eeec00640 1 -- 172.21.15.133:0/407290883 shutdown_connections 2024-03-07T06:51:26.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.929+0000 7f5eeec00640 1 -- 172.21.15.133:0/407290883 >> 172.21.15.133:0/407290883 conn(0x7f5ee80b3120 msgr2=0x7f5ee80b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:26.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.930+0000 7f5eeec00640 1 -- 172.21.15.133:0/407290883 shutdown_connections 2024-03-07T06:51:26.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:26.930+0000 7f5eeec00640 1 -- 172.21.15.133:0/407290883 wait complete. 2024-03-07T06:51:26.976 INFO:teuthology.orchestra.run.smithi133.stdout:55834574855 2024-03-07T06:51:26.976 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-07T06:51:26.977 INFO:teuthology.orchestra.run.smithi133.stdout:55834574856 2024-03-07T06:51:26.977 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-07T06:51:26.978 INFO:teuthology.orchestra.run.smithi133.stdout:55834574856 2024-03-07T06:51:26.978 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-07T06:51:26.979 INFO:teuthology.orchestra.run.smithi133.stdout:55834574856 2024-03-07T06:51:26.979 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-07T06:51:26.980 INFO:teuthology.orchestra.run.smithi133.stdout:55834574856 2024-03-07T06:51:26.980 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-07T06:51:26.981 INFO:teuthology.orchestra.run.smithi133.stdout:55834574855 2024-03-07T06:51:26.981 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-07T06:51:26.982 INFO:teuthology.orchestra.run.smithi133.stdout:55834574855 2024-03-07T06:51:26.982 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-07T06:51:27.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.294+0000 7fa70a800640 1 Processor -- start 2024-03-07T06:51:27.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.294+0000 7fa70a800640 1 -- start start 2024-03-07T06:51:27.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.297+0000 7fa70a800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6fc0e7710 0x7fa6fc04ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.297+0000 7fa70a800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa6fc053c40 0x7fa6fc007020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.298+0000 7fa70a800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc057d90 con 0x7fa6fc0589c0 2024-03-07T06:51:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.298+0000 7fa70a800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6fc006260 con 0x7fa6fc053c40 2024-03-07T06:51:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.298+0000 7fa709e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa6fc053c40 0x7fa6fc007020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.298+0000 7fa709e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa6fc053c40 0x7fa6fc007020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58888/0 (socket says 172.21.15.133:58888) 2024-03-07T06:51:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.298+0000 7fa709e00640 1 -- 172.21.15.133:0/392262514 learned_addr learned my addr 172.21.15.133:0/392262514 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.298+0000 7fa70a800640 1 -- 172.21.15.133:0/392262514 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc006540 con 0x7fa6fc0e7710 2024-03-07T06:51:27.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.299+0000 7fa709400640 1 --2- 172.21.15.133:0/392262514 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6fc0e7710 0x7fa6fc04ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.300+0000 7fa709e00640 1 -- 172.21.15.133:0/392262514 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6fc0e7710 msgr2=0x7fa6fc04ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:27.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.300+0000 7fa709e00640 1 --2- 172.21.15.133:0/392262514 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6fc0e7710 0x7fa6fc04ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.300+0000 7fa709e00640 1 -- 172.21.15.133:0/392262514 >> v1:172.21.15.133:6789/0 conn(0x7fa6fc0589c0 legacy=0x7fa6fc053530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.300+0000 7fa709e00640 1 -- 172.21.15.133:0/392262514 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc006820 con 0x7fa6fc053c40 2024-03-07T06:51:27.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.302+0000 7fa709e00640 1 --2- 172.21.15.133:0/392262514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6fc053c40 0x7fa6fc007020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009870 tx=0x7fa6f002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9a1a0c7de019d6a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.303+0000 7fa703e00640 1 -- 172.21.15.133:0/392262514 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0035020 con 0x7fa6fc053c40 2024-03-07T06:51:27.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.303+0000 7fa703e00640 1 -- 172.21.15.133:0/392262514 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f0031870 con 0x7fa6fc053c40 2024-03-07T06:51:27.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.303+0000 7fa703e00640 1 -- 172.21.15.133:0/392262514 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0031c80 con 0x7fa6fc053c40 2024-03-07T06:51:27.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.305+0000 7fa70a800640 1 -- 172.21.15.133:0/392262514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6fc053c40 msgr2=0x7fa6fc007020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.305+0000 7fa70a800640 1 --2- 172.21.15.133:0/392262514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6fc053c40 0x7fa6fc007020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009870 tx=0x7fa6f002b910 comp rx=0 tx=0).stop 2024-03-07T06:51:27.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.305+0000 7fa70a800640 1 -- 172.21.15.133:0/392262514 shutdown_connections 2024-03-07T06:51:27.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.305+0000 7fa70a800640 1 --2- 172.21.15.133:0/392262514 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6fc053c40 0x7fa6fc007020 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.305+0000 7fa70a800640 1 --2- 172.21.15.133:0/392262514 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6fc0e7710 0x7fa6fc04ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.305+0000 7fa70a800640 1 -- 172.21.15.133:0/392262514 >> 172.21.15.133:0/392262514 conn(0x7fa6fc058ee0 msgr2=0x7fa6fc0592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.306+0000 7fa70a800640 1 -- 172.21.15.133:0/392262514 shutdown_connections 2024-03-07T06:51:27.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.306+0000 7fa70a800640 1 -- 172.21.15.133:0/392262514 wait complete. 2024-03-07T06:51:27.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.309+0000 7fa70a800640 1 Processor -- start 2024-03-07T06:51:27.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.310+0000 7fa70a800640 1 -- start start 2024-03-07T06:51:27.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.311+0000 7f8047e00640 1 Processor -- start 2024-03-07T06:51:27.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.311+0000 7f8047e00640 1 -- start start 2024-03-07T06:51:27.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.312+0000 7fa70a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6fc053c40 0x7fa6fc03dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.313+0000 7fa709400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6fc053c40 0x7fa6fc03dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.313+0000 7fa709400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6fc053c40 0x7fa6fc03dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38576/0 (socket says 172.21.15.133:38576) 2024-03-07T06:51:27.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.313+0000 7fa709400640 1 -- 172.21.15.133:0/3645416259 learned_addr learned my addr 172.21.15.133:0/3645416259 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:27.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.313+0000 7fa70a800640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6fc0589c0 0x7fa6fc180d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.314+0000 7fa70a800640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6fc0e7710 0x7fa6fc03bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.314+0000 7f8047e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f804003e840 0x7f8040053bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.314+0000 7fa70a800640 1 -- 172.21.15.133:0/3645416259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0579a0 con 0x7fa6fc053c40 2024-03-07T06:51:27.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.314+0000 7f8047e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8040054130 0x7f80400068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.315+0000 7f8047e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8040058570 con 0x7f80400e30c0 2024-03-07T06:51:27.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.314+0000 7fa708a00640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6fc0589c0 0x7fa6fc180d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.315+0000 7fa70a800640 1 -- 172.21.15.133:0/3645416259 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc04cf70 con 0x7fa6fc0e7710 2024-03-07T06:51:27.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.315+0000 7fa70a800640 1 -- 172.21.15.133:0/3645416259 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc057b20 con 0x7fa6fc0589c0 2024-03-07T06:51:27.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.315+0000 7fa709e00640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6fc0e7710 0x7fa6fc03bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.315+0000 7f8046000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f804003e840 0x7f8040053bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.315+0000 7f8046000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f804003e840 0x7f8040053bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38582/0 (socket says 172.21.15.133:38582) 2024-03-07T06:51:27.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.315+0000 7f8046000640 1 -- 172.21.15.133:0/325925979 learned_addr learned my addr 172.21.15.133:0/325925979 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:27.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.315+0000 7f8047e00640 1 -- 172.21.15.133:0/325925979 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80400545a0 con 0x7f804003e840 2024-03-07T06:51:27.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.315+0000 7f8047e00640 1 -- 172.21.15.133:0/325925979 --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8040058850 con 0x7f8040054130 2024-03-07T06:51:27.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.316+0000 7f8047400640 1 --2- 172.21.15.133:0/325925979 >> v2:172.21.15.161:3300/0 conn(0x7f8040054130 0x7f80400068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.316+0000 7fa709400640 1 -- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6fc0589c0 msgr2=0x7fa6fc180d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.316+0000 7fa709400640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6fc0589c0 0x7fa6fc180d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.316+0000 7fa709400640 1 -- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6fc0e7710 msgr2=0x7fa6fc03bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.316+0000 7fa709400640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6fc0e7710 0x7fa6fc03bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.316+0000 7fa709400640 1 -- 172.21.15.133:0/3645416259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc03e150 con 0x7fa6fc053c40 2024-03-07T06:51:27.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.316+0000 7f8046000640 1 -- 172.21.15.133:0/325925979 >> v2:172.21.15.161:3300/0 conn(0x7f8040054130 msgr2=0x7f80400068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:27.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.316+0000 7f8045600640 1 -- 172.21.15.133:0/325925979 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860240620 0 0) 0x7f8040058570 con 0x7f80400e30c0 2024-03-07T06:51:27.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.316+0000 7f8046000640 1 --2- 172.21.15.133:0/325925979 >> v2:172.21.15.161:3300/0 conn(0x7f8040054130 0x7f80400068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.316+0000 7f8046000640 1 -- 172.21.15.133:0/325925979 >> v1:172.21.15.161:6789/0 conn(0x7f80400e30c0 legacy=0x7f8040057d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.317+0000 7f8046000640 1 -- 172.21.15.133:0/325925979 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8040058b30 con 0x7f804003e840 2024-03-07T06:51:27.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.317+0000 7fa709400640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6fc053c40 0x7fa6fc03dab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa704067da0 tx=0x7fa704074e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.318+0000 7fa701600640 1 -- 172.21.15.133:0/3645416259 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040747e0 con 0x7fa6fc053c40 2024-03-07T06:51:27.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.318+0000 7fa701600640 1 -- 172.21.15.133:0/3645416259 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa704069190 con 0x7fa6fc053c40 2024-03-07T06:51:27.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.318+0000 7fa701600640 1 -- 172.21.15.133:0/3645416259 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70404f3c0 con 0x7fa6fc053c40 2024-03-07T06:51:27.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.318+0000 7fa70a800640 1 -- 172.21.15.133:0/3645416259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc03e430 con 0x7fa6fc053c40 2024-03-07T06:51:27.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.318+0000 7fa70a800640 1 -- 172.21.15.133:0/3645416259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc03e9f0 con 0x7fa6fc053c40 2024-03-07T06:51:27.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.318+0000 7f8046000640 1 --2- 172.21.15.133:0/325925979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f804003e840 0x7f8040053bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f80400e7cd0 tx=0x7f803802ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a020b03d03c008d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.319+0000 7f8045600640 1 -- 172.21.15.133:0/325925979 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803802f870 con 0x7f804003e840 2024-03-07T06:51:27.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.319+0000 7f8045600640 1 -- 172.21.15.133:0/325925979 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803802fa10 con 0x7f804003e840 2024-03-07T06:51:27.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.320+0000 7f8047e00640 1 -- 172.21.15.133:0/325925979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f804003e840 msgr2=0x7f8040053bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.320+0000 7f8045600640 1 -- 172.21.15.133:0/325925979 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803802f870 con 0x7f804003e840 2024-03-07T06:51:27.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.320+0000 7f8047e00640 1 --2- 172.21.15.133:0/325925979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f804003e840 0x7f8040053bf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f80400e7cd0 tx=0x7f803802ba50 comp rx=0 tx=0).stop 2024-03-07T06:51:27.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.321+0000 7f8047e00640 1 -- 172.21.15.133:0/325925979 shutdown_connections 2024-03-07T06:51:27.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.321+0000 7f8047e00640 1 --2- 172.21.15.133:0/325925979 >> v2:172.21.15.161:3300/0 conn(0x7f8040054130 0x7f80400068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.321+0000 7f8047e00640 1 --2- 172.21.15.133:0/325925979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f804003e840 0x7f8040053bf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.321+0000 7f8047e00640 1 -- 172.21.15.133:0/325925979 >> 172.21.15.133:0/325925979 conn(0x7f80400559e0 msgr2=0x7f8040055df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.321+0000 7f8047e00640 1 -- 172.21.15.133:0/325925979 shutdown_connections 2024-03-07T06:51:27.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.321+0000 7fa70a800640 1 -- 172.21.15.133:0/3645416259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e0005290 con 0x7fa6fc053c40 2024-03-07T06:51:27.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.322+0000 7f8047e00640 1 -- 172.21.15.133:0/325925979 wait complete. 2024-03-07T06:51:27.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.324+0000 7f7cc8c00640 1 Processor -- start 2024-03-07T06:51:27.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.324+0000 7f7cc8c00640 1 -- start start 2024-03-07T06:51:27.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.324+0000 7f8047e00640 1 Processor -- start 2024-03-07T06:51:27.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.324+0000 7f8047e00640 1 -- start start 2024-03-07T06:51:27.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.326+0000 7f8047e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f804003e840 0x7f8040178eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.326+0000 7f8047e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8040054130 0x7f80400e0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.326+0000 7f8047e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80400e30c0 0x7f80400e1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.326+0000 7f8047e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80400e34a0 con 0x7f8040054130 2024-03-07T06:51:27.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.326+0000 7f8047e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804003bfc0 con 0x7f804003e840 2024-03-07T06:51:27.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.326+0000 7f8047e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8040006e10 con 0x7f80400e30c0 2024-03-07T06:51:27.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.327+0000 7f7cc8c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cc415a3e0 0x7f7cc4072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.327+0000 7f3b1ba00640 1 Processor -- start 2024-03-07T06:51:27.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.327+0000 7f7cc8c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7cc40acd00 0x7f7cc40ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.327+0000 7f3b1ba00640 1 -- start start 2024-03-07T06:51:27.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.327+0000 7f7cc8c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7cc40ae880 0x7f7cc406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.327+0000 7f7cc8c00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc4072c50 con 0x7f7cc40ae880 2024-03-07T06:51:27.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.327+0000 7f7cc8c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc40ad2a0 con 0x7f7cc40acd00 2024-03-07T06:51:27.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.327+0000 7f7cc8c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40ad580 con 0x7f7cc415a3e0 2024-03-07T06:51:27.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.327+0000 7f8046a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f804003e840 0x7f8040178eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.327+0000 7f8046000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8040054130 0x7f80400e0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.327+0000 7f8046000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8040054130 0x7f80400e0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38584/0 (socket says 172.21.15.133:38584) 2024-03-07T06:51:27.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.327+0000 7f8046000640 1 -- 172.21.15.133:0/1088157022 learned_addr learned my addr 172.21.15.133:0/1088157022 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:27.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.327+0000 7f8047400640 1 --2- 172.21.15.133:0/1088157022 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80400e30c0 0x7f80400e1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.328+0000 7f8047400640 1 -- 172.21.15.133:0/1088157022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f804003e840 msgr2=0x7f8040178eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.328+0000 7f8047400640 1 --2- 172.21.15.133:0/1088157022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f804003e840 0x7f8040178eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.328+0000 7f8047400640 1 -- 172.21.15.133:0/1088157022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8040054130 msgr2=0x7f80400e0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.328+0000 7f7cc2800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7cc40ae880 0x7f7cc406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.328+0000 7f7cc1400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7cc40acd00 0x7f7cc40ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.328+0000 7f8047400640 1 --2- 172.21.15.133:0/1088157022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8040054130 0x7f80400e0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.328+0000 7f8047400640 1 -- 172.21.15.133:0/1088157022 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80401793f0 con 0x7f80400e30c0 2024-03-07T06:51:27.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.328+0000 7f7cc2800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7cc40ae880 0x7f7cc406d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38594/0 (socket says 172.21.15.133:38594) 2024-03-07T06:51:27.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.328+0000 7f7cc2800640 1 -- 172.21.15.133:0/3765001665 learned_addr learned my addr 172.21.15.133:0/3765001665 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:27.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.328+0000 7f8046000640 1 --2- 172.21.15.133:0/1088157022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8040054130 0x7f80400e0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:51:27.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.324+0000 7fa701600640 1 -- 172.21.15.133:0/3645416259 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa7040b4820 con 0x7fa6fc053c40 2024-03-07T06:51:27.333 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.325+0000 7fa701600640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa6e406b5a0 0x7fa6e406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.329+0000 7f8047400640 1 --2- 172.21.15.133:0/1088157022 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80400e30c0 0x7f80400e1470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8048064b50 tx=0x7f8048074e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.329+0000 7f7cc1e00640 1 --2- 172.21.15.133:0/3765001665 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cc415a3e0 0x7f7cc4072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.329+0000 7f7cc2800640 1 -- 172.21.15.133:0/3765001665 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cc415a3e0 msgr2=0x7f7cc4072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:27.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.329+0000 7f7cc2800640 1 --2- 172.21.15.133:0/3765001665 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cc415a3e0 0x7f7cc4072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.329+0000 7f7cc2800640 1 -- 172.21.15.133:0/3765001665 >> v2:172.21.15.161:3300/0 conn(0x7f7cc40acd00 msgr2=0x7f7cc40ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:27.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.329+0000 7f7cc2800640 1 --2- 172.21.15.133:0/3765001665 >> v2:172.21.15.161:3300/0 conn(0x7f7cc40acd00 0x7f7cc40ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.329+0000 7f7cc2800640 1 -- 172.21.15.133:0/3765001665 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc40ad860 con 0x7f7cc40ae880 2024-03-07T06:51:27.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.329+0000 7fa701600640 1 -- 172.21.15.133:0/3645416259 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa704104780 con 0x7fa6fc053c40 2024-03-07T06:51:27.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.329+0000 7fa708a00640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa6e406b5a0 0x7fa6e406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.329+0000 7f3b1ba00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b140b5dd0 0x7f3b14144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.329+0000 7f3b1ba00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3b140b43f0 0x7f3b14162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.329+0000 7f803ea00640 1 -- 172.21.15.133:0/1088157022 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804804fc10 con 0x7f80400e30c0 2024-03-07T06:51:27.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.330+0000 7f3b1ba00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b140b4bd0 con 0x7f3b14162ab0 2024-03-07T06:51:27.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.330+0000 7f3b1ba00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b14141700 con 0x7f3b140b43f0 2024-03-07T06:51:27.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.330+0000 7f3b1ba00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14161e00 con 0x7f3b140b5dd0 2024-03-07T06:51:27.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.330+0000 7f803ea00640 1 -- 172.21.15.133:0/1088157022 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80480747e0 con 0x7f80400e30c0 2024-03-07T06:51:27.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.330+0000 7f803ea00640 1 -- 172.21.15.133:0/1088157022 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80480008d0 con 0x7f80400e30c0 2024-03-07T06:51:27.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.330+0000 7fa7e3e00640 1 Processor -- start 2024-03-07T06:51:27.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.330+0000 7fb473e00640 1 Processor -- start 2024-03-07T06:51:27.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.330+0000 7fa7e3e00640 1 -- start start 2024-03-07T06:51:27.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.330+0000 7f3b19000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b140b5dd0 0x7f3b14144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.331+0000 7f7cc2800640 1 --2- 172.21.15.133:0/3765001665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cc40ae880 0x7f7cc406d2a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7cc406cf90 tx=0x7f7cb402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806f3d290c75f823 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.331+0000 7fb473e00640 1 -- start start 2024-03-07T06:51:27.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.331+0000 7f3b13e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3b140b43f0 0x7f3b14162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.331+0000 7f3b19a00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f3b14162ab0 0x7f3b14071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.133:60176/0 (socket says 172.21.15.133:60176) 2024-03-07T06:51:27.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.331+0000 7f3b19a00640 1 -- 172.21.15.133:0/1787508916 learned_addr learned my addr 172.21.15.133:0/1787508916 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:27.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.330+0000 7f8047e00640 1 -- 172.21.15.133:0/1088157022 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80400e6790 con 0x7f80400e30c0 2024-03-07T06:51:27.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.332+0000 7fa7e3e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7e4154e10 0x7fa7e40d32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.331+0000 7f8047e00640 1 -- 172.21.15.133:0/1088157022 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80400e6d50 con 0x7f80400e30c0 2024-03-07T06:51:27.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.332+0000 7f3b13e00640 1 -- 172.21.15.133:0/1787508916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b140b5dd0 msgr2=0x7f3b14144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:27.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.332+0000 7f3b13e00640 1 --2- 172.21.15.133:0/1787508916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b140b5dd0 0x7f3b14144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.332+0000 7f3b13e00640 1 -- 172.21.15.133:0/1787508916 >> v1:172.21.15.161:6789/0 conn(0x7f3b14162ab0 legacy=0x7f3b14071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.332+0000 7fa7e3e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7e40d3810 0x7fa7e4162460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.332+0000 7fa7e3e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e40b43f0 con 0x7fa7e41551f0 2024-03-07T06:51:27.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.332+0000 7fa7e3e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e40d3c80 con 0x7fa7e40d3810 2024-03-07T06:51:27.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.332+0000 7fa7e3e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40b4690 con 0x7fa7e4154e10 2024-03-07T06:51:27.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.332+0000 7f3b13400640 1 -- 172.21.15.133:0/1787508916 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812904574 0 0) 0x7f3b140b4bd0 con 0x7f3b14162ab0 2024-03-07T06:51:27.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.332+0000 7f3b13e00640 1 -- 172.21.15.133:0/1787508916 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b140b4eb0 con 0x7f3b140b43f0 2024-03-07T06:51:27.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.332+0000 7fb473e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb46c005e90 0x7fb46c0062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.333+0000 7fb473e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb46c006810 0x7fb46c006ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.333+0000 7fb473e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46c053720 con 0x7fb46c0ead00 2024-03-07T06:51:27.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.333+0000 7fb473e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46c054330 con 0x7fb46c005e90 2024-03-07T06:51:27.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.333+0000 7fb473e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c053a00 con 0x7fb46c006810 2024-03-07T06:51:27.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.333+0000 7fa7e2a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7e4154e10 0x7fa7e40d32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.333+0000 7f66f2a00640 1 Processor -- start 2024-03-07T06:51:27.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.333+0000 7fa7e2a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7e4154e10 0x7fa7e40d32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:42084/0 (socket says 172.21.15.133:42084) 2024-03-07T06:51:27.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.333+0000 7fa7e2a00640 1 -- 172.21.15.133:0/572426371 learned_addr learned my addr 172.21.15.133:0/572426371 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:27.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.333+0000 7fa7e3400640 1 --2- 172.21.15.133:0/572426371 >> v2:172.21.15.161:3300/0 conn(0x7fa7e40d3810 0x7fa7e4162460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.333+0000 7fb472a00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb46c0ead00 0x7fb46c04fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44152/0 (socket says 172.21.15.133:44152) 2024-03-07T06:51:27.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.333+0000 7fb472a00640 1 -- 172.21.15.133:0/1940434481 learned_addr learned my addr 172.21.15.133:0/1940434481 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:27.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.333+0000 7f8047e00640 1 -- 172.21.15.133:0/1088157022 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80400057e0 con 0x7f80400e30c0 2024-03-07T06:51:27.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.333+0000 7f66f2a00640 1 -- start start 2024-03-07T06:51:27.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.333+0000 7fb472000640 1 --2- 172.21.15.133:0/1940434481 >> v2:172.21.15.133:3300/0 conn(0x7fb46c005e90 0x7fb46c0062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.334+0000 7fb473400640 1 --2- 172.21.15.133:0/1940434481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb46c006810 0x7fb46c006ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.334+0000 7fa7e2a00640 1 -- 172.21.15.133:0/572426371 >> v2:172.21.15.161:3300/0 conn(0x7fa7e40d3810 msgr2=0x7fa7e4162460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:27.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.334+0000 7fa7e2a00640 1 --2- 172.21.15.133:0/572426371 >> v2:172.21.15.161:3300/0 conn(0x7fa7e40d3810 0x7fa7e4162460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.334+0000 7fa7e2a00640 1 -- 172.21.15.133:0/572426371 >> v1:172.21.15.161:6789/0 conn(0x7fa7e41551f0 legacy=0x7fa7e4161d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:27.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.334+0000 7fa7e2a00640 1 -- 172.21.15.133:0/572426371 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e40b4970 con 0x7fa7e4154e10 2024-03-07T06:51:27.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.334+0000 7fa708a00640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa6e406b5a0 0x7fa6e406da60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0d6d60 tx=0x7fa6f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.334+0000 7f3b13e00640 1 --2- 172.21.15.133:0/1787508916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b140b43f0 0x7f3b14162570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3b14161f80 tx=0x7f3b0002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cd2a460bf1b7d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.335+0000 7fb473400640 1 -- 172.21.15.133:0/1940434481 >> v2:172.21.15.133:3300/0 conn(0x7fb46c005e90 msgr2=0x7fb46c0062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:27.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.335+0000 7fb473400640 1 --2- 172.21.15.133:0/1940434481 >> v2:172.21.15.133:3300/0 conn(0x7fb46c005e90 0x7fb46c0062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.335+0000 7fb473400640 1 -- 172.21.15.133:0/1940434481 >> v1:172.21.15.133:6789/0 conn(0x7fb46c0ead00 legacy=0x7fb46c04fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.335+0000 7fb473400640 1 -- 172.21.15.133:0/1940434481 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c053ce0 con 0x7fb46c006810 2024-03-07T06:51:27.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.335+0000 7fa7e2a00640 1 --2- 172.21.15.133:0/572426371 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7e4154e10 0x7fa7e40d32d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009870 tx=0x7fa7dc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dad68b448b4bc043 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f66f2a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f66ec15e400 0x7f66ec0afb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7fb473400640 1 --2- 172.21.15.133:0/1940434481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb46c006810 0x7fb46c006ca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb474050ba0 tx=0x7fb4740a6c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d730ef86bebabf3f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f66f2a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f66ec0b3c10 0x7f66ec071980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f66f2a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66ec0cdf80 con 0x7f66ec071ec0 2024-03-07T06:51:27.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f66f2a00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66ec073040 con 0x7f66ec0b3c10 2024-03-07T06:51:27.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f66f2a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66ec0ce1f0 con 0x7f66ec15e400 2024-03-07T06:51:27.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f7cc0a00640 1 -- 172.21.15.133:0/3765001665 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4035020 con 0x7f7cc40ae880 2024-03-07T06:51:27.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f7cc0a00640 1 -- 172.21.15.133:0/3765001665 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb4031870 con 0x7f7cc40ae880 2024-03-07T06:51:27.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f7cc0a00640 1 -- 172.21.15.133:0/3765001665 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4031c40 con 0x7f7cc40ae880 2024-03-07T06:51:27.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f3b13400640 1 -- 172.21.15.133:0/1787508916 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00031780 con 0x7f3b140b43f0 2024-03-07T06:51:27.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f3b13400640 1 -- 172.21.15.133:0/1787508916 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b00037070 con 0x7f3b140b43f0 2024-03-07T06:51:27.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f3b13400640 1 -- 172.21.15.133:0/1787508916 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00031b40 con 0x7f3b140b43f0 2024-03-07T06:51:27.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f66ebe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f66ec15e400 0x7f66ec0afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f66ebe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f66ec15e400 0x7f66ec0afb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58940/0 (socket says 172.21.15.133:58940) 2024-03-07T06:51:27.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f66ebe00640 1 -- 172.21.15.133:0/2103854508 learned_addr learned my addr 172.21.15.133:0/2103854508 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:27.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.336+0000 7f803ea00640 1 -- 172.21.15.133:0/1088157022 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8048093070 con 0x7f80400e30c0 2024-03-07T06:51:27.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.337+0000 7f66e3e00640 1 --2- 172.21.15.133:0/2103854508 >> v2:172.21.15.133:3300/0 conn(0x7f66ec0b3c10 0x7f66ec071980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.337+0000 7f3b1ba00640 1 -- 172.21.15.133:0/1787508916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b140b43f0 msgr2=0x7f3b14162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.337+0000 7f3b1ba00640 1 --2- 172.21.15.133:0/1787508916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b140b43f0 0x7f3b14162570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3b14161f80 tx=0x7f3b0002b820 comp rx=0 tx=0).stop 2024-03-07T06:51:27.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.337+0000 7fa701600640 1 -- 172.21.15.133:0/3645416259 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa70409a070 con 0x7fa6fc053c40 2024-03-07T06:51:27.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.337+0000 7f3b1ba00640 1 -- 172.21.15.133:0/1787508916 shutdown_connections 2024-03-07T06:51:27.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.338+0000 7f803ea00640 1 --2- 172.21.15.133:0/1088157022 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f802806b5d0 0x7f802806da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.337+0000 7f66ebe00640 1 -- 172.21.15.133:0/2103854508 >> v2:172.21.15.133:3300/0 conn(0x7f66ec0b3c10 msgr2=0x7f66ec071980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:27.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.337+0000 7f66ebe00640 1 --2- 172.21.15.133:0/2103854508 >> v2:172.21.15.133:3300/0 conn(0x7f66ec0b3c10 0x7f66ec071980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.337+0000 7f66ebe00640 1 -- 172.21.15.133:0/2103854508 >> v1:172.21.15.161:6789/0 conn(0x7f66ec071ec0 legacy=0x7f66ec161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.337+0000 7f66ebe00640 1 -- 172.21.15.133:0/2103854508 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ec073320 con 0x7f66ec15e400 2024-03-07T06:51:27.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.337+0000 7f3b1ba00640 1 --2- 172.21.15.133:0/1787508916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b140b43f0 0x7f3b14162570 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.337+0000 7f3b1ba00640 1 --2- 172.21.15.133:0/1787508916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b140b5dd0 0x7f3b14144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.337+0000 7f3b1ba00640 1 -- 172.21.15.133:0/1787508916 >> 172.21.15.133:0/1787508916 conn(0x7f3b140d0e70 msgr2=0x7f3b140d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.338+0000 7f803ea00640 1 -- 172.21.15.133:0/1088157022 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8048104ec0 con 0x7f80400e30c0 2024-03-07T06:51:27.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.337+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3765001665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cc40ae880 msgr2=0x7f7cc406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.337+0000 7f7cc8c00640 1 --2- 172.21.15.133:0/3765001665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cc40ae880 0x7f7cc406d2a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7cc406cf90 tx=0x7f7cb402b910 comp rx=0 tx=0).stop 2024-03-07T06:51:27.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.338+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3765001665 shutdown_connections 2024-03-07T06:51:27.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.338+0000 7f7cc8c00640 1 --2- 172.21.15.133:0/3765001665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cc40ae880 0x7f7cc406d2a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.338+0000 7f7cc8c00640 1 --2- 172.21.15.133:0/3765001665 >> v2:172.21.15.161:3300/0 conn(0x7f7cc40acd00 0x7f7cc40ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.338+0000 7f7cc8c00640 1 --2- 172.21.15.133:0/3765001665 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cc415a3e0 0x7f7cc4072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.338+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3765001665 >> 172.21.15.133:0/3765001665 conn(0x7f7cc40b3120 msgr2=0x7f7cc40b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.338+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3765001665 shutdown_connections 2024-03-07T06:51:27.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.338+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3765001665 wait complete. 2024-03-07T06:51:27.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.339+0000 7f8046a00640 1 --2- 172.21.15.133:0/1088157022 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f802806b5d0 0x7f802806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.338+0000 7f3b1ba00640 1 -- 172.21.15.133:0/1787508916 shutdown_connections 2024-03-07T06:51:27.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.339+0000 7f3b1ba00640 1 -- 172.21.15.133:0/1787508916 wait complete. 2024-03-07T06:51:27.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.340+0000 7f66ebe00640 1 --2- 172.21.15.133:0/2103854508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ec15e400 0x7f66ec0afb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66ec06cda0 tx=0x7f66d402b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b172b5b8fe3ae92 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.340+0000 7f7cc8c00640 1 Processor -- start 2024-03-07T06:51:27.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.340+0000 7f8046a00640 1 --2- 172.21.15.133:0/1088157022 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f802806b5d0 0x7f802806da90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8034000f60 tx=0x7f8034009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.340+0000 7f3b1ba00640 1 Processor -- start 2024-03-07T06:51:27.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.340+0000 7f66eb400640 1 -- 172.21.15.133:0/2103854508 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d4035020 con 0x7f66ec15e400 2024-03-07T06:51:27.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.340+0000 7f66eb400640 1 -- 172.21.15.133:0/2103854508 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d4031880 con 0x7f66ec15e400 2024-03-07T06:51:27.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.340+0000 7f7cc8c00640 1 -- start start 2024-03-07T06:51:27.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.341+0000 7f3b1ba00640 1 -- start start 2024-03-07T06:51:27.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.341+0000 7f7cc8c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cc40acd00 0x7f7cc414e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.341+0000 7f7cc8c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cc415a3e0 0x7f7cc4166370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.341+0000 7f66f2a00640 1 -- 172.21.15.133:0/2103854508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ec15e400 msgr2=0x7f66ec0afb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.341+0000 7f66f2a00640 1 --2- 172.21.15.133:0/2103854508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ec15e400 0x7f66ec0afb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f66ec06cda0 tx=0x7f66d402b920 comp rx=0 tx=0).stop 2024-03-07T06:51:27.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.341+0000 7f66f2a00640 1 -- 172.21.15.133:0/2103854508 shutdown_connections 2024-03-07T06:51:27.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.341+0000 7f66f2a00640 1 --2- 172.21.15.133:0/2103854508 >> v2:172.21.15.133:3300/0 conn(0x7f66ec0b3c10 0x7f66ec071980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.341+0000 7f66f2a00640 1 --2- 172.21.15.133:0/2103854508 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ec15e400 0x7f66ec0afb30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.341+0000 7f66f2a00640 1 -- 172.21.15.133:0/2103854508 >> 172.21.15.133:0/2103854508 conn(0x7f66ec0b3120 msgr2=0x7f66ec0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.341+0000 7f7cc8c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cc41668b0 0x7f7cc40aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.341+0000 7f7cc8c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40b3c10 con 0x7f7cc415a3e0 2024-03-07T06:51:27.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.341+0000 7f7cc8c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40ade20 con 0x7f7cc41668b0 2024-03-07T06:51:27.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.341+0000 7f7cc8c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40b3d90 con 0x7f7cc40acd00 2024-03-07T06:51:27.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f7cc1e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cc40acd00 0x7f7cc414e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f7cc1400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cc415a3e0 0x7f7cc4166370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f66f2a00640 1 -- 172.21.15.133:0/2103854508 shutdown_connections 2024-03-07T06:51:27.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f66f2a00640 1 -- 172.21.15.133:0/2103854508 wait complete. 2024-03-07T06:51:27.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f7cc1e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cc40acd00 0x7f7cc414e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:42090/0 (socket says 172.21.15.133:42090) 2024-03-07T06:51:27.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f7cc1e00640 1 -- 172.21.15.133:0/3989862055 learned_addr learned my addr 172.21.15.133:0/3989862055 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:27.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f703f600640 1 Processor -- start 2024-03-07T06:51:27.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f3b1ba00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b140b5dd0 0x7f3b140d39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f3b1ba00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b14137dd0 0x7f3b141f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f3b1ba00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b1415b0a0 0x7f3b1415b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f3b1ba00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b140b61b0 con 0x7f3b14137dd0 2024-03-07T06:51:27.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f3b1ba00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14162050 con 0x7f3b1415b0a0 2024-03-07T06:51:27.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f3b1ba00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b140b4870 con 0x7f3b140b5dd0 2024-03-07T06:51:27.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f3b13e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b14137dd0 0x7f3b141f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f3b13e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b14137dd0 0x7f3b141f80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38634/0 (socket says 172.21.15.133:38634) 2024-03-07T06:51:27.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f3b13e00640 1 -- 172.21.15.133:0/2434942846 learned_addr learned my addr 172.21.15.133:0/2434942846 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:27.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.342+0000 7f7cc2800640 1 --2- 172.21.15.133:0/3989862055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cc41668b0 0x7f7cc40aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f3b19000640 1 --2- 172.21.15.133:0/2434942846 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b140b5dd0 0x7f3b140d39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f703f600640 1 -- start start 2024-03-07T06:51:27.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f3b19a00640 1 --2- 172.21.15.133:0/2434942846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b1415b0a0 0x7f3b1415b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f3b13e00640 1 -- 172.21.15.133:0/2434942846 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b140b5dd0 msgr2=0x7f3b140d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f7cc1400640 1 -- 172.21.15.133:0/3989862055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cc40acd00 msgr2=0x7f7cc414e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f7cc1400640 1 --2- 172.21.15.133:0/3989862055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cc40acd00 0x7f7cc414e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f7cc1400640 1 -- 172.21.15.133:0/3989862055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cc41668b0 msgr2=0x7f7cc40aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f7cc1400640 1 --2- 172.21.15.133:0/3989862055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cc41668b0 0x7f7cc40aff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f7cc1400640 1 -- 172.21.15.133:0/3989862055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc4166e30 con 0x7f7cc415a3e0 2024-03-07T06:51:27.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f3b13e00640 1 --2- 172.21.15.133:0/2434942846 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b140b5dd0 0x7f3b140d39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f3b13e00640 1 -- 172.21.15.133:0/2434942846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b1415b0a0 msgr2=0x7f3b1415b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f3b13e00640 1 --2- 172.21.15.133:0/2434942846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b1415b0a0 0x7f3b1415b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.343+0000 7f3b13e00640 1 -- 172.21.15.133:0/2434942846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b14138250 con 0x7f3b14137dd0 2024-03-07T06:51:27.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.344+0000 7f7cc1400640 1 --2- 172.21.15.133:0/3989862055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cc415a3e0 0x7f7cc4166370 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc007e60 tx=0x7f7cbc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.344+0000 7f3b13e00640 1 --2- 172.21.15.133:0/2434942846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b14137dd0 0x7f3b141f80c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3b00034040 tx=0x7f3b000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.344+0000 7f3b10c00640 1 -- 172.21.15.133:0/2434942846 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00004030 con 0x7f3b14137dd0 2024-03-07T06:51:27.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.344+0000 7f7cba000640 1 -- 172.21.15.133:0/3989862055 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc014420 con 0x7f7cc415a3e0 2024-03-07T06:51:27.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.344+0000 7f3b10c00640 1 -- 172.21.15.133:0/2434942846 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b000041d0 con 0x7f3b14137dd0 2024-03-07T06:51:27.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.344+0000 7f3b1ba00640 1 -- 172.21.15.133:0/2434942846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1406b1e0 con 0x7f3b14137dd0 2024-03-07T06:51:27.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.344+0000 7f3b10c00640 1 -- 172.21.15.133:0/2434942846 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b00003610 con 0x7f3b14137dd0 2024-03-07T06:51:27.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.344+0000 7f3b1ba00640 1 -- 172.21.15.133:0/2434942846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1406b7a0 con 0x7f3b14137dd0 2024-03-07T06:51:27.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.345+0000 7f66f2a00640 1 Processor -- start 2024-03-07T06:51:27.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.345+0000 7f66f2a00640 1 -- start start 2024-03-07T06:51:27.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.345+0000 7f703f600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7038066eb0 0x7f7038070400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.345+0000 7f703f600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f703806e1c0 0x7f7038070940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.345+0000 7f703f600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7038071af0 con 0x7f7038067290 2024-03-07T06:51:27.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.345+0000 7f703f600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7038070f70 con 0x7f703806e1c0 2024-03-07T06:51:27.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.345+0000 7f703f600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7038071250 con 0x7f7038066eb0 2024-03-07T06:51:27.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.345+0000 7f703cc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7038066eb0 0x7f7038070400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.345+0000 7f703cc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7038066eb0 0x7f7038070400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:42102/0 (socket says 172.21.15.133:42102) 2024-03-07T06:51:27.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.345+0000 7f703cc00640 1 -- 172.21.15.133:0/4256205498 learned_addr learned my addr 172.21.15.133:0/4256205498 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:27.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.345+0000 7f703d600640 1 --2- 172.21.15.133:0/4256205498 >> v2:172.21.15.133:3300/0 conn(0x7f703806e1c0 0x7f7038070940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.344+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3989862055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc409c190 con 0x7f7cc415a3e0 2024-03-07T06:51:27.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.344+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3989862055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc409c750 con 0x7f7cc415a3e0 2024-03-07T06:51:27.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.346+0000 7f703cc00640 1 -- 172.21.15.133:0/4256205498 >> v2:172.21.15.133:3300/0 conn(0x7f703806e1c0 msgr2=0x7f7038070940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:27.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.346+0000 7f703cc00640 1 --2- 172.21.15.133:0/4256205498 >> v2:172.21.15.133:3300/0 conn(0x7f703806e1c0 0x7f7038070940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.346+0000 7f703cc00640 1 -- 172.21.15.133:0/4256205498 >> v1:172.21.15.161:6789/0 conn(0x7f7038067290 legacy=0x7f703806dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.346+0000 7f703cc00640 1 -- 172.21.15.133:0/4256205498 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70380a9640 con 0x7f7038066eb0 2024-03-07T06:51:27.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.345+0000 7f7cba000640 1 -- 172.21.15.133:0/3989862055 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cbc019070 con 0x7f7cc415a3e0 2024-03-07T06:51:27.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.346+0000 7f7cba000640 1 -- 172.21.15.133:0/3989862055 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc014800 con 0x7f7cc415a3e0 2024-03-07T06:51:27.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.347+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3989862055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c9c005290 con 0x7f7cc415a3e0 2024-03-07T06:51:27.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.347+0000 7f3b0a000640 1 -- 172.21.15.133:0/2434942846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae8005290 con 0x7f3b14137dd0 2024-03-07T06:51:27.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.347+0000 7f703cc00640 1 --2- 172.21.15.133:0/4256205498 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7038066eb0 0x7f7038070400 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7028009870 tx=0x7f702802ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c1600d260c95967 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.348+0000 7f66f2a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f66ec071ec0 0x7f66ec09d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.348+0000 7f66f2a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ec0b3c10 0x7f66ec14a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.349+0000 7f7cba000640 1 -- 172.21.15.133:0/3989862055 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7cbc0149a0 con 0x7f7cc415a3e0 2024-03-07T06:51:27.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.349+0000 7f66ebe00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f66ec071ec0 0x7f66ec09d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.349+0000 7f66ebe00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f66ec071ec0 0x7f66ec09d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:42104/0 (socket says 172.21.15.133:42104) 2024-03-07T06:51:27.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.349+0000 7f66f2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66ec14ce10 0x7f66ec14d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.349+0000 7f66ebe00640 1 -- 172.21.15.133:0/4182643260 learned_addr learned my addr 172.21.15.133:0/4182643260 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:27.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.349+0000 7f66f2a00640 1 -- 172.21.15.133:0/4182643260 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec0724b0 con 0x7f66ec14ce10 2024-03-07T06:51:27.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.349+0000 7f66f2a00640 1 -- 172.21.15.133:0/4182643260 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec072330 con 0x7f66ec0b3c10 2024-03-07T06:51:27.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.349+0000 7f66f2a00640 1 -- 172.21.15.133:0/4182643260 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ec072630 con 0x7f66ec071ec0 2024-03-07T06:51:27.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.349+0000 7f3b10c00640 1 -- 172.21.15.133:0/2434942846 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3b000037b0 con 0x7f3b14137dd0 2024-03-07T06:51:27.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.349+0000 7f7cba000640 1 --2- 172.21.15.133:0/3989862055 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7ca406b4d0 0x7f7ca406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.350+0000 7f7cba000640 1 -- 172.21.15.133:0/3989862055 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cbc08b710 con 0x7f7cc415a3e0 2024-03-07T06:51:27.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.350+0000 7f66e3e00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ec0b3c10 0x7f66ec14a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.350+0000 7f66ebe00640 1 -- 172.21.15.133:0/4182643260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ec0b3c10 msgr2=0x7f66ec14a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.350+0000 7f66ebe00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ec0b3c10 0x7f66ec14a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.350+0000 7f66ebe00640 1 -- 172.21.15.133:0/4182643260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66ec14ce10 msgr2=0x7f66ec14d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.350+0000 7f66f0a00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66ec14ce10 0x7f66ec14d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.350+0000 7f7cc1e00640 1 --2- 172.21.15.133:0/3989862055 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7ca406b4d0 0x7f7ca406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.350+0000 7f66ebe00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66ec14ce10 0x7f66ec14d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.350+0000 7f66ebe00640 1 -- 172.21.15.133:0/4182643260 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ec09d560 con 0x7f66ec071ec0 2024-03-07T06:51:27.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.350+0000 7f66f0a00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66ec14ce10 0x7f66ec14d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:27.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.351+0000 7f66ebe00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f66ec071ec0 0x7f66ec09d020 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f66d4000c00 tx=0x7f66d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.351+0000 7f3b10c00640 1 --2- 172.21.15.133:0/2434942846 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3af406b5d0 0x7f3af406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.351+0000 7f3b10c00640 1 -- 172.21.15.133:0/2434942846 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b000ac210 con 0x7f3b14137dd0 2024-03-07T06:51:27.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.351+0000 7f7cc1e00640 1 --2- 172.21.15.133:0/3989862055 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7ca406b4d0 0x7f7ca406d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0007a60 tx=0x7f7cb0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.351+0000 7f3b19000640 1 --2- 172.21.15.133:0/2434942846 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3af406b5d0 0x7f3af406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.352+0000 7f3b19000640 1 --2- 172.21.15.133:0/2434942846 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3af406b5d0 0x7f3af406da90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009ad0 tx=0x7f3b04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.355+0000 7fa7e1600640 1 -- 172.21.15.133:0/572426371 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc034070 con 0x7fa7e4154e10 2024-03-07T06:51:27.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.355+0000 7fb471600640 1 -- 172.21.15.133:0/1940434481 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740b4070 con 0x7fb46c006810 2024-03-07T06:51:27.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.355+0000 7fa7e1600640 1 -- 172.21.15.133:0/572426371 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7dc030840 con 0x7fa7e4154e10 2024-03-07T06:51:27.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.355+0000 7fa7e1600640 1 -- 172.21.15.133:0/572426371 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7dc030c10 con 0x7fa7e4154e10 2024-03-07T06:51:27.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.355+0000 7f66e8c00640 1 -- 172.21.15.133:0/4182643260 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d4040070 con 0x7f66ec071ec0 2024-03-07T06:51:27.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.356+0000 7f66f2a00640 1 -- 172.21.15.133:0/4182643260 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ec09d7e0 con 0x7f66ec071ec0 2024-03-07T06:51:27.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.356+0000 7fb471600640 1 -- 172.21.15.133:0/1940434481 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4740a7800 con 0x7fb46c006810 2024-03-07T06:51:27.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.356+0000 7fb471600640 1 -- 172.21.15.133:0/1940434481 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740a7c30 con 0x7fb46c006810 2024-03-07T06:51:27.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.355+0000 7f7037e00640 1 -- 172.21.15.133:0/4256205498 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702802f830 con 0x7f7038066eb0 2024-03-07T06:51:27.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.356+0000 7f66e8c00640 1 -- 172.21.15.133:0/4182643260 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d403a830 con 0x7f66ec071ec0 2024-03-07T06:51:27.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.356+0000 7f66f2a00640 1 -- 172.21.15.133:0/4182643260 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ec14b100 con 0x7f66ec071ec0 2024-03-07T06:51:27.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.356+0000 7f66e8c00640 1 -- 172.21.15.133:0/4182643260 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d4035040 con 0x7f66ec071ec0 2024-03-07T06:51:27.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.356+0000 7f803ea00640 1 -- 172.21.15.133:0/1088157022 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f80401793f0 con 0x7f80400e30c0 2024-03-07T06:51:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.356+0000 7f7037e00640 1 -- 172.21.15.133:0/4256205498 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702802f9d0 con 0x7f7038066eb0 2024-03-07T06:51:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.356+0000 7f7037e00640 1 -- 172.21.15.133:0/4256205498 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702802fda0 con 0x7f7038066eb0 2024-03-07T06:51:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7fa7e3e00640 1 -- 172.21.15.133:0/572426371 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7e4154e10 msgr2=0x7fa7e40d32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7fa7e3e00640 1 --2- 172.21.15.133:0/572426371 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7e4154e10 0x7fa7e40d32d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009870 tx=0x7fa7dc02b980 comp rx=0 tx=0).stop 2024-03-07T06:51:27.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7fb473e00640 1 -- 172.21.15.133:0/1940434481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb46c006810 msgr2=0x7fb46c006ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7fb473e00640 1 --2- 172.21.15.133:0/1940434481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb46c006810 0x7fb46c006ca0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb474050ba0 tx=0x7fb4740a6c60 comp rx=0 tx=0).stop 2024-03-07T06:51:27.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7fa7e3e00640 1 -- 172.21.15.133:0/572426371 shutdown_connections 2024-03-07T06:51:27.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7fa7e3e00640 1 --2- 172.21.15.133:0/572426371 >> v2:172.21.15.161:3300/0 conn(0x7fa7e40d3810 0x7fa7e4162460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7fa7e3e00640 1 --2- 172.21.15.133:0/572426371 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7e4154e10 0x7fa7e40d32d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7fa7e3e00640 1 -- 172.21.15.133:0/572426371 >> 172.21.15.133:0/572426371 conn(0x7fa7e4070400 msgr2=0x7fa7e40d2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7f66f2a00640 1 -- 172.21.15.133:0/4182643260 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66ec0cf610 con 0x7f66ec071ec0 2024-03-07T06:51:27.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7f703f600640 1 -- 172.21.15.133:0/4256205498 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7038066eb0 msgr2=0x7f7038070400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7f703f600640 1 --2- 172.21.15.133:0/4256205498 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7038066eb0 0x7f7038070400 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7028009870 tx=0x7f702802ba10 comp rx=0 tx=0).stop 2024-03-07T06:51:27.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7fa7e3e00640 1 -- 172.21.15.133:0/572426371 shutdown_connections 2024-03-07T06:51:27.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7fa7e3e00640 1 -- 172.21.15.133:0/572426371 wait complete. 2024-03-07T06:51:27.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7f703f600640 1 -- 172.21.15.133:0/4256205498 shutdown_connections 2024-03-07T06:51:27.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7f703f600640 1 --2- 172.21.15.133:0/4256205498 >> v2:172.21.15.133:3300/0 conn(0x7f703806e1c0 0x7f7038070940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7f703f600640 1 --2- 172.21.15.133:0/4256205498 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7038066eb0 0x7f7038070400 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.357+0000 7f703f600640 1 -- 172.21.15.133:0/4256205498 >> 172.21.15.133:0/4256205498 conn(0x7f70380cd160 msgr2=0x7f70380cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.358+0000 7f703f600640 1 -- 172.21.15.133:0/4256205498 shutdown_connections 2024-03-07T06:51:27.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.358+0000 7f703f600640 1 -- 172.21.15.133:0/4256205498 wait complete. 2024-03-07T06:51:27.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.359+0000 7f66e8c00640 1 -- 172.21.15.133:0/4182643260 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f66d404b060 con 0x7f66ec071ec0 2024-03-07T06:51:27.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.360+0000 7f3b10c00640 1 -- 172.21.15.133:0/2434942846 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3b00040070 con 0x7f3b14137dd0 2024-03-07T06:51:27.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.360+0000 7fb473e00640 1 -- 172.21.15.133:0/1940434481 shutdown_connections 2024-03-07T06:51:27.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.360+0000 7fb473e00640 1 --2- 172.21.15.133:0/1940434481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb46c006810 0x7fb46c006ca0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.360+0000 7fb473e00640 1 --2- 172.21.15.133:0/1940434481 >> v2:172.21.15.133:3300/0 conn(0x7fb46c005e90 0x7fb46c0062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.360+0000 7fb473e00640 1 -- 172.21.15.133:0/1940434481 >> 172.21.15.133:0/1940434481 conn(0x7fb46c009fb0 msgr2=0x7fb46c00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.360+0000 7fb473e00640 1 -- 172.21.15.133:0/1940434481 shutdown_connections 2024-03-07T06:51:27.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.360+0000 7fb473e00640 1 -- 172.21.15.133:0/1940434481 wait complete. 2024-03-07T06:51:27.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.360+0000 7f66e8c00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f66c806b500 0x7f66c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.361+0000 7f66e8c00640 1 -- 172.21.15.133:0/4182643260 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66d40ac6d0 con 0x7f66ec071ec0 2024-03-07T06:51:27.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.361+0000 7fa7e3e00640 1 Processor -- start 2024-03-07T06:51:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.361+0000 7f66e3e00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f66c806b500 0x7f66c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.361+0000 7fa7e3e00640 1 -- start start 2024-03-07T06:51:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.362+0000 7f703f600640 1 Processor -- start 2024-03-07T06:51:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.362+0000 7f703f600640 1 -- start start 2024-03-07T06:51:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.362+0000 7fb473e00640 1 Processor -- start 2024-03-07T06:51:27.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.363+0000 7fb473e00640 1 -- start start 2024-03-07T06:51:27.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.363+0000 7fa7e3e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7e40d3810 0x7fa7e413e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.363+0000 7f66e3e00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f66c806b500 0x7f66c806d9c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f66dc006fd0 tx=0x7f66dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.363+0000 7fa7e3e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7e4154e10 0x7fa7e414c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.363+0000 7fa7e3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e41551f0 0x7fa7e41555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.363+0000 7fa7e3e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40b6350 con 0x7fa7e41551f0 2024-03-07T06:51:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.363+0000 7fa7e3e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40b61d0 con 0x7fa7e40d3810 2024-03-07T06:51:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.363+0000 7fa7e3e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e40b64d0 con 0x7fa7e4154e10 2024-03-07T06:51:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.363+0000 7fa7e3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e41551f0 0x7fa7e41555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.363+0000 7fa7e3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e41551f0 0x7fa7e41555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38668/0 (socket says 172.21.15.133:38668) 2024-03-07T06:51:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.363+0000 7fa7e3400640 1 -- 172.21.15.133:0/1485931404 learned_addr learned my addr 172.21.15.133:0/1485931404 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:27.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7fa7e2a00640 1 --2- 172.21.15.133:0/1485931404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7e40d3810 0x7fa7e413e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7fb473e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb46c005e90 0x7fb46c0e8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7fb473e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb46c006810 0x7fb46c044af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7fb473e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb46c0ead00 0x7fb46c04aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7fb473e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0547c0 con 0x7fb46c0ead00 2024-03-07T06:51:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7fb473e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0501a0 con 0x7fb46c006810 2024-03-07T06:51:27.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7fb473e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c054940 con 0x7fb46c005e90 2024-03-07T06:51:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7fa7e2000640 1 --2- 172.21.15.133:0/1485931404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7e4154e10 0x7fa7e414c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7fa7e3400640 1 -- 172.21.15.133:0/1485931404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7e4154e10 msgr2=0x7fa7e414c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7f703f600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7038066eb0 0x7f70381342c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7f703f600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7038067290 0x7f70381493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7f703f600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f703806e1c0 0x7f7038138900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7f703f600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703815a820 con 0x7f7038067290 2024-03-07T06:51:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7f703f600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703815a6a0 con 0x7f7038066eb0 2024-03-07T06:51:27.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7f703f600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703815a9a0 con 0x7f703806e1c0 2024-03-07T06:51:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7fa7e3400640 1 --2- 172.21.15.133:0/1485931404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7e4154e10 0x7fa7e414c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7fb472a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb46c005e90 0x7fb46c0e8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.365+0000 7fb473400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb46c0ead00 0x7fb46c04aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.365+0000 7fb472a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb46c005e90 0x7fb46c0e8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:42126/0 (socket says 172.21.15.133:42126) 2024-03-07T06:51:27.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.365+0000 7fb472a00640 1 -- 172.21.15.133:0/4134163351 learned_addr learned my addr 172.21.15.133:0/4134163351 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:27.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7fa7e3400640 1 -- 172.21.15.133:0/1485931404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7e40d3810 msgr2=0x7fa7e413e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.365+0000 7fb472000640 1 --2- 172.21.15.133:0/4134163351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb46c006810 0x7fb46c044af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7fa7e3400640 1 --2- 172.21.15.133:0/1485931404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7e40d3810 0x7fa7e413e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.364+0000 7fa7e3400640 1 -- 172.21.15.133:0/1485931404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e4155b30 con 0x7fa7e41551f0 2024-03-07T06:51:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.365+0000 7f703cc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7038066eb0 0x7f70381342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.365+0000 7f703cc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7038066eb0 0x7f70381342c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:58992/0 (socket says 172.21.15.133:58992) 2024-03-07T06:51:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.365+0000 7f703cc00640 1 -- 172.21.15.133:0/3339072282 learned_addr learned my addr 172.21.15.133:0/3339072282 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.365+0000 7f703d600640 1 --2- 172.21.15.133:0/3339072282 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f703806e1c0 0x7f7038138900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.365+0000 7f702fe00640 1 --2- 172.21.15.133:0/3339072282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7038067290 0x7f70381493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.365+0000 7fb473400640 1 -- 172.21.15.133:0/4134163351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb46c005e90 msgr2=0x7fb46c0e8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.365+0000 7fb473400640 1 --2- 172.21.15.133:0/4134163351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb46c005e90 0x7fb46c0e8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.365+0000 7fb473400640 1 -- 172.21.15.133:0/4134163351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb46c006810 msgr2=0x7fb46c044af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.365+0000 7fb473400640 1 --2- 172.21.15.133:0/4134163351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb46c006810 0x7fb46c044af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.365+0000 7fb473400640 1 -- 172.21.15.133:0/4134163351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c0e88d0 con 0x7fb46c0ead00 2024-03-07T06:51:27.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.366+0000 7fb473400640 1 --2- 172.21.15.133:0/4134163351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb46c0ead00 0x7fb46c04aa10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb474067da0 tx=0x7fb4740738f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.366+0000 7f702fe00640 1 -- 172.21.15.133:0/3339072282 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f703806e1c0 msgr2=0x7f7038138900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.366+0000 7f702fe00640 1 --2- 172.21.15.133:0/3339072282 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f703806e1c0 0x7f7038138900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.366+0000 7f702fe00640 1 -- 172.21.15.133:0/3339072282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7038066eb0 msgr2=0x7f70381342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.366+0000 7f702fe00640 1 --2- 172.21.15.133:0/3339072282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7038066eb0 0x7f70381342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.366+0000 7f702fe00640 1 -- 172.21.15.133:0/3339072282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703814a790 con 0x7f7038067290 2024-03-07T06:51:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.366+0000 7f702fe00640 1 --2- 172.21.15.133:0/3339072282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7038067290 0x7f70381493e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f702000c390 tx=0x7f702000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.367+0000 7fa7e3400640 1 --2- 172.21.15.133:0/1485931404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e41551f0 0x7fa7e41555f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400e960 tx=0x7fa7d400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.367+0000 7fa7db400640 1 -- 172.21.15.133:0/1485931404 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4017070 con 0x7fa7e41551f0 2024-03-07T06:51:27.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.368+0000 7fa7e3e00640 1 -- 172.21.15.133:0/1485931404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e4155dd0 con 0x7fa7e41551f0 2024-03-07T06:51:27.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.367+0000 7fb46aa00640 1 -- 172.21.15.133:0/4134163351 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740c1070 con 0x7fb46c0ead00 2024-03-07T06:51:27.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.367+0000 7fb473e00640 1 -- 172.21.15.133:0/4134163351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46c0453f0 con 0x7fb46c0ead00 2024-03-07T06:51:27.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.367+0000 7fb46aa00640 1 -- 172.21.15.133:0/4134163351 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4740a7600 con 0x7fb46c0ead00 2024-03-07T06:51:27.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.367+0000 7fb473e00640 1 -- 172.21.15.133:0/4134163351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46c043bb0 con 0x7fb46c0ead00 2024-03-07T06:51:27.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.368+0000 7fb46aa00640 1 -- 172.21.15.133:0/4134163351 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740b4040 con 0x7fb46c0ead00 2024-03-07T06:51:27.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.368+0000 7fa7e3e00640 1 -- 172.21.15.133:0/1485931404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e4156390 con 0x7fa7e41551f0 2024-03-07T06:51:27.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.369+0000 7fa7db400640 1 -- 172.21.15.133:0/1485931404 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d400c980 con 0x7fa7e41551f0 2024-03-07T06:51:27.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.369+0000 7fa7db400640 1 -- 172.21.15.133:0/1485931404 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d400cd70 con 0x7fa7e41551f0 2024-03-07T06:51:27.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.370+0000 7f7cba000640 1 -- 172.21.15.133:0/3989862055 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7cbc059f20 con 0x7f7cc415a3e0 2024-03-07T06:51:27.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.370+0000 7f7035600640 1 -- 172.21.15.133:0/3339072282 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020005220 con 0x7f7038067290 2024-03-07T06:51:27.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.370+0000 7f7035600640 1 -- 172.21.15.133:0/3339072282 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70200053c0 con 0x7f7038067290 2024-03-07T06:51:27.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.370+0000 7f7035600640 1 -- 172.21.15.133:0/3339072282 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020007230 con 0x7f7038067290 2024-03-07T06:51:27.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.370+0000 7fb473e00640 1 -- 172.21.15.133:0/4134163351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46c005390 con 0x7fb46c0ead00 2024-03-07T06:51:27.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.370+0000 7fa7e3e00640 1 -- 172.21.15.133:0/1485931404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e4062c90 con 0x7fa7e41551f0 2024-03-07T06:51:27.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.370+0000 7f703f600640 1 -- 172.21.15.133:0/3339072282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703815afe0 con 0x7f7038067290 2024-03-07T06:51:27.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.371+0000 7f703f600640 1 -- 172.21.15.133:0/3339072282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703815b5a0 con 0x7f7038067290 2024-03-07T06:51:27.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.371+0000 7fb46aa00640 1 -- 172.21.15.133:0/4134163351 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb4740af440 con 0x7fb46c0ead00 2024-03-07T06:51:27.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.372+0000 7fb46aa00640 1 --2- 172.21.15.133:0/4134163351 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb45406b700 0x7fb45406dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.372+0000 7fb46aa00640 1 -- 172.21.15.133:0/4134163351 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4740a77b0 con 0x7fb46c0ead00 2024-03-07T06:51:27.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.373+0000 7fb472a00640 1 --2- 172.21.15.133:0/4134163351 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb45406b700 0x7fb45406dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.372+0000 7f703f600640 1 -- 172.21.15.133:0/3339072282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7038062c90 con 0x7f7038067290 2024-03-07T06:51:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.374+0000 7fb472a00640 1 --2- 172.21.15.133:0/4134163351 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb45406b700 0x7fb45406dbc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb464005880 tx=0x7fb464009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.374+0000 7f66e8c00640 1 -- 172.21.15.133:0/4182643260 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f66ec0cf610 con 0x7f66ec071ec0 2024-03-07T06:51:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.376+0000 7fa7db400640 1 -- 172.21.15.133:0/1485931404 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa7d401a070 con 0x7fa7e41551f0 2024-03-07T06:51:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.378+0000 7fa7db400640 1 --2- 172.21.15.133:0/1485931404 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa7c006b4d0 0x7fa7c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.377+0000 7f7035600640 1 -- 172.21.15.133:0/3339072282 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7020019020 con 0x7f7038067290 2024-03-07T06:51:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.378+0000 7f7035600640 1 --2- 172.21.15.133:0/3339072282 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f701406b520 0x7f701406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.379+0000 7f703cc00640 1 --2- 172.21.15.133:0/3339072282 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f701406b520 0x7f701406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.380+0000 7f703cc00640 1 --2- 172.21.15.133:0/3339072282 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f701406b520 0x7f701406d9e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7028009870 tx=0x7f7028008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.381+0000 7fa7db400640 1 -- 172.21.15.133:0/1485931404 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7d4057240 con 0x7fa7e41551f0 2024-03-07T06:51:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.382+0000 7fa7e2a00640 1 --2- 172.21.15.133:0/1485931404 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa7c006b4d0 0x7fa7c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.382+0000 7f7035600640 1 -- 172.21.15.133:0/3339072282 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7020056170 con 0x7f7038067290 2024-03-07T06:51:27.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.382+0000 7fb46aa00640 1 -- 172.21.15.133:0/4134163351 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb46c0e88d0 con 0x7fb46c0ead00 2024-03-07T06:51:27.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.383+0000 7fa7e2a00640 1 --2- 172.21.15.133:0/1485931404 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa7c006b4d0 0x7fa7c006d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc036000 tx=0x7fa7dc03b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:27.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.392+0000 7fa7db400640 1 -- 172.21.15.133:0/1485931404 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa7d40598a0 con 0x7fa7e41551f0 2024-03-07T06:51:27.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.405+0000 7f7035600640 1 -- 172.21.15.133:0/3339072282 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7020020020 con 0x7f7038067290 2024-03-07T06:51:27.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.748+0000 7fa70a800640 1 -- 172.21.15.133:0/3645416259 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa6e0005580 con 0x7fa6fc053c40 2024-03-07T06:51:27.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.749+0000 7fa701600640 1 -- 172.21.15.133:0/3645416259 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa7040d35e0 con 0x7fa6fc053c40 2024-03-07T06:51:27.749 INFO:teuthology.orchestra.run.smithi133.stdout:55834574856 2024-03-07T06:51:27.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.753+0000 7fa70a800640 1 -- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa6e406b5a0 msgr2=0x7fa6e406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.753+0000 7fa70a800640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa6e406b5a0 0x7fa6e406da60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0d6d60 tx=0x7fa6f8009040 comp rx=0 tx=0).stop 2024-03-07T06:51:27.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.753+0000 7fa70a800640 1 -- 172.21.15.133:0/3645416259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6fc053c40 msgr2=0x7fa6fc03dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.753+0000 7fa70a800640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6fc053c40 0x7fa6fc03dab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa704067da0 tx=0x7fa704074e70 comp rx=0 tx=0).stop 2024-03-07T06:51:27.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.754+0000 7fa70a800640 1 -- 172.21.15.133:0/3645416259 shutdown_connections 2024-03-07T06:51:27.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.754+0000 7fa70a800640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa6e406b5a0 0x7fa6e406da60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.754+0000 7fa70a800640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6fc0e7710 0x7fa6fc03bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.754+0000 7fa70a800640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6fc0589c0 0x7fa6fc180d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.754+0000 7fa70a800640 1 --2- 172.21.15.133:0/3645416259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6fc053c40 0x7fa6fc03dab0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.754+0000 7fa70a800640 1 -- 172.21.15.133:0/3645416259 >> 172.21.15.133:0/3645416259 conn(0x7fa6fc058ee0 msgr2=0x7fa6fc0592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.755+0000 7fa70a800640 1 -- 172.21.15.133:0/3645416259 shutdown_connections 2024-03-07T06:51:27.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.755+0000 7fa70a800640 1 -- 172.21.15.133:0/3645416259 wait complete. 2024-03-07T06:51:27.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2024-03-07T06:51:27.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.786+0000 7f66f2a00640 1 -- 172.21.15.133:0/4182643260 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f66ec062c90 con 0x7f66ec071ec0 2024-03-07T06:51:27.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.787+0000 7f66e8c00640 1 -- 172.21.15.133:0/4182643260 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f66ec09d560 con 0x7f66ec071ec0 2024-03-07T06:51:27.787 INFO:teuthology.orchestra.run.smithi133.stdout:55834574855 2024-03-07T06:51:27.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.791+0000 7f66f2a00640 1 -- 172.21.15.133:0/4182643260 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f66c806b500 msgr2=0x7f66c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.791+0000 7f8047e00640 1 -- 172.21.15.133:0/1088157022 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f80400e2230 con 0x7f80400e30c0 2024-03-07T06:51:27.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.791+0000 7f66f2a00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f66c806b500 0x7f66c806d9c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f66dc006fd0 tx=0x7f66dc008040 comp rx=0 tx=0).stop 2024-03-07T06:51:27.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.791+0000 7f66f2a00640 1 -- 172.21.15.133:0/4182643260 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f66ec071ec0 msgr2=0x7f66ec09d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.791+0000 7f66f2a00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f66ec071ec0 0x7f66ec09d020 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f66d4000c00 tx=0x7f66d40046c0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.791+0000 7f66f2a00640 1 -- 172.21.15.133:0/4182643260 shutdown_connections 2024-03-07T06:51:27.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.791+0000 7f66f2a00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66ec14ce10 0x7f66ec14d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.791+0000 7f66f2a00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f66c806b500 0x7f66c806d9c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.791+0000 7f66f2a00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f66ec0b3c10 0x7f66ec14a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.791+0000 7f66f2a00640 1 --2- 172.21.15.133:0/4182643260 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f66ec071ec0 0x7f66ec09d020 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.791+0000 7f66f2a00640 1 -- 172.21.15.133:0/4182643260 >> 172.21.15.133:0/4182643260 conn(0x7f66ec0b3120 msgr2=0x7f66ec0afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.791+0000 7f66f2a00640 1 -- 172.21.15.133:0/4182643260 shutdown_connections 2024-03-07T06:51:27.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.791+0000 7f66f2a00640 1 -- 172.21.15.133:0/4182643260 wait complete. 2024-03-07T06:51:27.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.792+0000 7f803ea00640 1 -- 172.21.15.133:0/1088157022 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f80480d3c90 con 0x7f80400e30c0 2024-03-07T06:51:27.793 INFO:teuthology.orchestra.run.smithi133.stdout:55834574854 2024-03-07T06:51:27.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.792+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3989862055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7c9c005850 con 0x7f7cc415a3e0 2024-03-07T06:51:27.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.793+0000 7f7cba000640 1 -- 172.21.15.133:0/3989862055 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7cbc05dfc0 con 0x7f7cc415a3e0 2024-03-07T06:51:27.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.793+0000 7f3b0a000640 1 -- 172.21.15.133:0/2434942846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3ae8005850 con 0x7f3b14137dd0 2024-03-07T06:51:27.794 INFO:teuthology.orchestra.run.smithi133.stdout:55834574855 2024-03-07T06:51:27.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.795+0000 7fb473e00640 1 -- 172.21.15.133:0/4134163351 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb46c039080 con 0x7fb46c0ead00 2024-03-07T06:51:27.795 INFO:teuthology.orchestra.run.smithi133.stdout:55834574855 2024-03-07T06:51:27.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.796+0000 7f8047e00640 1 -- 172.21.15.133:0/1088157022 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f802806b5d0 msgr2=0x7f802806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.796+0000 7fb46aa00640 1 -- 172.21.15.133:0/4134163351 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb4740f4930 con 0x7fb46c0ead00 2024-03-07T06:51:27.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.796+0000 7f8047e00640 1 --2- 172.21.15.133:0/1088157022 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f802806b5d0 0x7f802806da90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8034000f60 tx=0x7f8034009040 comp rx=0 tx=0).stop 2024-03-07T06:51:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.796+0000 7f8047e00640 1 -- 172.21.15.133:0/1088157022 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80400e30c0 msgr2=0x7f80400e1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.796+0000 7f8047e00640 1 --2- 172.21.15.133:0/1088157022 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80400e30c0 0x7f80400e1470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8048064b50 tx=0x7f8048074e70 comp rx=0 tx=0).stop 2024-03-07T06:51:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.796+0000 7f8047e00640 1 -- 172.21.15.133:0/1088157022 shutdown_connections 2024-03-07T06:51:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.796+0000 7f8047e00640 1 --2- 172.21.15.133:0/1088157022 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80400e30c0 0x7f80400e1470 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.797 INFO:teuthology.orchestra.run.smithi133.stdout:55834574854 2024-03-07T06:51:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.796+0000 7f8047e00640 1 --2- 172.21.15.133:0/1088157022 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8040054130 0x7f80400e0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.796+0000 7f8047e00640 1 --2- 172.21.15.133:0/1088157022 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f802806b5d0 0x7f802806da90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.796+0000 7f8047e00640 1 --2- 172.21.15.133:0/1088157022 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f804003e840 0x7f8040178eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.795+0000 7f3b10c00640 1 -- 172.21.15.133:0/2434942846 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3b0002f050 con 0x7f3b14137dd0 2024-03-07T06:51:27.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.796+0000 7f8047e00640 1 -- 172.21.15.133:0/1088157022 >> 172.21.15.133:0/1088157022 conn(0x7f80400559e0 msgr2=0x7f8040039af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.797+0000 7f8047e00640 1 -- 172.21.15.133:0/1088157022 shutdown_connections 2024-03-07T06:51:27.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.797+0000 7f8047e00640 1 -- 172.21.15.133:0/1088157022 wait complete. 2024-03-07T06:51:27.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.797+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3989862055 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7ca406b4d0 msgr2=0x7f7ca406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.797+0000 7f7cc8c00640 1 --2- 172.21.15.133:0/3989862055 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7ca406b4d0 0x7f7ca406d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0007a60 tx=0x7f7cb0008040 comp rx=0 tx=0).stop 2024-03-07T06:51:27.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.797+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3989862055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cc415a3e0 msgr2=0x7f7cc4166370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.797+0000 7f7cc8c00640 1 --2- 172.21.15.133:0/3989862055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cc415a3e0 0x7f7cc4166370 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc007e60 tx=0x7f7cbc00e4b0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.797+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3989862055 shutdown_connections 2024-03-07T06:51:27.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.797+0000 7f7cc8c00640 1 --2- 172.21.15.133:0/3989862055 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7ca406b4d0 0x7f7ca406d990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.797+0000 7f7cc8c00640 1 --2- 172.21.15.133:0/3989862055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cc41668b0 0x7f7cc40aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.797+0000 7f7cc8c00640 1 --2- 172.21.15.133:0/3989862055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cc415a3e0 0x7f7cc4166370 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.797+0000 7f7cc8c00640 1 --2- 172.21.15.133:0/3989862055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cc40acd00 0x7f7cc414e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.797+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3989862055 >> 172.21.15.133:0/3989862055 conn(0x7f7cc40b3120 msgr2=0x7f7cc41598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.798+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3989862055 shutdown_connections 2024-03-07T06:51:27.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.798+0000 7f7cc8c00640 1 -- 172.21.15.133:0/3989862055 wait complete. 2024-03-07T06:51:27.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.799+0000 7f3b0a000640 1 -- 172.21.15.133:0/2434942846 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3af406b5d0 msgr2=0x7f3af406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.799+0000 7f3b0a000640 1 --2- 172.21.15.133:0/2434942846 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3af406b5d0 0x7f3af406da90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009ad0 tx=0x7f3b04008040 comp rx=0 tx=0).stop 2024-03-07T06:51:27.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.799+0000 7f3b0a000640 1 -- 172.21.15.133:0/2434942846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b14137dd0 msgr2=0x7f3b141f80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.799+0000 7f3b0a000640 1 --2- 172.21.15.133:0/2434942846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b14137dd0 0x7f3b141f80c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3b00034040 tx=0x7f3b000046c0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.800+0000 7fb473e00640 1 -- 172.21.15.133:0/4134163351 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb45406b700 msgr2=0x7fb45406dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.800+0000 7fb473e00640 1 --2- 172.21.15.133:0/4134163351 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb45406b700 0x7fb45406dbc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb464005880 tx=0x7fb464009040 comp rx=0 tx=0).stop 2024-03-07T06:51:27.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.800+0000 7f3b0a000640 1 -- 172.21.15.133:0/2434942846 shutdown_connections 2024-03-07T06:51:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.800+0000 7f3b0a000640 1 --2- 172.21.15.133:0/2434942846 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3af406b5d0 0x7f3af406da90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.800+0000 7f3b0a000640 1 --2- 172.21.15.133:0/2434942846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b1415b0a0 0x7f3b1415b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.800+0000 7f3b0a000640 1 --2- 172.21.15.133:0/2434942846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b14137dd0 0x7f3b141f80c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.800+0000 7f3b0a000640 1 --2- 172.21.15.133:0/2434942846 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b140b5dd0 0x7f3b140d39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.800+0000 7f3b0a000640 1 -- 172.21.15.133:0/2434942846 >> 172.21.15.133:0/2434942846 conn(0x7f3b140d0e70 msgr2=0x7f3b140d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.800+0000 7f3b0a000640 1 -- 172.21.15.133:0/2434942846 shutdown_connections 2024-03-07T06:51:27.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.800+0000 7f3b0a000640 1 -- 172.21.15.133:0/2434942846 wait complete. 2024-03-07T06:51:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.800+0000 7fb473e00640 1 -- 172.21.15.133:0/4134163351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb46c0ead00 msgr2=0x7fb46c04aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.800+0000 7fb473e00640 1 --2- 172.21.15.133:0/4134163351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb46c0ead00 0x7fb46c04aa10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb474067da0 tx=0x7fb4740738f0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.801+0000 7fb473e00640 1 -- 172.21.15.133:0/4134163351 shutdown_connections 2024-03-07T06:51:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.801+0000 7fb473e00640 1 --2- 172.21.15.133:0/4134163351 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb46c0ead00 0x7fb46c04aa10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.801+0000 7fb473e00640 1 --2- 172.21.15.133:0/4134163351 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb45406b700 0x7fb45406dbc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.801+0000 7fb473e00640 1 --2- 172.21.15.133:0/4134163351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb46c006810 0x7fb46c044af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.801+0000 7fb473e00640 1 --2- 172.21.15.133:0/4134163351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb46c005e90 0x7fb46c0e8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.801+0000 7fb473e00640 1 -- 172.21.15.133:0/4134163351 >> 172.21.15.133:0/4134163351 conn(0x7fb46c009fb0 msgr2=0x7fb46c049c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.801+0000 7fb473e00640 1 -- 172.21.15.133:0/4134163351 shutdown_connections 2024-03-07T06:51:27.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.801+0000 7fb473e00640 1 -- 172.21.15.133:0/4134163351 wait complete. 2024-03-07T06:51:27.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.815+0000 7fa7e3e00640 1 -- 172.21.15.133:0/1485931404 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa7e40b8a50 con 0x7fa7e41551f0 2024-03-07T06:51:27.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.816+0000 7fa7db400640 1 -- 172.21.15.133:0/1485931404 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa7d40591a0 con 0x7fa7e41551f0 2024-03-07T06:51:27.815 INFO:teuthology.orchestra.run.smithi133.stdout:55834574855 2024-03-07T06:51:27.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.819+0000 7fa7e3e00640 1 -- 172.21.15.133:0/1485931404 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa7c006b4d0 msgr2=0x7fa7c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.819+0000 7fa7e3e00640 1 --2- 172.21.15.133:0/1485931404 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa7c006b4d0 0x7fa7c006d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc036000 tx=0x7fa7dc03b040 comp rx=0 tx=0).stop 2024-03-07T06:51:27.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.819+0000 7fa7e3e00640 1 -- 172.21.15.133:0/1485931404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e41551f0 msgr2=0x7fa7e41555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.819+0000 7fa7e3e00640 1 --2- 172.21.15.133:0/1485931404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e41551f0 0x7fa7e41555f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa7d400e960 tx=0x7fa7d400ee20 comp rx=0 tx=0).stop 2024-03-07T06:51:27.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.820+0000 7f703f600640 1 -- 172.21.15.133:0/3339072282 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f70380ce080 con 0x7f7038067290 2024-03-07T06:51:27.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.820+0000 7fa7e3e00640 1 -- 172.21.15.133:0/1485931404 shutdown_connections 2024-03-07T06:51:27.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.820+0000 7fa7e3e00640 1 --2- 172.21.15.133:0/1485931404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa7e41551f0 0x7fa7e41555f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.820+0000 7fa7e3e00640 1 --2- 172.21.15.133:0/1485931404 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa7e4154e10 0x7fa7e414c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.820+0000 7fa7e3e00640 1 --2- 172.21.15.133:0/1485931404 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa7c006b4d0 0x7fa7c006d990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.820+0000 7fa7e3e00640 1 --2- 172.21.15.133:0/1485931404 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7e40d3810 0x7fa7e413e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.820+0000 7fa7e3e00640 1 -- 172.21.15.133:0/1485931404 >> 172.21.15.133:0/1485931404 conn(0x7fa7e4070400 msgr2=0x7fa7e4154070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.820+0000 7fa7e3e00640 1 -- 172.21.15.133:0/1485931404 shutdown_connections 2024-03-07T06:51:27.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.820+0000 7fa7e3e00640 1 -- 172.21.15.133:0/1485931404 wait complete. 2024-03-07T06:51:27.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.820+0000 7f7035600640 1 -- 172.21.15.133:0/3339072282 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f70200580d0 con 0x7f7038067290 2024-03-07T06:51:27.822 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2024-03-07T06:51:27.822 INFO:teuthology.orchestra.run.smithi133.stdout:55834574854 2024-03-07T06:51:27.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2024-03-07T06:51:27.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.824+0000 7f703f600640 1 -- 172.21.15.133:0/3339072282 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f701406b520 msgr2=0x7f701406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.824+0000 7f703f600640 1 --2- 172.21.15.133:0/3339072282 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f701406b520 0x7f701406d9e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7028009870 tx=0x7f7028008030 comp rx=0 tx=0).stop 2024-03-07T06:51:27.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.824+0000 7f703f600640 1 -- 172.21.15.133:0/3339072282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7038067290 msgr2=0x7f70381493e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:27.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.824+0000 7f703f600640 1 --2- 172.21.15.133:0/3339072282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7038067290 0x7f70381493e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f702000c390 tx=0x7f702000c850 comp rx=0 tx=0).stop 2024-03-07T06:51:27.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.824+0000 7f703f600640 1 -- 172.21.15.133:0/3339072282 shutdown_connections 2024-03-07T06:51:27.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.824+0000 7f703f600640 1 --2- 172.21.15.133:0/3339072282 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f703806e1c0 0x7f7038138900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.824+0000 7f703f600640 1 --2- 172.21.15.133:0/3339072282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7038067290 0x7f70381493e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.824+0000 7f703f600640 1 --2- 172.21.15.133:0/3339072282 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f701406b520 0x7f701406d9e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.824+0000 7f703f600640 1 --2- 172.21.15.133:0/3339072282 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7038066eb0 0x7f70381342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:27.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.824+0000 7f703f600640 1 -- 172.21.15.133:0/3339072282 >> 172.21.15.133:0/3339072282 conn(0x7f70380cd160 msgr2=0x7f7038147960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:27.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.824+0000 7f703f600640 1 -- 172.21.15.133:0/3339072282 shutdown_connections 2024-03-07T06:51:27.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:27.825+0000 7f703f600640 1 -- 172.21.15.133:0/3339072282 wait complete. 2024-03-07T06:51:27.827 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-03-07T06:51:27.828 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2024-03-07T06:51:27.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-03-07T06:51:27.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2024-03-07T06:51:27.842 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2024-03-07T06:51:28.781 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-07T06:51:28.823 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-07T06:51:28.823 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-07T06:51:28.828 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-07T06:51:28.828 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-07T06:51:28.830 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-07T06:51:28.838 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-07T06:51:28.842 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-07T06:51:29.031 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:51:29.047 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:51:29.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.088+0000 7f8295200640 1 Processor -- start 2024-03-07T06:51:29.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.088+0000 7f8295200640 1 -- start start 2024-03-07T06:51:29.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.090+0000 7f8295200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f82900b52c0 0x7f8290162c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.090+0000 7f8295200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8290163190 0x7f8290071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.090+0000 7f8295200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8290163740 con 0x7f8290071950 2024-03-07T06:51:29.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.090+0000 7f8295200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8290163e10 con 0x7f82900b52c0 2024-03-07T06:51:29.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.090+0000 7f8295200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8290071d80 con 0x7f8290163190 2024-03-07T06:51:29.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.091+0000 7f828d400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8290163190 0x7f8290071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.091+0000 7f828d400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8290163190 0x7f8290071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:42148/0 (socket says 172.21.15.133:42148) 2024-03-07T06:51:29.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.091+0000 7f828d400640 1 -- 172.21.15.133:0/1783274623 learned_addr learned my addr 172.21.15.133:0/1783274623 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:29.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.092+0000 7f828d400640 1 -- 172.21.15.133:0/1783274623 >> v2:172.21.15.133:3300/0 conn(0x7f82900b52c0 msgr2=0x7f8290162c50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-07T06:51:29.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.092+0000 7f828d400640 1 --2- 172.21.15.133:0/1783274623 >> v2:172.21.15.133:3300/0 conn(0x7f82900b52c0 0x7f8290162c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.092+0000 7f828d400640 1 -- 172.21.15.133:0/1783274623 >> v1:172.21.15.133:6789/0 conn(0x7f8290071950 legacy=0x7f829006e070 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-07T06:51:29.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.092+0000 7f828d400640 1 -- 172.21.15.133:0/1783274623 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8290163a20 con 0x7f8290163190 2024-03-07T06:51:29.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.095+0000 7f828d400640 1 --2- 172.21.15.133:0/1783274623 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8290163190 0x7f8290071410 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f827c009870 tx=0x7f827c02b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=439c206b6fb96be0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.102+0000 7f828ca00640 1 -- 172.21.15.133:0/1783274623 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c035020 con 0x7f8290163190 2024-03-07T06:51:29.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.102+0000 7f828ca00640 1 -- 172.21.15.133:0/1783274623 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827c031870 con 0x7f8290163190 2024-03-07T06:51:29.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.102+0000 7f828ca00640 1 -- 172.21.15.133:0/1783274623 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827c031c40 con 0x7f8290163190 2024-03-07T06:51:29.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.103+0000 7f8295200640 1 -- 172.21.15.133:0/1783274623 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8290163190 msgr2=0x7f8290071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.103+0000 7f8295200640 1 --2- 172.21.15.133:0/1783274623 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8290163190 0x7f8290071410 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f827c009870 tx=0x7f827c02b910 comp rx=0 tx=0).stop 2024-03-07T06:51:29.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.106+0000 7f190de00640 1 Processor -- start 2024-03-07T06:51:29.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.106+0000 7f190de00640 1 -- start start 2024-03-07T06:51:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.108+0000 7f190de00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f19000e3160 0x7f19000e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.108+0000 7f190de00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f190003c0a0 0x7f1900058050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.108+0000 7f190de00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1900058590 con 0x7f1900053cb0 2024-03-07T06:51:29.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.108+0000 7f190de00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1900007b70 con 0x7f19000e3160 2024-03-07T06:51:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.108+0000 7f190de00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1900058810 con 0x7f190003c0a0 2024-03-07T06:51:29.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.108+0000 7f8295200640 1 -- 172.21.15.133:0/1783274623 shutdown_connections 2024-03-07T06:51:29.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.108+0000 7f8295200640 1 --2- 172.21.15.133:0/1783274623 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8290163190 0x7f8290071410 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.108+0000 7f8295200640 1 --2- 172.21.15.133:0/1783274623 >> v2:172.21.15.133:3300/0 conn(0x7f82900b52c0 0x7f8290162c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.108+0000 7f8295200640 1 -- 172.21.15.133:0/1783274623 >> 172.21.15.133:0/1783274623 conn(0x7f82900d0e70 msgr2=0x7f82900d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.108+0000 7f8295200640 1 -- 172.21.15.133:0/1783274623 shutdown_connections 2024-03-07T06:51:29.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.108+0000 7f8295200640 1 -- 172.21.15.133:0/1783274623 wait complete. 2024-03-07T06:51:29.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.111+0000 7f1907e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f190003c0a0 0x7f1900058050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.111+0000 7f1907e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f190003c0a0 0x7f1900058050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:42158/0 (socket says 172.21.15.133:42158) 2024-03-07T06:51:29.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.111+0000 7f1907e00640 1 -- 172.21.15.133:0/4169454557 learned_addr learned my addr 172.21.15.133:0/4169454557 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:29.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.111+0000 7f190ca00640 1 --2- 172.21.15.133:0/4169454557 >> v2:172.21.15.133:3300/0 conn(0x7f19000e3160 0x7f19000e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.111+0000 7f1907e00640 1 -- 172.21.15.133:0/4169454557 >> v2:172.21.15.133:3300/0 conn(0x7f19000e3160 msgr2=0x7f19000e7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:29.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.111+0000 7f1907e00640 1 --2- 172.21.15.133:0/4169454557 >> v2:172.21.15.133:3300/0 conn(0x7f19000e3160 0x7f19000e7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.111+0000 7f1907400640 1 -- 172.21.15.133:0/4169454557 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474775470 0 0) 0x7f18fc001110 con 0x7f1900053cb0 2024-03-07T06:51:29.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.112+0000 7f1907e00640 1 -- 172.21.15.133:0/4169454557 >> v1:172.21.15.133:6789/0 conn(0x7f1900053cb0 legacy=0x7f1900054120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.112+0000 7f1907e00640 1 -- 172.21.15.133:0/4169454557 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1900058af0 con 0x7f190003c0a0 2024-03-07T06:51:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.112+0000 7f190ca00640 1 --2- 172.21.15.133:0/4169454557 >> v2:172.21.15.133:3300/0 conn(0x7f19000e3160 0x7f19000e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T06:51:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.113+0000 7f1907e00640 1 --2- 172.21.15.133:0/4169454557 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f190003c0a0 0x7f1900058050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1908069c20 tx=0x7f19080a6c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1ad253c961e1024 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.114+0000 7f1907400640 1 -- 172.21.15.133:0/4169454557 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19080b4070 con 0x7f190003c0a0 2024-03-07T06:51:29.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.114+0000 7f1907400640 1 -- 172.21.15.133:0/4169454557 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19080a78a0 con 0x7f190003c0a0 2024-03-07T06:51:29.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.114+0000 7f1907400640 1 -- 172.21.15.133:0/4169454557 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19080a7cd0 con 0x7f190003c0a0 2024-03-07T06:51:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.115+0000 7f190de00640 1 -- 172.21.15.133:0/4169454557 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f190003c0a0 msgr2=0x7f1900058050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.115+0000 7f190de00640 1 --2- 172.21.15.133:0/4169454557 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f190003c0a0 0x7f1900058050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1908069c20 tx=0x7f19080a6c40 comp rx=0 tx=0).stop 2024-03-07T06:51:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.116+0000 7f190de00640 1 -- 172.21.15.133:0/4169454557 shutdown_connections 2024-03-07T06:51:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.116+0000 7f190de00640 1 --2- 172.21.15.133:0/4169454557 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f190003c0a0 0x7f1900058050 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.116+0000 7f8295200640 1 Processor -- start 2024-03-07T06:51:29.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.117+0000 7fb5a4a00640 1 Processor -- start 2024-03-07T06:51:29.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.116+0000 7f190de00640 1 --2- 172.21.15.133:0/4169454557 >> v2:172.21.15.133:3300/0 conn(0x7f19000e3160 0x7f19000e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.118+0000 7f190de00640 1 -- 172.21.15.133:0/4169454557 >> 172.21.15.133:0/4169454557 conn(0x7f1900058ee0 msgr2=0x7f19000592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.118+0000 7f190de00640 1 -- 172.21.15.133:0/4169454557 shutdown_connections 2024-03-07T06:51:29.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.118+0000 7f190de00640 1 -- 172.21.15.133:0/4169454557 wait complete. 2024-03-07T06:51:29.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.116+0000 7f8295200640 1 -- start start 2024-03-07T06:51:29.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.118+0000 7f8295200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8290071950 0x7f829015fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.118+0000 7f8295200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82900b52c0 0x7f829014fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.118+0000 7f8295200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8290163190 0x7f8290150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.118+0000 7f8295200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82900b7010 con 0x7f82900b52c0 2024-03-07T06:51:29.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.118+0000 7f8295200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8290164340 con 0x7f8290163190 2024-03-07T06:51:29.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.118+0000 7f8295200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82900b56a0 con 0x7f8290071950 2024-03-07T06:51:29.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.118+0000 7f828de00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8290071950 0x7f829015fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.118+0000 7f828de00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8290071950 0x7f829015fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:42162/0 (socket says 172.21.15.133:42162) 2024-03-07T06:51:29.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.117+0000 7fb5a4a00640 1 -- start start 2024-03-07T06:51:29.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.119+0000 7fb5a4a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb5a0141370 0x7fb5a00b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.119+0000 7fb5a4a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb5a00b9180 0x7fb5a015e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.119+0000 7fb5a4a00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a015f160 con 0x7fb5a015ecf0 2024-03-07T06:51:29.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.119+0000 7fb5a4a00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a00b95f0 con 0x7fb5a0141370 2024-03-07T06:51:29.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.119+0000 7fb5a4a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a00b6440 con 0x7fb5a00b9180 2024-03-07T06:51:29.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.119+0000 7fb59de00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb5a0141370 0x7fb5a00b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.119+0000 7fb59de00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb5a0141370 0x7fb5a00b8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38708/0 (socket says 172.21.15.133:38708) 2024-03-07T06:51:29.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.119+0000 7fb59de00640 1 -- 172.21.15.133:0/2615905317 learned_addr learned my addr 172.21.15.133:0/2615905317 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:29.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.118+0000 7f828de00640 1 -- 172.21.15.133:0/3308218926 learned_addr learned my addr 172.21.15.133:0/3308218926 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:29.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.119+0000 7f828de00640 1 -- 172.21.15.133:0/3308218926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8290163190 msgr2=0x7f8290150270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-07T06:51:29.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.119+0000 7f828de00640 1 --2- 172.21.15.133:0/3308218926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8290163190 0x7f8290150270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.119+0000 7f828de00640 1 -- 172.21.15.133:0/3308218926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82900b52c0 msgr2=0x7f829014fd30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-07T06:51:29.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.119+0000 7f828de00640 1 --2- 172.21.15.133:0/3308218926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82900b52c0 0x7f829014fd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.119+0000 7f828de00640 1 -- 172.21.15.133:0/3308218926 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829015f6f0 con 0x7f8290071950 2024-03-07T06:51:29.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.121+0000 7f190de00640 1 Processor -- start 2024-03-07T06:51:29.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.121+0000 7fb59de00640 1 -- 172.21.15.133:0/2615905317 >> v2:172.21.15.161:3300/0 conn(0x7fb5a00b9180 msgr2=0x7fb5a015e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:29.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.121+0000 7fb59de00640 1 --2- 172.21.15.133:0/2615905317 >> v2:172.21.15.161:3300/0 conn(0x7fb5a00b9180 0x7fb5a015e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.121+0000 7fb59de00640 1 -- 172.21.15.133:0/2615905317 >> v1:172.21.15.133:6789/0 conn(0x7fb5a015ecf0 legacy=0x7fb5a00b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.121+0000 7fb59de00640 1 -- 172.21.15.133:0/2615905317 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a0141980 con 0x7fb5a0141370 2024-03-07T06:51:29.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.122+0000 7f828de00640 1 --2- 172.21.15.133:0/3308218926 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8290071950 0x7f829015fab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f828000b480 tx=0x7f828000b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.122+0000 7f8286000640 1 -- 172.21.15.133:0/3308218926 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280014070 con 0x7f8290071950 2024-03-07T06:51:29.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.122+0000 7f8286000640 1 -- 172.21.15.133:0/3308218926 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8280010890 con 0x7f8290071950 2024-03-07T06:51:29.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.122+0000 7f8286000640 1 -- 172.21.15.133:0/3308218926 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8280010c80 con 0x7f8290071950 2024-03-07T06:51:29.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.122+0000 7f8295200640 1 -- 172.21.15.133:0/3308218926 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829014eb40 con 0x7f8290071950 2024-03-07T06:51:29.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.122+0000 7f8295200640 1 -- 172.21.15.133:0/3308218926 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829014f100 con 0x7f8290071950 2024-03-07T06:51:29.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.121+0000 7f190de00640 1 -- start start 2024-03-07T06:51:29.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.123+0000 7fb59de00640 1 --2- 172.21.15.133:0/2615905317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5a0141370 0x7fb5a00b8c40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00d42d0 tx=0x7fb58c02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ae3cd4bb7f0828 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.124+0000 7fb59ca00640 1 -- 172.21.15.133:0/2615905317 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c035020 con 0x7fb5a0141370 2024-03-07T06:51:29.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.124+0000 7f190de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f190003c0a0 0x7f19000d6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.124+0000 7fb59ca00640 1 -- 172.21.15.133:0/2615905317 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58c031880 con 0x7fb5a0141370 2024-03-07T06:51:29.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.124+0000 7fb59ca00640 1 -- 172.21.15.133:0/2615905317 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58c031c50 con 0x7fb5a0141370 2024-03-07T06:51:29.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.126+0000 7fb5a4a00640 1 -- 172.21.15.133:0/2615905317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5a0141370 msgr2=0x7fb5a00b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.126+0000 7fb5a4a00640 1 --2- 172.21.15.133:0/2615905317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5a0141370 0x7fb5a00b8c40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00d42d0 tx=0x7fb58c02b920 comp rx=0 tx=0).stop 2024-03-07T06:51:29.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.126+0000 7f190de00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1900053cb0 0x7f190004e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.126+0000 7fb5a4a00640 1 -- 172.21.15.133:0/2615905317 shutdown_connections 2024-03-07T06:51:29.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.126+0000 7f190de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19000e3160 0x7f19000e3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7f190de00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19000e3540 con 0x7f190003c0a0 2024-03-07T06:51:29.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7f190de00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1900006640 con 0x7f19000e3160 2024-03-07T06:51:29.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7f190de00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1900006b60 con 0x7f1900053cb0 2024-03-07T06:51:29.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7fb5a4a00640 1 --2- 172.21.15.133:0/2615905317 >> v2:172.21.15.161:3300/0 conn(0x7fb5a00b9180 0x7fb5a015e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7fb5a4a00640 1 --2- 172.21.15.133:0/2615905317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5a0141370 0x7fb5a00b8c40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7fb5a4a00640 1 -- 172.21.15.133:0/2615905317 >> 172.21.15.133:0/2615905317 conn(0x7fb5a0072fc0 msgr2=0x7fb5a00b7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7f1907e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1900053cb0 0x7f190004e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f1907e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1900053cb0 0x7f190004e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:42176/0 (socket says 172.21.15.133:42176) 2024-03-07T06:51:29.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f1907e00640 1 -- 172.21.15.133:0/1353553104 learned_addr learned my addr 172.21.15.133:0/1353553104 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:29.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.124+0000 7f231da00640 1 Processor -- start 2024-03-07T06:51:29.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.124+0000 7f231da00640 1 -- start start 2024-03-07T06:51:29.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.126+0000 7f231da00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2318143d10 0x7f23180cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.126+0000 7f231da00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f23180b1110 0x7f2318072400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7f2316000640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f23180ce420 0x7f23180b0a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.133:60216/0 (socket says 172.21.15.133:60216) 2024-03-07T06:51:29.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7f2316a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2318143d10 0x7f23180cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7f2316a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2318143d10 0x7f23180cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38736/0 (socket says 172.21.15.133:38736) 2024-03-07T06:51:29.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7f2316000640 1 -- 172.21.15.133:0/3841279462 learned_addr learned my addr 172.21.15.133:0/3841279462 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:29.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7f2317400640 1 --2- 172.21.15.133:0/3841279462 >> v2:172.21.15.161:3300/0 conn(0x7f23180b1110 0x7f2318072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7f231da00640 1 -- 172.21.15.133:0/3841279462 --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23180afe20 con 0x7f23180ce420 2024-03-07T06:51:29.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7f231da00640 1 -- 172.21.15.133:0/3841279462 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23180ce880 con 0x7f2318143d10 2024-03-07T06:51:29.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.127+0000 7f231da00640 1 -- 172.21.15.133:0/3841279462 --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2318071920 con 0x7f23180b1110 2024-03-07T06:51:29.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 -- 172.21.15.133:0/3841279462 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025576061 0 0) 0x7f23180afe20 con 0x7f23180ce420 2024-03-07T06:51:29.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 -- 172.21.15.133:0/3841279462 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22fc003520 con 0x7f23180ce420 2024-03-07T06:51:29.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f190ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f190003c0a0 0x7f19000d6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 -- 172.21.15.133:0/3841279462 <== mon.1 v1:172.21.15.161:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2997684025 0 0) 0x7f22fc003520 con 0x7f23180ce420 2024-03-07T06:51:29.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 -- 172.21.15.133:0/3841279462 >> v2:172.21.15.161:3300/0 conn(0x7f23180b1110 msgr2=0x7f2318072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:29.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 --2- 172.21.15.133:0/3841279462 >> v2:172.21.15.161:3300/0 conn(0x7f23180b1110 0x7f2318072400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 -- 172.21.15.133:0/3841279462 >> v2:172.21.15.133:3300/0 conn(0x7f2318143d10 msgr2=0x7f23180cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:29.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 --2- 172.21.15.133:0/3841279462 >> v2:172.21.15.133:3300/0 conn(0x7f2318143d10 0x7f23180cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 -- 172.21.15.133:0/3841279462 --> v1:172.21.15.161:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23180716b0 con 0x7f23180ce420 2024-03-07T06:51:29.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 -- 172.21.15.133:0/3841279462 <== mon.1 v1:172.21.15.161:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f23100019f0 con 0x7f23180ce420 2024-03-07T06:51:29.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 -- 172.21.15.133:0/3841279462 >> v1:172.21.15.161:6789/0 conn(0x7f23180ce420 legacy=0x7f23180b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f190ca00640 1 -- 172.21.15.133:0/1353553104 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1900053cb0 msgr2=0x7f190004e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f190ca00640 1 --2- 172.21.15.133:0/1353553104 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1900053cb0 0x7f190004e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f190ca00640 1 -- 172.21.15.133:0/1353553104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19000e3160 msgr2=0x7f19000e3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f8286000640 1 -- 172.21.15.133:0/3308218926 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8280017070 con 0x7f8290071950 2024-03-07T06:51:29.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.129+0000 7f190d400640 1 --2- 172.21.15.133:0/1353553104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19000e3160 0x7f19000e3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.129+0000 7f190ca00640 1 --2- 172.21.15.133:0/1353553104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19000e3160 0x7f19000e3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 --2- 172.21.15.133:0/3841279462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22fc0037a0 0x7f22fc003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 --2- 172.21.15.133:0/3841279462 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22fc004100 0x7f22fc00b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 --2- 172.21.15.133:0/3841279462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22fc00b550 0x7f22fc00d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 -- 172.21.15.133:0/3841279462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2318071920 con 0x7f22fc00b550 2024-03-07T06:51:29.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 -- 172.21.15.133:0/3841279462 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc004de0 con 0x7f22fc0037a0 2024-03-07T06:51:29.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.128+0000 7f2315600640 1 -- 172.21.15.133:0/3841279462 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22fc004fe0 con 0x7f22fc004100 2024-03-07T06:51:29.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.129+0000 7f2316a00640 1 --2- 172.21.15.133:0/3841279462 >> v2:172.21.15.133:3300/0 conn(0x7f2318143d10 0x7f23180cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:51:29.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.129+0000 7f2316000640 1 --2- 172.21.15.133:0/3841279462 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22fc004100 0x7f22fc00b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.129+0000 7f190ca00640 1 -- 172.21.15.133:0/1353553104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190004d4c0 con 0x7f190003c0a0 2024-03-07T06:51:29.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.130+0000 7f2317400640 1 --2- 172.21.15.133:0/3841279462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22fc00b550 0x7f22fc00d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.130+0000 7f8286000640 1 --2- 172.21.15.133:0/3308218926 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f827006b5d0 0x7f827006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.130+0000 7f8286000640 1 -- 172.21.15.133:0/3308218926 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8280037060 con 0x7f8290071950 2024-03-07T06:51:29.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.130+0000 7f8295200640 1 -- 172.21.15.133:0/3308218926 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8290062c90 con 0x7f8290071950 2024-03-07T06:51:29.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.130+0000 7fb5a4a00640 1 -- 172.21.15.133:0/2615905317 shutdown_connections 2024-03-07T06:51:29.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.130+0000 7fb5a4a00640 1 -- 172.21.15.133:0/2615905317 wait complete. 2024-03-07T06:51:29.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.130+0000 7f190d400640 1 --2- 172.21.15.133:0/1353553104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19000e3160 0x7f19000e3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:29.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.130+0000 7f828d400640 1 --2- 172.21.15.133:0/3308218926 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f827006b5d0 0x7f827006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.130+0000 7f2316000640 1 -- 172.21.15.133:0/3841279462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22fc0037a0 msgr2=0x7f22fc003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:29.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.130+0000 7f2316000640 1 --2- 172.21.15.133:0/3841279462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22fc0037a0 0x7f22fc003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.130+0000 7f2316000640 1 -- 172.21.15.133:0/3841279462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22fc00b550 msgr2=0x7f22fc00d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:29.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.130+0000 7f2316000640 1 --2- 172.21.15.133:0/3841279462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22fc00b550 0x7f22fc00d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.130+0000 7f2316000640 1 -- 172.21.15.133:0/3841279462 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23180716b0 con 0x7f22fc004100 2024-03-07T06:51:29.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.130+0000 7f2317400640 1 -- 172.21.15.133:0/3841279462 reap_dead start 2024-03-07T06:51:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.131+0000 7f190ca00640 1 --2- 172.21.15.133:0/1353553104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f190003c0a0 0x7f19000d6830 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f18f400e870 tx=0x7f18f400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.131+0000 7f65a3200640 1 Processor -- start 2024-03-07T06:51:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.132+0000 7f1904c00640 1 -- 172.21.15.133:0/1353553104 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f4017070 con 0x7f190003c0a0 2024-03-07T06:51:29.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.132+0000 7f828d400640 1 --2- 172.21.15.133:0/3308218926 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f827006b5d0 0x7f827006da90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f827c036000 tx=0x7f827c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.132+0000 7f1904c00640 1 -- 172.21.15.133:0/1353553104 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f400c940 con 0x7f190003c0a0 2024-03-07T06:51:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.132+0000 7f1904c00640 1 -- 172.21.15.133:0/1353553104 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f400cd10 con 0x7f190003c0a0 2024-03-07T06:51:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.132+0000 7f190de00640 1 -- 172.21.15.133:0/1353553104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190004f300 con 0x7f190003c0a0 2024-03-07T06:51:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.133+0000 7f190de00640 1 -- 172.21.15.133:0/1353553104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190004ed10 con 0x7f190003c0a0 2024-03-07T06:51:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.132+0000 7f65a3200640 1 -- start start 2024-03-07T06:51:29.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.134+0000 7f65a3200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f659c165ed0 0x7f659c06ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.134+0000 7f2316000640 1 --2- 172.21.15.133:0/3841279462 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22fc004100 0x7f22fc00b010 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f231000a900 tx=0x7f22fc004820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c054b542714080d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.134+0000 7fb5a4a00640 1 Processor -- start 2024-03-07T06:51:29.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.134+0000 7f65a3200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f659c0b3c10 0x7f659c071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.134+0000 7f65a3200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f659c071fe0 con 0x7f659c071b50 2024-03-07T06:51:29.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.134+0000 7f65a3200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659c0734a0 con 0x7f659c165ed0 2024-03-07T06:51:29.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.134+0000 7f65a3200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659c0ce540 con 0x7f659c0b3c10 2024-03-07T06:51:29.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.134+0000 7fb5a4a00640 1 -- start start 2024-03-07T06:51:29.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.134+0000 7f659b400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f659c0b3c10 0x7f659c071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.134+0000 7fa23a800640 1 Processor -- start 2024-03-07T06:51:29.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.134+0000 7f659b400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f659c0b3c10 0x7f659c071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:42192/0 (socket says 172.21.15.133:42192) 2024-03-07T06:51:29.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.134+0000 7f659be00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f659c165ed0 0x7f659c06ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.134+0000 7f659b400640 1 -- 172.21.15.133:0/952495844 learned_addr learned my addr 172.21.15.133:0/952495844 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:29.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.135+0000 7fa23a800640 1 -- start start 2024-03-07T06:51:29.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.135+0000 7f659be00640 1 -- 172.21.15.133:0/952495844 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f659c0b3c10 msgr2=0x7f659c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:29.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.135+0000 7f659be00640 1 --2- 172.21.15.133:0/952495844 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f659c0b3c10 0x7f659c071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.135+0000 7f659be00640 1 -- 172.21.15.133:0/952495844 >> v1:172.21.15.133:6789/0 conn(0x7f659c071b50 legacy=0x7f659c06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.135+0000 7f659be00640 1 -- 172.21.15.133:0/952495844 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c06f200 con 0x7f659c165ed0 2024-03-07T06:51:29.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.136+0000 7fb5a4a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a00b9180 0x7fb5a015a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.136+0000 7fb5a4a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5a0141370 0x7fb5a013fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.136+0000 7fb5a4a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5a015ecf0 0x7fb5a01f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.136+0000 7fb5a4a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a0141c40 con 0x7fb5a0141370 2024-03-07T06:51:29.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.136+0000 7fb5a4a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a015f6d0 con 0x7fb5a00b9180 2024-03-07T06:51:29.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.136+0000 7fb5a4a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a0143450 con 0x7fb5a015ecf0 2024-03-07T06:51:29.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.137+0000 7fb59de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a00b9180 0x7fb5a015a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.137+0000 7fb59e800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5a015ecf0 0x7fb5a01f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.137+0000 7fb59e800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5a015ecf0 0x7fb5a01f3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:42198/0 (socket says 172.21.15.133:42198) 2024-03-07T06:51:29.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.137+0000 7fb59e800640 1 -- 172.21.15.133:0/3471771457 learned_addr learned my addr 172.21.15.133:0/3471771457 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:29.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.137+0000 7fa23a800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa234162260 0x7fa23415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.137+0000 7f190de00640 1 -- 172.21.15.133:0/1353553104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1900005390 con 0x7f190003c0a0 2024-03-07T06:51:29.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.137+0000 7f659be00640 1 --2- 172.21.15.133:0/952495844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f659c165ed0 0x7f659c06ecc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65940097a0 tx=0x7f659402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efe638b2a793834f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.137+0000 7fa23a800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa2340d39e0 0x7fa2341414e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.137+0000 7fa23a800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa234141210 con 0x7fa234162640 2024-03-07T06:51:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.137+0000 7fa23a800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2340b4fc0 con 0x7fa2340d39e0 2024-03-07T06:51:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.137+0000 7fa23a800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2340b43f0 con 0x7fa234162260 2024-03-07T06:51:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.137+0000 7fb59d400640 1 --2- 172.21.15.133:0/3471771457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5a0141370 0x7fb5a013fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.138+0000 7fb59e800640 1 -- 172.21.15.133:0/3471771457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a00b9180 msgr2=0x7fb5a015a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.138+0000 7fb59e800640 1 --2- 172.21.15.133:0/3471771457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a00b9180 0x7fb5a015a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.138+0000 7fb59e800640 1 -- 172.21.15.133:0/3471771457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5a0141370 msgr2=0x7fb5a013fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.138+0000 7f659aa00640 1 -- 172.21.15.133:0/952495844 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594035020 con 0x7f659c165ed0 2024-03-07T06:51:29.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.138+0000 7f659aa00640 1 -- 172.21.15.133:0/952495844 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6594031810 con 0x7f659c165ed0 2024-03-07T06:51:29.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.138+0000 7f659aa00640 1 -- 172.21.15.133:0/952495844 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594031be0 con 0x7f659c165ed0 2024-03-07T06:51:29.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.138+0000 7f1904c00640 1 -- 172.21.15.133:0/1353553104 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f18f401a070 con 0x7f190003c0a0 2024-03-07T06:51:29.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.138+0000 7fb59e800640 1 --2- 172.21.15.133:0/3471771457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5a0141370 0x7fb5a013fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.138+0000 7fb59e800640 1 -- 172.21.15.133:0/3471771457 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a015a970 con 0x7fb5a015ecf0 2024-03-07T06:51:29.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.138+0000 7fb59d400640 1 --2- 172.21.15.133:0/3471771457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5a0141370 0x7fb5a013fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:29.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.138+0000 7fb59e800640 1 --2- 172.21.15.133:0/3471771457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5a015ecf0 0x7fb5a01f3f70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb59800c730 tx=0x7fb59800cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7f65a3200640 1 -- 172.21.15.133:0/952495844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f659c165ed0 msgr2=0x7f659c06ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7f65a3200640 1 --2- 172.21.15.133:0/952495844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f659c165ed0 0x7f659c06ecc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65940097a0 tx=0x7f659402b8b0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7f65a3200640 1 -- 172.21.15.133:0/952495844 shutdown_connections 2024-03-07T06:51:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7f65a3200640 1 --2- 172.21.15.133:0/952495844 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f659c0b3c10 0x7f659c071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7f65a3200640 1 --2- 172.21.15.133:0/952495844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f659c165ed0 0x7f659c06ecc0 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65940097a0 tx=0x7f659402b8b0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7f65a3200640 1 -- 172.21.15.133:0/952495844 >> 172.21.15.133:0/952495844 conn(0x7f659c0b3120 msgr2=0x7f659c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7f65a3200640 1 -- 172.21.15.133:0/952495844 shutdown_connections 2024-03-07T06:51:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7f65a3200640 1 -- 172.21.15.133:0/952495844 wait complete. 2024-03-07T06:51:29.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.138+0000 7fa239e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa2340d39e0 0x7fa2341414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.138+0000 7fa239400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa234162260 0x7fa23415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7fa239400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa234162260 0x7fa23415ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:59056/0 (socket says 172.21.15.133:59056) 2024-03-07T06:51:29.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7fa239400640 1 -- 172.21.15.133:0/493949830 learned_addr learned my addr 172.21.15.133:0/493949830 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:29.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7fa233e00640 1 -- 172.21.15.133:0/493949830 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1734356023 0 0) 0x7fa234141210 con 0x7fa234162640 2024-03-07T06:51:29.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7fa233e00640 1 -- 172.21.15.133:0/493949830 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa21c003520 con 0x7fa234162640 2024-03-07T06:51:29.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7fa239e00640 1 -- 172.21.15.133:0/493949830 >> v2:172.21.15.161:3300/0 conn(0x7fa234162260 msgr2=0x7fa23415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:29.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7fa239e00640 1 --2- 172.21.15.133:0/493949830 >> v2:172.21.15.161:3300/0 conn(0x7fa234162260 0x7fa23415ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7fa239e00640 1 -- 172.21.15.133:0/493949830 >> v1:172.21.15.161:6789/0 conn(0x7fa234162640 legacy=0x7fa2340d32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.139+0000 7fa239e00640 1 -- 172.21.15.133:0/493949830 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340b4690 con 0x7fa2340d39e0 2024-03-07T06:51:29.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.141+0000 7fa239e00640 1 --2- 172.21.15.133:0/493949830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340d39e0 0x7fa2341414e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa2340b3b80 tx=0x7fa22002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32319845e3428d05 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.142+0000 7f65a3200640 1 Processor -- start 2024-03-07T06:51:29.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.142+0000 7f1904c00640 1 --2- 172.21.15.133:0/1353553104 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f18e806b520 0x7f18e806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.142+0000 7f1904c00640 1 -- 172.21.15.133:0/1353553104 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18f4089ff0 con 0x7f190003c0a0 2024-03-07T06:51:29.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.142+0000 7f65a3200640 1 -- start start 2024-03-07T06:51:29.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.142+0000 7f1907e00640 1 --2- 172.21.15.133:0/1353553104 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f18e806b520 0x7f18e806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.143+0000 7f1907e00640 1 --2- 172.21.15.133:0/1353553104 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f18e806b520 0x7f18e806d9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1908050170 tx=0x7f19080b7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.144+0000 7f65a3200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f659c071b50 0x7f659c09ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.144+0000 7f65a3200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f659c0b3c10 0x7f659c157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.144+0000 7f65a3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f659c09c070 0x7f659c15e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.144+0000 7f65a3200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c072270 con 0x7f659c09c070 2024-03-07T06:51:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.144+0000 7f65a3200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c0ce260 con 0x7f659c0b3c10 2024-03-07T06:51:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.144+0000 7f659be00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f659c071b50 0x7f659c09ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.144+0000 7f65a3200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659c072550 con 0x7f659c071b50 2024-03-07T06:51:29.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.145+0000 7f659be00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f659c071b50 0x7f659c09ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:42208/0 (socket says 172.21.15.133:42208) 2024-03-07T06:51:29.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.145+0000 7f659be00640 1 -- 172.21.15.133:0/3926102934 learned_addr learned my addr 172.21.15.133:0/3926102934 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:29.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.145+0000 7f659b400640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f659c0b3c10 0x7f659c157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.145+0000 7f65a1200640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f659c09c070 0x7f659c15e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.145+0000 7f659be00640 1 -- 172.21.15.133:0/3926102934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f659c0b3c10 msgr2=0x7f659c157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.145+0000 7f659be00640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f659c0b3c10 0x7f659c157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.145+0000 7f659be00640 1 -- 172.21.15.133:0/3926102934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f659c09c070 msgr2=0x7f659c15e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.145+0000 7f659be00640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f659c09c070 0x7f659c15e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.145+0000 7f659be00640 1 -- 172.21.15.133:0/3926102934 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c158060 con 0x7f659c071b50 2024-03-07T06:51:29.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.145+0000 7f65a1200640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f659c09c070 0x7f659c15e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:29.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.145+0000 7f659be00640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f659c071b50 0x7f659c09ccf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f659402be40 tx=0x7f65940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.147+0000 7fa233e00640 1 -- 172.21.15.133:0/493949830 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa220034070 con 0x7fa2340d39e0 2024-03-07T06:51:29.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.147+0000 7fa233e00640 1 -- 172.21.15.133:0/493949830 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa220030840 con 0x7fa2340d39e0 2024-03-07T06:51:29.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.147+0000 7fa233e00640 1 -- 172.21.15.133:0/493949830 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa220030c30 con 0x7fa2340d39e0 2024-03-07T06:51:29.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.147+0000 7f1904c00640 1 -- 172.21.15.133:0/1353553104 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f18f4058790 con 0x7f190003c0a0 2024-03-07T06:51:29.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.148+0000 7fa23a800640 1 -- 172.21.15.133:0/493949830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340d39e0 msgr2=0x7fa2341414e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.148+0000 7fa23a800640 1 --2- 172.21.15.133:0/493949830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340d39e0 0x7fa2341414e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa2340b3b80 tx=0x7fa22002b980 comp rx=0 tx=0).stop 2024-03-07T06:51:29.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.148+0000 7fa23a800640 1 -- 172.21.15.133:0/493949830 shutdown_connections 2024-03-07T06:51:29.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.148+0000 7fa23a800640 1 --2- 172.21.15.133:0/493949830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340d39e0 0x7fa2341414e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.148+0000 7fa23a800640 1 --2- 172.21.15.133:0/493949830 >> v2:172.21.15.161:3300/0 conn(0x7fa234162260 0x7fa23415ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.148+0000 7fa23a800640 1 -- 172.21.15.133:0/493949830 >> 172.21.15.133:0/493949830 conn(0x7fa234070400 msgr2=0x7fa2340d2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.149+0000 7fb596000640 1 -- 172.21.15.133:0/3471771457 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598016020 con 0x7fb5a015ecf0 2024-03-07T06:51:29.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.149+0000 7f2315600640 1 -- 172.21.15.133:0/3841279462 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231000a4d0 con 0x7f22fc004100 2024-03-07T06:51:29.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.149+0000 7fb596000640 1 -- 172.21.15.133:0/3471771457 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb598006d50 con 0x7fb5a015ecf0 2024-03-07T06:51:29.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.149+0000 7f2315600640 1 -- 172.21.15.133:0/3841279462 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2310034650 con 0x7f22fc004100 2024-03-07T06:51:29.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.149+0000 7f2315600640 1 -- 172.21.15.133:0/3841279462 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310034a80 con 0x7f22fc004100 2024-03-07T06:51:29.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.149+0000 7f8286000640 1 -- 172.21.15.133:0/3308218926 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8280010e20 con 0x7f8290071950 2024-03-07T06:51:29.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.149+0000 7f6593e00640 1 -- 172.21.15.133:0/3926102934 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594041070 con 0x7f659c071b50 2024-03-07T06:51:29.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.149+0000 7fb596000640 1 -- 172.21.15.133:0/3471771457 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb598003bb0 con 0x7fb5a015ecf0 2024-03-07T06:51:29.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.149+0000 7fb5a4a00640 1 -- 172.21.15.133:0/3471771457 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a0159390 con 0x7fb5a015ecf0 2024-03-07T06:51:29.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.150+0000 7f6593e00640 1 -- 172.21.15.133:0/3926102934 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659403ac80 con 0x7f659c071b50 2024-03-07T06:51:29.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.150+0000 7f65a3200640 1 -- 172.21.15.133:0/3926102934 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659c159a80 con 0x7f659c071b50 2024-03-07T06:51:29.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.150+0000 7f65a3200640 1 -- 172.21.15.133:0/3926102934 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659c15a040 con 0x7f659c071b50 2024-03-07T06:51:29.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.151+0000 7f231da00640 1 -- 172.21.15.133:0/3841279462 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22fc004100 msgr2=0x7f22fc00b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.151+0000 7f231da00640 1 --2- 172.21.15.133:0/3841279462 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22fc004100 0x7f22fc00b010 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f231000a900 tx=0x7f22fc004820 comp rx=0 tx=0).stop 2024-03-07T06:51:29.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.151+0000 7f231da00640 1 -- 172.21.15.133:0/3841279462 shutdown_connections 2024-03-07T06:51:29.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.151+0000 7f231da00640 1 --2- 172.21.15.133:0/3841279462 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22fc004100 0x7f22fc00b010 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.151+0000 7f231da00640 1 -- 172.21.15.133:0/3841279462 >> 172.21.15.133:0/3841279462 conn(0x7f23180b3120 msgr2=0x7f23180b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.151+0000 7fb5a4a00640 1 -- 172.21.15.133:0/3471771457 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a0159950 con 0x7fb5a015ecf0 2024-03-07T06:51:29.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.152+0000 7f231da00640 1 -- 172.21.15.133:0/3841279462 shutdown_connections 2024-03-07T06:51:29.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.152+0000 7fa23a800640 1 -- 172.21.15.133:0/493949830 shutdown_connections 2024-03-07T06:51:29.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.152+0000 7fa23a800640 1 -- 172.21.15.133:0/493949830 wait complete. 2024-03-07T06:51:29.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.150+0000 7f6593e00640 1 -- 172.21.15.133:0/3926102934 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6594035040 con 0x7f659c071b50 2024-03-07T06:51:29.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.155+0000 7fa23a800640 1 Processor -- start 2024-03-07T06:51:29.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.155+0000 7fa23a800640 1 -- start start 2024-03-07T06:51:29.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.153+0000 7fb596000640 1 -- 172.21.15.133:0/3471771457 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb5980050d0 con 0x7fb5a015ecf0 2024-03-07T06:51:29.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.154+0000 7fb596000640 1 --2- 172.21.15.133:0/3471771457 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb58006b4d0 0x7fb58006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.154+0000 7fb596000640 1 -- 172.21.15.133:0/3471771457 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb598089900 con 0x7fb5a015ecf0 2024-03-07T06:51:29.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.155+0000 7fb59de00640 1 --2- 172.21.15.133:0/3471771457 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb58006b4d0 0x7fb58006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.156+0000 7f6593e00640 1 -- 172.21.15.133:0/3926102934 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f659404c060 con 0x7f659c071b50 2024-03-07T06:51:29.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.156+0000 7fb59de00640 1 --2- 172.21.15.133:0/3471771457 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb58006b4d0 0x7fb58006d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb58c036000 tx=0x7fb58c03c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.157+0000 7fb5a4a00640 1 -- 172.21.15.133:0/3471771457 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a0062c90 con 0x7fb5a015ecf0 2024-03-07T06:51:29.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.152+0000 7f231da00640 1 -- 172.21.15.133:0/3841279462 wait complete. 2024-03-07T06:51:29.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.158+0000 7f6593e00640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f657806b500 0x7f657806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.154+0000 7f231da00640 1 Processor -- start 2024-03-07T06:51:29.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.158+0000 7f65a3200640 1 -- 172.21.15.133:0/3926102934 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6574005290 con 0x7f659c071b50 2024-03-07T06:51:29.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.158+0000 7f6593e00640 1 -- 172.21.15.133:0/3926102934 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65940abb50 con 0x7f659c071b50 2024-03-07T06:51:29.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.158+0000 7f231da00640 1 -- start start 2024-03-07T06:51:29.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.157+0000 7fa23a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340d39e0 0x7fa2341f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.157+0000 7fa23a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa234162260 0x7fa234144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.159+0000 7f659b400640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f657806b500 0x7f657806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.157+0000 7fa23a800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa234162640 0x7fa234136400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.157+0000 7fa23a800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23415f120 con 0x7fa2340d39e0 2024-03-07T06:51:29.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.160+0000 7f659b400640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f657806b500 0x7f657806d9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f659c0b1360 tx=0x7f658c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.157+0000 7fa23a800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2340b4c50 con 0x7fa234162260 2024-03-07T06:51:29.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.157+0000 7fa23a800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23415f2a0 con 0x7fa234162640 2024-03-07T06:51:29.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.157+0000 7fa239400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340d39e0 0x7fa2341f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.158+0000 7fa239400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340d39e0 0x7fa2341f8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38786/0 (socket says 172.21.15.133:38786) 2024-03-07T06:51:29.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.158+0000 7fa239400640 1 -- 172.21.15.133:0/3580510831 learned_addr learned my addr 172.21.15.133:0/3580510831 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:29.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.158+0000 7fa239e00640 1 --2- 172.21.15.133:0/3580510831 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa234162640 0x7fa234136400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.158+0000 7fa239400640 1 -- 172.21.15.133:0/3580510831 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa234162640 msgr2=0x7fa234136400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.158+0000 7fa239400640 1 --2- 172.21.15.133:0/3580510831 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa234162640 0x7fa234136400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.158+0000 7fa239400640 1 -- 172.21.15.133:0/3580510831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa234162260 msgr2=0x7fa234144eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-07T06:51:29.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.158+0000 7fa239400640 1 --2- 172.21.15.133:0/3580510831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa234162260 0x7fa234144eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.158+0000 7fa239400640 1 -- 172.21.15.133:0/3580510831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa234137330 con 0x7fa2340d39e0 2024-03-07T06:51:29.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.159+0000 7fa239400640 1 --2- 172.21.15.133:0/3580510831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340d39e0 0x7fa2341f8110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009d90 tx=0x7fa22c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.160+0000 7f231da00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22fc004100 0x7f231813eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.160+0000 7f231da00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23181f08d0 0x7f2318155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.160+0000 7f231da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2318155ad0 0x7f23181f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.160+0000 7f2316a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22fc004100 0x7f231813eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.160+0000 7f231da00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231815ba40 con 0x7f2318155ad0 2024-03-07T06:51:29.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.160+0000 7f231da00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2318071ee0 con 0x7f22fc004100 2024-03-07T06:51:29.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.161+0000 7f2317400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2318155ad0 0x7f23181f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.161+0000 7fa231600640 1 -- 172.21.15.133:0/3580510831 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c00cda0 con 0x7fa2340d39e0 2024-03-07T06:51:29.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.161+0000 7fa231600640 1 -- 172.21.15.133:0/3580510831 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c019070 con 0x7fa2340d39e0 2024-03-07T06:51:29.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.161+0000 7fa231600640 1 -- 172.21.15.133:0/3580510831 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c014680 con 0x7fa2340d39e0 2024-03-07T06:51:29.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.161+0000 7f2317400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2318155ad0 0x7f23181f31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38796/0 (socket says 172.21.15.133:38796) 2024-03-07T06:51:29.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.161+0000 7f2317400640 1 -- 172.21.15.133:0/2607078117 learned_addr learned my addr 172.21.15.133:0/2607078117 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:29.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.161+0000 7f231da00640 1 -- 172.21.15.133:0/2607078117 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23180b1610 con 0x7f23181f08d0 2024-03-07T06:51:29.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.161+0000 7f2317400640 1 -- 172.21.15.133:0/2607078117 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23181f08d0 msgr2=0x7f2318155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.161+0000 7f2317400640 1 --2- 172.21.15.133:0/2607078117 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23181f08d0 0x7f2318155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.161+0000 7f2317400640 1 -- 172.21.15.133:0/2607078117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22fc004100 msgr2=0x7f231813eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.161+0000 7f2317400640 1 --2- 172.21.15.133:0/2607078117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22fc004100 0x7f231813eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.161+0000 7f2317400640 1 -- 172.21.15.133:0/2607078117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231813f430 con 0x7f2318155ad0 2024-03-07T06:51:29.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.162+0000 7fa23a800640 1 -- 172.21.15.133:0/3580510831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa234136d70 con 0x7fa2340d39e0 2024-03-07T06:51:29.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.162+0000 7fa23a800640 1 -- 172.21.15.133:0/3580510831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2341446b0 con 0x7fa2340d39e0 2024-03-07T06:51:29.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.162+0000 7f2317400640 1 --2- 172.21.15.133:0/2607078117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2318155ad0 0x7f23181f31d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f23180688a0 tx=0x7f2308010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.164+0000 7f230ea00640 1 -- 172.21.15.133:0/2607078117 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308004020 con 0x7f2318155ad0 2024-03-07T06:51:29.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.164+0000 7f230ea00640 1 -- 172.21.15.133:0/2607078117 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23080041c0 con 0x7f2318155ad0 2024-03-07T06:51:29.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.164+0000 7f230ea00640 1 -- 172.21.15.133:0/2607078117 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308004370 con 0x7f2318155ad0 2024-03-07T06:51:29.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.164+0000 7fa23a800640 1 -- 172.21.15.133:0/3580510831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2340b3a10 con 0x7fa2340d39e0 2024-03-07T06:51:29.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.164+0000 7f231da00640 1 -- 172.21.15.133:0/2607078117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23181f3e90 con 0x7f2318155ad0 2024-03-07T06:51:29.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.165+0000 7f231da00640 1 -- 172.21.15.133:0/2607078117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23181f00d0 con 0x7f2318155ad0 2024-03-07T06:51:29.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.165+0000 7fb596000640 1 -- 172.21.15.133:0/3471771457 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb598058780 con 0x7fb5a015ecf0 2024-03-07T06:51:29.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.166+0000 7fa231600640 1 -- 172.21.15.133:0/3580510831 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa22c005d10 con 0x7fa2340d39e0 2024-03-07T06:51:29.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.167+0000 7fa231600640 1 --2- 172.21.15.133:0/3580510831 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa21406b5a0 0x7fa21406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.168+0000 7fa231600640 1 -- 172.21.15.133:0/3580510831 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa22c08a760 con 0x7fa2340d39e0 2024-03-07T06:51:29.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.171+0000 7fa238a00640 1 --2- 172.21.15.133:0/3580510831 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa21406b5a0 0x7fa21406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.172+0000 7fa238a00640 1 --2- 172.21.15.133:0/3580510831 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa21406b5a0 0x7fa21406da60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa23413a9b0 tx=0x7fa228007840 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.173+0000 7f6593e00640 1 -- 172.21.15.133:0/3926102934 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f659407a9c0 con 0x7f659c071b50 2024-03-07T06:51:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.169+0000 7f231da00640 1 -- 172.21.15.133:0/2607078117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23181efbe0 con 0x7f2318155ad0 2024-03-07T06:51:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.170+0000 7f230ea00640 1 -- 172.21.15.133:0/2607078117 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f230801e020 con 0x7f2318155ad0 2024-03-07T06:51:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.172+0000 7f230ea00640 1 --2- 172.21.15.133:0/2607078117 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f22f806b5a0 0x7f22f806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.172+0000 7f230ea00640 1 -- 172.21.15.133:0/2607078117 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2308089e80 con 0x7f2318155ad0 2024-03-07T06:51:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.173+0000 7f2316a00640 1 --2- 172.21.15.133:0/2607078117 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f22f806b5a0 0x7f22f806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.174+0000 7f2316a00640 1 --2- 172.21.15.133:0/2607078117 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f22f806b5a0 0x7f22f806da60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f23181435b0 tx=0x7f2304004730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.176+0000 7fa231600640 1 -- 172.21.15.133:0/3580510831 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa2340b3a10 con 0x7fa2340d39e0 2024-03-07T06:51:29.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.185+0000 7f230ea00640 1 -- 172.21.15.133:0/2607078117 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2308058ce0 con 0x7f2318155ad0 2024-03-07T06:51:29.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.188+0000 7fc215400640 1 Processor -- start 2024-03-07T06:51:29.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.188+0000 7fc215400640 1 -- start start 2024-03-07T06:51:29.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.190+0000 7fc215400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc210165870 0x7fc2100718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.190+0000 7fc205e00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fc210071410 0x7fc21006e070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.133:60224/0 (socket says 172.21.15.133:60224) 2024-03-07T06:51:29.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.191+0000 7fc205e00640 1 -- 172.21.15.133:0/1502804286 learned_addr learned my addr 172.21.15.133:0/1502804286 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:29.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.191+0000 7fc215400640 1 -- 172.21.15.133:0/1502804286 --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc210163740 con 0x7fc2100b52c0 2024-03-07T06:51:29.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.191+0000 7fc215400640 1 -- 172.21.15.133:0/1502804286 --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc210163a20 con 0x7fc210071410 2024-03-07T06:51:29.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.191+0000 7fc215400640 1 -- 172.21.15.133:0/1502804286 --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc210163e10 con 0x7fc210165870 2024-03-07T06:51:29.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.191+0000 7fc20f200640 1 --2- 172.21.15.133:0/1502804286 >> v2:172.21.15.133:3300/0 conn(0x7fc210165870 0x7fc2100718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.192+0000 7fc20de00640 1 -- 172.21.15.133:0/1502804286 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556628390 0 0) 0x7fc210163740 con 0x7fc2100b52c0 2024-03-07T06:51:29.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.192+0000 7fc20de00640 1 -- 172.21.15.133:0/1502804286 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1f4003520 con 0x7fc2100b52c0 2024-03-07T06:51:29.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.192+0000 7fc20f200640 1 -- 172.21.15.133:0/1502804286 >> v1:172.21.15.161:6789/0 conn(0x7fc210071410 legacy=0x7fc21006e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:29.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.192+0000 7fc20de00640 1 -- 172.21.15.133:0/1502804286 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3757095995 0 0) 0x7fc1f4003520 con 0x7fc2100b52c0 2024-03-07T06:51:29.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.194+0000 7fc20f200640 1 -- 172.21.15.133:0/1502804286 >> v1:172.21.15.133:6789/0 conn(0x7fc2100b52c0 legacy=0x7fc210162c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.194+0000 7fc20f200640 1 -- 172.21.15.133:0/1502804286 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc210164060 con 0x7fc210165870 2024-03-07T06:51:29.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.195+0000 7fc20f200640 1 --2- 172.21.15.133:0/1502804286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc210165870 0x7fc2100718a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc208009d90 tx=0x7fc20802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed0132da1f6d907 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.196+0000 7fc20de00640 1 -- 172.21.15.133:0/1502804286 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208039070 con 0x7fc210165870 2024-03-07T06:51:29.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.196+0000 7fc20de00640 1 -- 172.21.15.133:0/1502804286 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20802c940 con 0x7fc210165870 2024-03-07T06:51:29.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.197+0000 7fc20de00640 1 -- 172.21.15.133:0/1502804286 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8001670 con 0x7fc210165870 2024-03-07T06:51:29.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.197+0000 7fc215400640 1 -- 172.21.15.133:0/1502804286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc210165870 msgr2=0x7fc2100718a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.197+0000 7fc215400640 1 --2- 172.21.15.133:0/1502804286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc210165870 0x7fc2100718a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc208009d90 tx=0x7fc20802be10 comp rx=0 tx=0).stop 2024-03-07T06:51:29.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.197+0000 7f483fe00640 1 Processor -- start 2024-03-07T06:51:29.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.197+0000 7f483fe00640 1 -- start start 2024-03-07T06:51:29.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.198+0000 7fc215400640 1 -- 172.21.15.133:0/1502804286 shutdown_connections 2024-03-07T06:51:29.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.198+0000 7fc215400640 1 --2- 172.21.15.133:0/1502804286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc210165870 0x7fc2100718a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.198+0000 7fc215400640 1 -- 172.21.15.133:0/1502804286 >> 172.21.15.133:0/1502804286 conn(0x7fc2100d0e70 msgr2=0x7fc2100d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.199+0000 7f483fe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f48380b5dd0 0x7f4838144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.199+0000 7f483fe00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48380b43f0 0x7f4838162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.199+0000 7f483fe00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48380b4bd0 con 0x7f4838162ab0 2024-03-07T06:51:29.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.199+0000 7fc215400640 1 -- 172.21.15.133:0/1502804286 shutdown_connections 2024-03-07T06:51:29.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.199+0000 7f483fe00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4838141700 con 0x7f48380b5dd0 2024-03-07T06:51:29.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.199+0000 7f483fe00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4838161e00 con 0x7f48380b43f0 2024-03-07T06:51:29.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.200+0000 7f483de00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4838162ab0 0x7f4838071410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44198/0 (socket says 172.21.15.133:44198) 2024-03-07T06:51:29.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.200+0000 7f483de00640 1 -- 172.21.15.133:0/439901610 learned_addr learned my addr 172.21.15.133:0/439901610 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:29.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.200+0000 7f483ca00640 1 --2- 172.21.15.133:0/439901610 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48380b43f0 0x7f4838162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.200+0000 7f483d400640 1 --2- 172.21.15.133:0/439901610 >> v2:172.21.15.133:3300/0 conn(0x7f48380b5dd0 0x7f4838144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.201+0000 7f4837e00640 1 -- 172.21.15.133:0/439901610 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1364824620 0 0) 0x7f48380b4bd0 con 0x7f4838162ab0 2024-03-07T06:51:29.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.201+0000 7f4837e00640 1 -- 172.21.15.133:0/439901610 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4820003520 con 0x7f4838162ab0 2024-03-07T06:51:29.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.201+0000 7f483ca00640 1 -- 172.21.15.133:0/439901610 >> v2:172.21.15.133:3300/0 conn(0x7f48380b5dd0 msgr2=0x7f4838144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:29.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.200+0000 7fc215400640 1 -- 172.21.15.133:0/1502804286 wait complete. 2024-03-07T06:51:29.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.203+0000 7fc215400640 1 Processor -- start 2024-03-07T06:51:29.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.203+0000 7fc215400640 1 -- start start 2024-03-07T06:51:29.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.202+0000 7f4837e00640 1 -- 172.21.15.133:0/439901610 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 796626451 0 0) 0x7f4820003520 con 0x7f4838162ab0 2024-03-07T06:51:29.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.203+0000 7f483ca00640 1 --2- 172.21.15.133:0/439901610 >> v2:172.21.15.133:3300/0 conn(0x7f48380b5dd0 0x7f4838144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.203+0000 7f483ca00640 1 -- 172.21.15.133:0/439901610 >> v1:172.21.15.133:6789/0 conn(0x7f4838162ab0 legacy=0x7f4838071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.203+0000 7f483ca00640 1 -- 172.21.15.133:0/439901610 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48380b4eb0 con 0x7f48380b43f0 2024-03-07T06:51:29.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.203+0000 7f483d400640 1 --2- 172.21.15.133:0/439901610 >> v2:172.21.15.133:3300/0 conn(0x7f48380b5dd0 0x7f4838144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T06:51:29.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.205+0000 7fc215400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc210071410 0x7fc2101fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.205+0000 7fc215400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2100b52c0 0x7fc2101f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.205+0000 7fc215400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc210165870 0x7fc21006d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.205+0000 7fc215400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc210163360 con 0x7fc2100b52c0 2024-03-07T06:51:29.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.205+0000 7fc215400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2100b7010 con 0x7fc210165870 2024-03-07T06:51:29.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.205+0000 7fc215400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2101634e0 con 0x7fc210071410 2024-03-07T06:51:29.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.205+0000 7f483ca00640 1 --2- 172.21.15.133:0/439901610 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48380b43f0 0x7f4838162570 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4838161f80 tx=0x7f482402b6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf41a1887ece397 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.205+0000 7fc205e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2100b52c0 0x7fc2101f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.205+0000 7fc20e800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc210071410 0x7fc2101fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.205+0000 7fc20f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc210165870 0x7fc21006d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.206+0000 7fc20f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc210165870 0x7fc21006d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:59094/0 (socket says 172.21.15.133:59094) 2024-03-07T06:51:29.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.206+0000 7fc20f200640 1 -- 172.21.15.133:0/2069103811 learned_addr learned my addr 172.21.15.133:0/2069103811 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:29.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.206+0000 7fc205e00640 1 --2- 172.21.15.133:0/2069103811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2100b52c0 0x7fc2101f8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38828/0 (socket says 172.21.15.133:38828) 2024-03-07T06:51:29.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.206+0000 7f4837e00640 1 -- 172.21.15.133:0/439901610 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824037070 con 0x7f48380b43f0 2024-03-07T06:51:29.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.206+0000 7f4837e00640 1 -- 172.21.15.133:0/439901610 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4824030510 con 0x7f48380b43f0 2024-03-07T06:51:29.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.206+0000 7f4837e00640 1 -- 172.21.15.133:0/439901610 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824037070 con 0x7f48380b43f0 2024-03-07T06:51:29.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.206+0000 7fc20f200640 1 -- 172.21.15.133:0/2069103811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc210071410 msgr2=0x7fc2101fafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.206+0000 7fc20f200640 1 --2- 172.21.15.133:0/2069103811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc210071410 0x7fc2101fafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.206+0000 7fc20f200640 1 -- 172.21.15.133:0/2069103811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2100b52c0 msgr2=0x7fc2101f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.207+0000 7fc20f200640 1 --2- 172.21.15.133:0/2069103811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2100b52c0 0x7fc2101f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.207+0000 7fc20f200640 1 -- 172.21.15.133:0/2069103811 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2101f8cd0 con 0x7fc210165870 2024-03-07T06:51:29.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.207+0000 7fc205e00640 1 --2- 172.21.15.133:0/2069103811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2100b52c0 0x7fc2101f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T06:51:29.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.207+0000 7f483fe00640 1 -- 172.21.15.133:0/439901610 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48380b43f0 msgr2=0x7f4838162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.207+0000 7f483fe00640 1 --2- 172.21.15.133:0/439901610 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48380b43f0 0x7f4838162570 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4838161f80 tx=0x7f482402b6f0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.208+0000 7f483fe00640 1 -- 172.21.15.133:0/439901610 shutdown_connections 2024-03-07T06:51:29.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.208+0000 7f483fe00640 1 --2- 172.21.15.133:0/439901610 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48380b43f0 0x7f4838162570 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.208+0000 7f483fe00640 1 --2- 172.21.15.133:0/439901610 >> v2:172.21.15.133:3300/0 conn(0x7f48380b5dd0 0x7f4838144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.208+0000 7f483fe00640 1 -- 172.21.15.133:0/439901610 >> 172.21.15.133:0/439901610 conn(0x7f48380d0e70 msgr2=0x7f48380d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.208+0000 7f483fe00640 1 -- 172.21.15.133:0/439901610 shutdown_connections 2024-03-07T06:51:29.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.208+0000 7fc20f200640 1 --2- 172.21.15.133:0/2069103811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc210165870 0x7fc21006d620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc208009eb0 tx=0x7fc208004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.208+0000 7fc207400640 1 -- 172.21.15.133:0/2069103811 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208044030 con 0x7fc210165870 2024-03-07T06:51:29.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.208+0000 7f483fe00640 1 -- 172.21.15.133:0/439901610 wait complete. 2024-03-07T06:51:29.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.209+0000 7fc215400640 1 -- 172.21.15.133:0/2069103811 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2101f8fb0 con 0x7fc210165870 2024-03-07T06:51:29.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.209+0000 7fc207400640 1 -- 172.21.15.133:0/2069103811 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20803a040 con 0x7fc210165870 2024-03-07T06:51:29.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.209+0000 7fc207400640 1 -- 172.21.15.133:0/2069103811 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208039070 con 0x7fc210165870 2024-03-07T06:51:29.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.209+0000 7fc215400640 1 -- 172.21.15.133:0/2069103811 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21015d790 con 0x7fc210165870 2024-03-07T06:51:29.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.210+0000 7f483fe00640 1 Processor -- start 2024-03-07T06:51:29.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.211+0000 7fc215400640 1 -- 172.21.15.133:0/2069103811 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21015d990 con 0x7fc210165870 2024-03-07T06:51:29.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.211+0000 7f483fe00640 1 -- start start 2024-03-07T06:51:29.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.213+0000 7f483fe00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48380b43f0 0x7f483813ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.213+0000 7f483fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48380b5dd0 0x7f48381f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.213+0000 7f483fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48380716c0 0x7f48381fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.213+0000 7f483fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48380b61b0 con 0x7f48380716c0 2024-03-07T06:51:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.213+0000 7f483fe00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4838162050 con 0x7f48380b5dd0 2024-03-07T06:51:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.213+0000 7f483fe00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48380b4870 con 0x7f48380b43f0 2024-03-07T06:51:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.213+0000 7f483d400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48380b43f0 0x7f483813ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.213+0000 7f483de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48380716c0 0x7f48381fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.213+0000 7f483d400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48380b43f0 0x7f483813ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:42256/0 (socket says 172.21.15.133:42256) 2024-03-07T06:51:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.213+0000 7f483d400640 1 -- 172.21.15.133:0/2581512933 learned_addr learned my addr 172.21.15.133:0/2581512933 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:29.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.214+0000 7f483ca00640 1 --2- 172.21.15.133:0/2581512933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48380b5dd0 0x7f48381f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.214+0000 7f483de00640 1 -- 172.21.15.133:0/2581512933 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48380b43f0 msgr2=0x7f483813ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.214+0000 7f483de00640 1 --2- 172.21.15.133:0/2581512933 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48380b43f0 0x7f483813ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.214+0000 7f483de00640 1 -- 172.21.15.133:0/2581512933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48380b5dd0 msgr2=0x7f48381f80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.214+0000 7f483de00640 1 --2- 172.21.15.133:0/2581512933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48380b5dd0 0x7f48381f80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.214+0000 7f483de00640 1 -- 172.21.15.133:0/2581512933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48381f8830 con 0x7f48380716c0 2024-03-07T06:51:29.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.215+0000 7f483de00640 1 --2- 172.21.15.133:0/2581512933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48380716c0 0x7f48381fbd70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f482c000c00 tx=0x7f482c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.215+0000 7f4835600640 1 -- 172.21.15.133:0/2581512933 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c017070 con 0x7f48380716c0 2024-03-07T06:51:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.215+0000 7f4835600640 1 -- 172.21.15.133:0/2581512933 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c005af0 con 0x7f48380716c0 2024-03-07T06:51:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.215+0000 7f483fe00640 1 -- 172.21.15.133:0/2581512933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48381f8b10 con 0x7f48380716c0 2024-03-07T06:51:29.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.215+0000 7f4835600640 1 -- 172.21.15.133:0/2581512933 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c007070 con 0x7f48380716c0 2024-03-07T06:51:29.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.215+0000 7fc207400640 1 -- 172.21.15.133:0/2069103811 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc208047070 con 0x7fc210165870 2024-03-07T06:51:29.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.216+0000 7f483fe00640 1 -- 172.21.15.133:0/2581512933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48381fb850 con 0x7f48380716c0 2024-03-07T06:51:29.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.217+0000 7fc207400640 1 --2- 172.21.15.133:0/2069103811 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc1ec06b4d0 0x7fc1ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.217+0000 7f483fe00640 1 -- 172.21.15.133:0/2581512933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4838062c90 con 0x7f48380716c0 2024-03-07T06:51:29.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.219+0000 7f4835600640 1 -- 172.21.15.133:0/2581512933 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f482c01a070 con 0x7f48380716c0 2024-03-07T06:51:29.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.217+0000 7fc207400640 1 -- 172.21.15.133:0/2069103811 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2080ad000 con 0x7fc210165870 2024-03-07T06:51:29.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.221+0000 7fc20e800640 1 --2- 172.21.15.133:0/2069103811 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc1ec06b4d0 0x7fc1ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.221+0000 7f4835600640 1 --2- 172.21.15.133:0/2581512933 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f481806b4d0 0x7f481806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.221+0000 7f4835600640 1 -- 172.21.15.133:0/2581512933 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f482c089710 con 0x7f48380716c0 2024-03-07T06:51:29.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.222+0000 7f483d400640 1 --2- 172.21.15.133:0/2581512933 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f481806b4d0 0x7f481806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.222+0000 7fc20e800640 1 --2- 172.21.15.133:0/2069103811 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc1ec06b4d0 0x7fc1ec06d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80017e0 tx=0x7fc1f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.223 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.223+0000 7f483d400640 1 --2- 172.21.15.133:0/2581512933 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f481806b4d0 0x7f481806d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4830009bd0 tx=0x7f483000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.229+0000 7fc207400640 1 -- 172.21.15.133:0/2069103811 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc21015d990 con 0x7fc210165870 2024-03-07T06:51:29.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.230+0000 7f4835600640 1 -- 172.21.15.133:0/2581512933 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f482c057f30 con 0x7f48380716c0 2024-03-07T06:51:29.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.560+0000 7f190de00640 1 -- 172.21.15.133:0/1353553104 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1900056bf0 con 0x7f190003c0a0 2024-03-07T06:51:29.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.561+0000 7f1904c00640 1 -- 172.21.15.133:0/1353553104 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f18f405c830 con 0x7f190003c0a0 2024-03-07T06:51:29.561 INFO:teuthology.orchestra.run.smithi133.stdout:55834574856 2024-03-07T06:51:29.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.565+0000 7f190de00640 1 -- 172.21.15.133:0/1353553104 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f18e806b520 msgr2=0x7f18e806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.565+0000 7f190de00640 1 --2- 172.21.15.133:0/1353553104 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f18e806b520 0x7f18e806d9e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1908050170 tx=0x7f19080b7040 comp rx=0 tx=0).stop 2024-03-07T06:51:29.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.565+0000 7f190de00640 1 -- 172.21.15.133:0/1353553104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f190003c0a0 msgr2=0x7f19000d6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.565+0000 7f190de00640 1 --2- 172.21.15.133:0/1353553104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f190003c0a0 0x7f19000d6830 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f18f400e870 tx=0x7f18f400ed30 comp rx=0 tx=0).stop 2024-03-07T06:51:29.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.565+0000 7f190de00640 1 -- 172.21.15.133:0/1353553104 shutdown_connections 2024-03-07T06:51:29.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.565+0000 7f190de00640 1 --2- 172.21.15.133:0/1353553104 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f18e806b520 0x7f18e806d9e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.565+0000 7f190de00640 1 --2- 172.21.15.133:0/1353553104 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19000e3160 0x7f19000e3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.565+0000 7f190de00640 1 --2- 172.21.15.133:0/1353553104 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1900053cb0 0x7f190004e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.565+0000 7f190de00640 1 --2- 172.21.15.133:0/1353553104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f190003c0a0 0x7f19000d6830 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.565+0000 7f190de00640 1 -- 172.21.15.133:0/1353553104 >> 172.21.15.133:0/1353553104 conn(0x7f1900058ee0 msgr2=0x7f1900006f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.566+0000 7f190de00640 1 -- 172.21.15.133:0/1353553104 shutdown_connections 2024-03-07T06:51:29.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.566+0000 7f190de00640 1 -- 172.21.15.133:0/1353553104 wait complete. 2024-03-07T06:51:29.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.570+0000 7f8295200640 1 -- 172.21.15.133:0/3308218926 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8290165b10 con 0x7f8290071950 2024-03-07T06:51:29.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.571+0000 7f8286000640 1 -- 172.21.15.133:0/3308218926 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8280058950 con 0x7f8290071950 2024-03-07T06:51:29.571 INFO:teuthology.orchestra.run.smithi133.stdout:55834574857 2024-03-07T06:51:29.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.572+0000 7fb5a4a00640 1 -- 172.21.15.133:0/3471771457 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb5a00b7200 con 0x7fb5a015ecf0 2024-03-07T06:51:29.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.572+0000 7fb596000640 1 -- 172.21.15.133:0/3471771457 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb598058080 con 0x7fb5a015ecf0 2024-03-07T06:51:29.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.575+0000 7f8295200640 1 -- 172.21.15.133:0/3308218926 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f827006b5d0 msgr2=0x7f827006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.575+0000 7f8295200640 1 --2- 172.21.15.133:0/3308218926 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f827006b5d0 0x7f827006da90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f827c036000 tx=0x7f827c008030 comp rx=0 tx=0).stop 2024-03-07T06:51:29.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.575+0000 7f8295200640 1 -- 172.21.15.133:0/3308218926 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8290071950 msgr2=0x7f829015fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.575+0000 7f8295200640 1 --2- 172.21.15.133:0/3308218926 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8290071950 0x7f829015fab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f828000b480 tx=0x7f828000b940 comp rx=0 tx=0).stop 2024-03-07T06:51:29.576 INFO:teuthology.orchestra.run.smithi133.stdout:55834574855 2024-03-07T06:51:29.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.575+0000 7f8295200640 1 -- 172.21.15.133:0/3308218926 shutdown_connections 2024-03-07T06:51:29.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.576+0000 7f8295200640 1 --2- 172.21.15.133:0/3308218926 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f827006b5d0 0x7f827006da90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.576+0000 7f8295200640 1 --2- 172.21.15.133:0/3308218926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8290163190 0x7f8290150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.576+0000 7f8295200640 1 --2- 172.21.15.133:0/3308218926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82900b52c0 0x7f829014fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.576+0000 7f8295200640 1 --2- 172.21.15.133:0/3308218926 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8290071950 0x7f829015fab0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.576+0000 7f8295200640 1 -- 172.21.15.133:0/3308218926 >> 172.21.15.133:0/3308218926 conn(0x7f82900d0e70 msgr2=0x7f82900d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.576+0000 7f8295200640 1 -- 172.21.15.133:0/3308218926 shutdown_connections 2024-03-07T06:51:29.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.576+0000 7f8295200640 1 -- 172.21.15.133:0/3308218926 wait complete. 2024-03-07T06:51:29.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.578+0000 7fb5a4a00640 1 -- 172.21.15.133:0/3471771457 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb58006b4d0 msgr2=0x7fb58006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.578+0000 7fb5a4a00640 1 --2- 172.21.15.133:0/3471771457 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb58006b4d0 0x7fb58006d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb58c036000 tx=0x7fb58c03c040 comp rx=0 tx=0).stop 2024-03-07T06:51:29.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.578+0000 7fb5a4a00640 1 -- 172.21.15.133:0/3471771457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5a015ecf0 msgr2=0x7fb5a01f3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.578+0000 7fb5a4a00640 1 --2- 172.21.15.133:0/3471771457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5a015ecf0 0x7fb5a01f3f70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb59800c730 tx=0x7fb59800cb30 comp rx=0 tx=0).stop 2024-03-07T06:51:29.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.578+0000 7fb5a4a00640 1 -- 172.21.15.133:0/3471771457 shutdown_connections 2024-03-07T06:51:29.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.578+0000 7fb5a4a00640 1 --2- 172.21.15.133:0/3471771457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5a015ecf0 0x7fb5a01f3f70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.578+0000 7fb5a4a00640 1 --2- 172.21.15.133:0/3471771457 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5a0141370 0x7fb5a013fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.578+0000 7fb5a4a00640 1 --2- 172.21.15.133:0/3471771457 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb58006b4d0 0x7fb58006d990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.578+0000 7fb5a4a00640 1 --2- 172.21.15.133:0/3471771457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5a00b9180 0x7fb5a015a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.578+0000 7fb5a4a00640 1 -- 172.21.15.133:0/3471771457 >> 172.21.15.133:0/3471771457 conn(0x7fb5a0072fc0 msgr2=0x7fb5a01405d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.579+0000 7fb5a4a00640 1 -- 172.21.15.133:0/3471771457 shutdown_connections 2024-03-07T06:51:29.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.579+0000 7fb5a4a00640 1 -- 172.21.15.133:0/3471771457 wait complete. 2024-03-07T06:51:29.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2024-03-07T06:51:29.594 DEBUG:teuthology.parallel:result is None 2024-03-07T06:51:29.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.601+0000 7f65a3200640 1 -- 172.21.15.133:0/3926102934 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6574005580 con 0x7f659c071b50 2024-03-07T06:51:29.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.601+0000 7f6593e00640 1 -- 172.21.15.133:0/3926102934 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f659407a2c0 con 0x7f659c071b50 2024-03-07T06:51:29.602 INFO:teuthology.orchestra.run.smithi133.stdout:55834574856 2024-03-07T06:51:29.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2024-03-07T06:51:29.602 DEBUG:teuthology.parallel:result is None 2024-03-07T06:51:29.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.604+0000 7f65a3200640 1 -- 172.21.15.133:0/3926102934 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f657806b500 msgr2=0x7f657806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.604+0000 7f65a3200640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f657806b500 0x7f657806d9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f659c0b1360 tx=0x7f658c00b040 comp rx=0 tx=0).stop 2024-03-07T06:51:29.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.604+0000 7f65a3200640 1 -- 172.21.15.133:0/3926102934 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f659c071b50 msgr2=0x7f659c09ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.604+0000 7f65a3200640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f659c071b50 0x7f659c09ccf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f659402be40 tx=0x7f65940046c0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.604+0000 7f65a3200640 1 -- 172.21.15.133:0/3926102934 shutdown_connections 2024-03-07T06:51:29.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.604+0000 7f65a3200640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f659c09c070 0x7f659c15e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.604+0000 7f65a3200640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f657806b500 0x7f657806d9c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.604+0000 7f65a3200640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f659c0b3c10 0x7f659c157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.604+0000 7f65a3200640 1 --2- 172.21.15.133:0/3926102934 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f659c071b50 0x7f659c09ccf0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f659402be40 tx=0x7f65940046c0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.605 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-03-07T06:51:29.605 DEBUG:teuthology.parallel:result is None 2024-03-07T06:51:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.604+0000 7f65a3200640 1 -- 172.21.15.133:0/3926102934 >> 172.21.15.133:0/3926102934 conn(0x7f659c0b3120 msgr2=0x7f659c1638d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.604+0000 7f65a3200640 1 -- 172.21.15.133:0/3926102934 shutdown_connections 2024-03-07T06:51:29.606 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.604+0000 7f65a3200640 1 -- 172.21.15.133:0/3926102934 wait complete. 2024-03-07T06:51:29.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.613+0000 7fa23a800640 1 -- 172.21.15.133:0/3580510831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa234062c90 con 0x7fa2340d39e0 2024-03-07T06:51:29.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.614+0000 7fa231600640 1 -- 172.21.15.133:0/3580510831 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa22c010040 con 0x7fa2340d39e0 2024-03-07T06:51:29.614 INFO:teuthology.orchestra.run.smithi133.stdout:55834574855 2024-03-07T06:51:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.618+0000 7fa23a800640 1 -- 172.21.15.133:0/3580510831 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa21406b5a0 msgr2=0x7fa21406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.618+0000 7fa23a800640 1 --2- 172.21.15.133:0/3580510831 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa21406b5a0 0x7fa21406da60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa23413a9b0 tx=0x7fa228007840 comp rx=0 tx=0).stop 2024-03-07T06:51:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.618+0000 7fa23a800640 1 -- 172.21.15.133:0/3580510831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340d39e0 msgr2=0x7fa2341f8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.618+0000 7fa23a800640 1 --2- 172.21.15.133:0/3580510831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340d39e0 0x7fa2341f8110 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009d90 tx=0x7fa22c00c490 comp rx=0 tx=0).stop 2024-03-07T06:51:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.618+0000 7fa23a800640 1 -- 172.21.15.133:0/3580510831 shutdown_connections 2024-03-07T06:51:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.618+0000 7fa23a800640 1 --2- 172.21.15.133:0/3580510831 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa234162640 0x7fa234136400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.618+0000 7fa23a800640 1 --2- 172.21.15.133:0/3580510831 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa21406b5a0 0x7fa21406da60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.618+0000 7fa23a800640 1 --2- 172.21.15.133:0/3580510831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa234162260 0x7fa234144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.618+0000 7fa23a800640 1 --2- 172.21.15.133:0/3580510831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa2340d39e0 0x7fa2341f8110 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.618+0000 7fa23a800640 1 -- 172.21.15.133:0/3580510831 >> 172.21.15.133:0/3580510831 conn(0x7fa234070400 msgr2=0x7fa2340b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.618+0000 7fa23a800640 1 -- 172.21.15.133:0/3580510831 shutdown_connections 2024-03-07T06:51:29.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.618+0000 7fa23a800640 1 -- 172.21.15.133:0/3580510831 wait complete. 2024-03-07T06:51:29.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.621+0000 7fc215400640 1 -- 172.21.15.133:0/2069103811 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc210062c90 con 0x7fc210165870 2024-03-07T06:51:29.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2024-03-07T06:51:29.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.622+0000 7fc207400640 1 -- 172.21.15.133:0/2069103811 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc20807bf80 con 0x7fc210165870 2024-03-07T06:51:29.622 INFO:teuthology.orchestra.run.smithi133.stdout:55834574856 2024-03-07T06:51:29.622 DEBUG:teuthology.parallel:result is None 2024-03-07T06:51:29.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.625+0000 7fc215400640 1 -- 172.21.15.133:0/2069103811 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc1ec06b4d0 msgr2=0x7fc1ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.625+0000 7fc215400640 1 --2- 172.21.15.133:0/2069103811 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc1ec06b4d0 0x7fc1ec06d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80017e0 tx=0x7fc1f8009040 comp rx=0 tx=0).stop 2024-03-07T06:51:29.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.625+0000 7fc215400640 1 -- 172.21.15.133:0/2069103811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc210165870 msgr2=0x7fc21006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.625+0000 7fc215400640 1 --2- 172.21.15.133:0/2069103811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc210165870 0x7fc21006d620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc208009eb0 tx=0x7fc208004a40 comp rx=0 tx=0).stop 2024-03-07T06:51:29.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.625+0000 7f231da00640 1 -- 172.21.15.133:0/2607078117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2318062c90 con 0x7f2318155ad0 2024-03-07T06:51:29.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.625+0000 7fc215400640 1 -- 172.21.15.133:0/2069103811 shutdown_connections 2024-03-07T06:51:29.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.625+0000 7fc215400640 1 --2- 172.21.15.133:0/2069103811 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc1ec06b4d0 0x7fc1ec06d990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.625+0000 7fc215400640 1 --2- 172.21.15.133:0/2069103811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc210165870 0x7fc21006d620 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.625+0000 7fc215400640 1 --2- 172.21.15.133:0/2069103811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2100b52c0 0x7fc2101f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.625+0000 7fc215400640 1 --2- 172.21.15.133:0/2069103811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc210071410 0x7fc2101fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.625+0000 7fc215400640 1 -- 172.21.15.133:0/2069103811 >> 172.21.15.133:0/2069103811 conn(0x7fc2100d0e70 msgr2=0x7fc210162c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.625+0000 7fc215400640 1 -- 172.21.15.133:0/2069103811 shutdown_connections 2024-03-07T06:51:29.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.625+0000 7fc215400640 1 -- 172.21.15.133:0/2069103811 wait complete. 2024-03-07T06:51:29.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.625+0000 7f230ea00640 1 -- 172.21.15.133:0/2607078117 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f23080585e0 con 0x7f2318155ad0 2024-03-07T06:51:29.627 INFO:teuthology.orchestra.run.smithi133.stdout:55834574856 2024-03-07T06:51:29.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.627+0000 7f483fe00640 1 -- 172.21.15.133:0/2581512933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f48380b8a10 con 0x7f48380716c0 2024-03-07T06:51:29.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.628+0000 7f4835600640 1 -- 172.21.15.133:0/2581512933 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f482c05bfd0 con 0x7f48380716c0 2024-03-07T06:51:29.627 INFO:teuthology.orchestra.run.smithi133.stdout:55834574855 2024-03-07T06:51:29.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.628+0000 7f231da00640 1 -- 172.21.15.133:0/2607078117 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f22f806b5a0 msgr2=0x7f22f806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.628+0000 7f231da00640 1 --2- 172.21.15.133:0/2607078117 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f22f806b5a0 0x7f22f806da60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f23181435b0 tx=0x7f2304004730 comp rx=0 tx=0).stop 2024-03-07T06:51:29.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.628+0000 7f231da00640 1 -- 172.21.15.133:0/2607078117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2318155ad0 msgr2=0x7f23181f31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.628+0000 7f231da00640 1 --2- 172.21.15.133:0/2607078117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2318155ad0 0x7f23181f31d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f23180688a0 tx=0x7f2308010c80 comp rx=0 tx=0).stop 2024-03-07T06:51:29.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.629+0000 7f231da00640 1 -- 172.21.15.133:0/2607078117 shutdown_connections 2024-03-07T06:51:29.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.629+0000 7f231da00640 1 --2- 172.21.15.133:0/2607078117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2318155ad0 0x7f23181f31d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.629+0000 7f231da00640 1 --2- 172.21.15.133:0/2607078117 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23181f08d0 0x7f2318155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.629+0000 7f231da00640 1 --2- 172.21.15.133:0/2607078117 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f22f806b5a0 0x7f22f806da60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.629+0000 7f231da00640 1 --2- 172.21.15.133:0/2607078117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22fc004100 0x7f231813eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.629+0000 7f231da00640 1 -- 172.21.15.133:0/2607078117 >> 172.21.15.133:0/2607078117 conn(0x7f23180b3120 msgr2=0x7f23180b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.629+0000 7f231da00640 1 -- 172.21.15.133:0/2607078117 shutdown_connections 2024-03-07T06:51:29.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.629+0000 7f231da00640 1 -- 172.21.15.133:0/2607078117 wait complete. 2024-03-07T06:51:29.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.630+0000 7f483fe00640 1 -- 172.21.15.133:0/2581512933 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f481806b4d0 msgr2=0x7f481806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.630+0000 7f483fe00640 1 --2- 172.21.15.133:0/2581512933 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f481806b4d0 0x7f481806d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4830009bd0 tx=0x7f483000b040 comp rx=0 tx=0).stop 2024-03-07T06:51:29.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.630+0000 7f483fe00640 1 -- 172.21.15.133:0/2581512933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48380716c0 msgr2=0x7f48381fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.630+0000 7f483fe00640 1 --2- 172.21.15.133:0/2581512933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48380716c0 0x7f48381fbd70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f482c000c00 tx=0x7f482c00cbe0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.630+0000 7f483fe00640 1 -- 172.21.15.133:0/2581512933 shutdown_connections 2024-03-07T06:51:29.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.630+0000 7f483fe00640 1 --2- 172.21.15.133:0/2581512933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48380716c0 0x7f48381fbd70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.630+0000 7f483fe00640 1 --2- 172.21.15.133:0/2581512933 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f481806b4d0 0x7f481806d990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.630+0000 7f483fe00640 1 --2- 172.21.15.133:0/2581512933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48380b5dd0 0x7f48381f80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.630+0000 7f483fe00640 1 --2- 172.21.15.133:0/2581512933 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48380b43f0 0x7f483813ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.630+0000 7f483fe00640 1 -- 172.21.15.133:0/2581512933 >> 172.21.15.133:0/2581512933 conn(0x7f48380d0e70 msgr2=0x7f48380b7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.631+0000 7f483fe00640 1 -- 172.21.15.133:0/2581512933 shutdown_connections 2024-03-07T06:51:29.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.631+0000 7f483fe00640 1 -- 172.21.15.133:0/2581512933 wait complete. 2024-03-07T06:51:29.640 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-03-07T06:51:29.640 DEBUG:teuthology.parallel:result is None 2024-03-07T06:51:29.647 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-03-07T06:51:29.647 DEBUG:teuthology.parallel:result is None 2024-03-07T06:51:29.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2024-03-07T06:51:29.648 DEBUG:teuthology.parallel:result is None 2024-03-07T06:51:29.650 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-03-07T06:51:29.650 DEBUG:teuthology.parallel:result is None 2024-03-07T06:51:29.650 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-07T06:51:29.650 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:51:29.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.890+0000 7f8e9e600640 1 Processor -- start 2024-03-07T06:51:29.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.890+0000 7f8e9e600640 1 -- start start 2024-03-07T06:51:29.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.891+0000 7f8e9e600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8e980af950 0x7f8e980b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.891+0000 7f8e9e600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e9806f140 con 0x7f8e980ce5f0 2024-03-07T06:51:29.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.891+0000 7f8e9e600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e9806f420 con 0x7f8e9815b1b0 2024-03-07T06:51:29.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.891+0000 7f8e9e600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e98071610 con 0x7f8e980af950 2024-03-07T06:51:29.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.892+0000 7f8e97e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8e980af950 0x7f8e980b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.892+0000 7f8e96a00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8e980ce5f0 0x7f8e98071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44200/0 (socket says 172.21.15.133:44200) 2024-03-07T06:51:29.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.892+0000 7f8e96a00640 1 -- 172.21.15.133:0/1236651710 learned_addr learned my addr 172.21.15.133:0/1236651710 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:29.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.892+0000 7f8e97e00640 1 -- 172.21.15.133:0/1236651710 >> v1:172.21.15.161:6789/0 conn(0x7f8e9815b1b0 legacy=0x7f8e980cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:29.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.892+0000 7f8e97e00640 1 -- 172.21.15.133:0/1236651710 >> v1:172.21.15.133:6789/0 conn(0x7f8e980ce5f0 legacy=0x7f8e98071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.892+0000 7f8e97e00640 1 -- 172.21.15.133:0/1236651710 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e98071860 con 0x7f8e980af950 2024-03-07T06:51:29.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.892+0000 7f8e96000640 1 -- 172.21.15.133:0/1236651710 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247392637 0 0) 0x7f8e9806f140 con 0x7f8e980ce5f0 2024-03-07T06:51:29.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.893+0000 7f8e97e00640 1 --2- 172.21.15.133:0/1236651710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e980af950 0x7f8e980b3c10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009d90 tx=0x7f8e9002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a27ec7c5d906abf server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.893+0000 7f8e96000640 1 -- 172.21.15.133:0/1236651710 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90039070 con 0x7f8e980af950 2024-03-07T06:51:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.893+0000 7f8e96000640 1 -- 172.21.15.133:0/1236651710 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9002cc60 con 0x7f8e980af950 2024-03-07T06:51:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.893+0000 7f8e96000640 1 -- 172.21.15.133:0/1236651710 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90034660 con 0x7f8e980af950 2024-03-07T06:51:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.894+0000 7f8e9e600640 1 -- 172.21.15.133:0/1236651710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e980af950 msgr2=0x7f8e980b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.894+0000 7f8e9e600640 1 --2- 172.21.15.133:0/1236651710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e980af950 0x7f8e980b3c10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8e90009d90 tx=0x7f8e9002beb0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.894+0000 7f8e9e600640 1 -- 172.21.15.133:0/1236651710 shutdown_connections 2024-03-07T06:51:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.894+0000 7f8e9e600640 1 --2- 172.21.15.133:0/1236651710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e980af950 0x7f8e980b3c10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.894+0000 7f8e9e600640 1 -- 172.21.15.133:0/1236651710 >> 172.21.15.133:0/1236651710 conn(0x7f8e980b3120 msgr2=0x7f8e980b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.894+0000 7f8e9e600640 1 -- 172.21.15.133:0/1236651710 shutdown_connections 2024-03-07T06:51:29.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.894+0000 7f8e9e600640 1 -- 172.21.15.133:0/1236651710 wait complete. 2024-03-07T06:51:29.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.896+0000 7f8e9e600640 1 Processor -- start 2024-03-07T06:51:29.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.896+0000 7f8e9e600640 1 -- start start 2024-03-07T06:51:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e9e600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e980af950 0x7f8e9815e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e9e600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e980ce5f0 0x7f8e980b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e9e600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9815b1b0 0x7f8e98145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e9e600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9815b590 con 0x7f8e980ce5f0 2024-03-07T06:51:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e9e600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9806ee30 con 0x7f8e9815b1b0 2024-03-07T06:51:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e9e600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9815b710 con 0x7f8e980af950 2024-03-07T06:51:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e96a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e980ce5f0 0x7f8e980b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e96a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e980ce5f0 0x7f8e980b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38852/0 (socket says 172.21.15.133:38852) 2024-03-07T06:51:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e96a00640 1 -- 172.21.15.133:0/2688136196 learned_addr learned my addr 172.21.15.133:0/2688136196 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e96a00640 1 -- 172.21.15.133:0/2688136196 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e980af950 msgr2=0x7f8e9815e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e96a00640 1 --2- 172.21.15.133:0/2688136196 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e980af950 0x7f8e9815e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e96a00640 1 -- 172.21.15.133:0/2688136196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9815b1b0 msgr2=0x7f8e98145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e97e00640 1 --2- 172.21.15.133:0/2688136196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9815b1b0 0x7f8e98145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e96a00640 1 --2- 172.21.15.133:0/2688136196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9815b1b0 0x7f8e98145880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e96a00640 1 -- 172.21.15.133:0/2688136196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e980adeb0 con 0x7f8e980ce5f0 2024-03-07T06:51:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.897+0000 7f8e96a00640 1 --2- 172.21.15.133:0/2688136196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e980ce5f0 0x7f8e980b5200 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c490 tx=0x7f8e8400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.898+0000 7f8e8f400640 1 -- 172.21.15.133:0/2688136196 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e840074e0 con 0x7f8e980ce5f0 2024-03-07T06:51:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.898+0000 7f8e8f400640 1 -- 172.21.15.133:0/2688136196 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e840047e0 con 0x7f8e980ce5f0 2024-03-07T06:51:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.898+0000 7f8e8f400640 1 -- 172.21.15.133:0/2688136196 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e84004bd0 con 0x7f8e980ce5f0 2024-03-07T06:51:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.898+0000 7f8e9e600640 1 -- 172.21.15.133:0/2688136196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e980ae190 con 0x7f8e980ce5f0 2024-03-07T06:51:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.898+0000 7f8e9e600640 1 -- 172.21.15.133:0/2688136196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e980ae750 con 0x7f8e980ce5f0 2024-03-07T06:51:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.899+0000 7f8e9e600640 1 -- 172.21.15.133:0/2688136196 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e98062c90 con 0x7f8e980ce5f0 2024-03-07T06:51:29.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.900+0000 7f8e8f400640 1 -- 172.21.15.133:0/2688136196 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8e84011070 con 0x7f8e980ce5f0 2024-03-07T06:51:29.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.901+0000 7f8e8f400640 1 --2- 172.21.15.133:0/2688136196 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8e7806b4d0 0x7f8e7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.901+0000 7f8e8f400640 1 -- 172.21.15.133:0/2688136196 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e84089eb0 con 0x7f8e980ce5f0 2024-03-07T06:51:29.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.902+0000 7f8e97400640 1 --2- 172.21.15.133:0/2688136196 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8e7806b4d0 0x7f8e7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:29.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.902+0000 7f8e97400640 1 --2- 172.21.15.133:0/2688136196 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8e7806b4d0 0x7f8e7806d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8e88000f60 tx=0x7f8e880079b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:29.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:29.906+0000 7f8e8f400640 1 -- 172.21.15.133:0/2688136196 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8e8401a2b0 con 0x7f8e980ce5f0 2024-03-07T06:51:30.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.081+0000 7f8e9e600640 1 -- 172.21.15.133:0/2688136196 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e980cc660 con 0x7f8e7806b4d0 2024-03-07T06:51:30.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.100+0000 7f8e8f400640 1 -- 172.21.15.133:0/2688136196 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251837 (secure 0 0 0) 0x7f8e980cc660 con 0x7f8e7806b4d0 2024-03-07T06:51:30.103 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:30.104 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:51:30.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.104+0000 7f8e9e600640 1 -- 172.21.15.133:0/2688136196 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8e7806b4d0 msgr2=0x7f8e7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:30.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.104+0000 7f8e9e600640 1 --2- 172.21.15.133:0/2688136196 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8e7806b4d0 0x7f8e7806d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8e88000f60 tx=0x7f8e880079b0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.104+0000 7f8e9e600640 1 -- 172.21.15.133:0/2688136196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e980ce5f0 msgr2=0x7f8e980b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:30.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.104+0000 7f8e9e600640 1 --2- 172.21.15.133:0/2688136196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e980ce5f0 0x7f8e980b5200 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8e8400c490 tx=0x7f8e8400c950 comp rx=0 tx=0).stop 2024-03-07T06:51:30.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.105+0000 7f8e9e600640 1 -- 172.21.15.133:0/2688136196 shutdown_connections 2024-03-07T06:51:30.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.105+0000 7f8e9e600640 1 --2- 172.21.15.133:0/2688136196 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8e7806b4d0 0x7f8e7806d990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.105+0000 7f8e9e600640 1 --2- 172.21.15.133:0/2688136196 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8e9815b1b0 0x7f8e98145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.105+0000 7f8e9e600640 1 --2- 172.21.15.133:0/2688136196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e980ce5f0 0x7f8e980b5200 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.105+0000 7f8e9e600640 1 --2- 172.21.15.133:0/2688136196 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8e980af950 0x7f8e9815e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.105+0000 7f8e9e600640 1 -- 172.21.15.133:0/2688136196 >> 172.21.15.133:0/2688136196 conn(0x7f8e980b3120 msgr2=0x7f8e98158520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:30.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.105+0000 7f8e9e600640 1 -- 172.21.15.133:0/2688136196 shutdown_connections 2024-03-07T06:51:30.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.105+0000 7f8e9e600640 1 -- 172.21.15.133:0/2688136196 wait complete. 2024-03-07T06:51:30.122 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-03-07T06:51:28.474684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219088,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755632,"statfs":{"total":773094113280,"available":772869767168,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004705"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190674+0000","last_change":"2024-03-07T06:51:15.202253+0000","last_active":"2024-03-07T06:51:16.190674+0000","last_peered":"2024-03-07T06:51:16.190674+0000","last_clean":"2024-03-07T06:51:16.190674+0000","last_became_active":"2024-03-07T06:51:15.201848+0000","last_became_peered":"2024-03-07T06:51:15.201848+0000","last_unstale":"2024-03-07T06:51:16.190674+0000","last_undegraded":"2024-03-07T06:51:16.190674+0000","last_fullsized":"2024-03-07T06:51:16.190674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:41:54.645832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205980+0000","last_change":"2024-03-07T06:51:15.188301+0000","last_active":"2024-03-07T06:51:16.205980+0000","last_peered":"2024-03-07T06:51:16.205980+0000","last_clean":"2024-03-07T06:51:16.205980+0000","last_became_active":"2024-03-07T06:51:15.188021+0000","last_became_peered":"2024-03-07T06:51:15.188021+0000","last_unstale":"2024-03-07T06:51:16.205980+0000","last_undegraded":"2024-03-07T06:51:16.205980+0000","last_fullsized":"2024-03-07T06:51:16.205980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412687+0000","last_change":"2024-03-07T06:51:15.188296+0000","last_active":"2024-03-07T06:51:17.412687+0000","last_peered":"2024-03-07T06:51:17.412687+0000","last_clean":"2024-03-07T06:51:17.412687+0000","last_became_active":"2024-03-07T06:51:15.187954+0000","last_became_peered":"2024-03-07T06:51:15.187954+0000","last_unstale":"2024-03-07T06:51:17.412687+0000","last_undegraded":"2024-03-07T06:51:17.412687+0000","last_fullsized":"2024-03-07T06:51:17.412687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414882+0000","last_change":"2024-03-07T06:51:15.200894+0000","last_active":"2024-03-07T06:51:17.414882+0000","last_peered":"2024-03-07T06:51:17.414882+0000","last_clean":"2024-03-07T06:51:17.414882+0000","last_became_active":"2024-03-07T06:51:15.200617+0000","last_became_peered":"2024-03-07T06:51:15.200617+0000","last_unstale":"2024-03-07T06:51:17.414882+0000","last_undegraded":"2024-03-07T06:51:17.414882+0000","last_fullsized":"2024-03-07T06:51:17.414882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190864+0000","last_change":"2024-03-07T06:51:15.189383+0000","last_active":"2024-03-07T06:51:16.190864+0000","last_peered":"2024-03-07T06:51:16.190864+0000","last_clean":"2024-03-07T06:51:16.190864+0000","last_became_active":"2024-03-07T06:51:15.188192+0000","last_became_peered":"2024-03-07T06:51:15.188192+0000","last_unstale":"2024-03-07T06:51:16.190864+0000","last_undegraded":"2024-03-07T06:51:16.190864+0000","last_fullsized":"2024-03-07T06:51:16.190864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:07:16.800545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414731+0000","last_change":"2024-03-07T06:51:15.195882+0000","last_active":"2024-03-07T06:51:17.414731+0000","last_peered":"2024-03-07T06:51:17.414731+0000","last_clean":"2024-03-07T06:51:17.414731+0000","last_became_active":"2024-03-07T06:51:15.195572+0000","last_became_peered":"2024-03-07T06:51:15.195572+0000","last_unstale":"2024-03-07T06:51:17.414731+0000","last_undegraded":"2024-03-07T06:51:17.414731+0000","last_fullsized":"2024-03-07T06:51:17.414731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189735+0000","last_change":"2024-03-07T06:51:15.190861+0000","last_active":"2024-03-07T06:51:16.189735+0000","last_peered":"2024-03-07T06:51:16.189735+0000","last_clean":"2024-03-07T06:51:16.189735+0000","last_became_active":"2024-03-07T06:51:15.190546+0000","last_became_peered":"2024-03-07T06:51:15.190546+0000","last_unstale":"2024-03-07T06:51:16.189735+0000","last_undegraded":"2024-03-07T06:51:16.189735+0000","last_fullsized":"2024-03-07T06:51:16.189735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:44:23.301530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415109+0000","last_change":"2024-03-07T06:51:15.202655+0000","last_active":"2024-03-07T06:51:17.415109+0000","last_peered":"2024-03-07T06:51:17.415109+0000","last_clean":"2024-03-07T06:51:17.415109+0000","last_became_active":"2024-03-07T06:51:15.197943+0000","last_became_peered":"2024-03-07T06:51:15.197943+0000","last_unstale":"2024-03-07T06:51:17.415109+0000","last_undegraded":"2024-03-07T06:51:17.415109+0000","last_fullsized":"2024-03-07T06:51:17.415109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190628+0000","last_change":"2024-03-07T06:51:15.209333+0000","last_active":"2024-03-07T06:51:16.190628+0000","last_peered":"2024-03-07T06:51:16.190628+0000","last_clean":"2024-03-07T06:51:16.190628+0000","last_became_active":"2024-03-07T06:51:15.209034+0000","last_became_peered":"2024-03-07T06:51:15.209034+0000","last_unstale":"2024-03-07T06:51:16.190628+0000","last_undegraded":"2024-03-07T06:51:16.190628+0000","last_fullsized":"2024-03-07T06:51:16.190628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:31:37.292017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193770+0000","last_change":"2024-03-07T06:51:15.196320+0000","last_active":"2024-03-07T06:51:16.193770+0000","last_peered":"2024-03-07T06:51:16.193770+0000","last_clean":"2024-03-07T06:51:16.193770+0000","last_became_active":"2024-03-07T06:51:15.196029+0000","last_became_peered":"2024-03-07T06:51:15.196029+0000","last_unstale":"2024-03-07T06:51:16.193770+0000","last_undegraded":"2024-03-07T06:51:16.193770+0000","last_fullsized":"2024-03-07T06:51:16.193770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192627+0000","last_change":"2024-03-07T06:51:15.212042+0000","last_active":"2024-03-07T06:51:16.192627+0000","last_peered":"2024-03-07T06:51:16.192627+0000","last_clean":"2024-03-07T06:51:16.192627+0000","last_became_active":"2024-03-07T06:51:15.211737+0000","last_became_peered":"2024-03-07T06:51:15.211737+0000","last_unstale":"2024-03-07T06:51:16.192627+0000","last_undegraded":"2024-03-07T06:51:16.192627+0000","last_fullsized":"2024-03-07T06:51:16.192627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192585+0000","last_change":"2024-03-07T06:51:15.200183+0000","last_active":"2024-03-07T06:51:16.192585+0000","last_peered":"2024-03-07T06:51:16.192585+0000","last_clean":"2024-03-07T06:51:16.192585+0000","last_became_active":"2024-03-07T06:51:15.199322+0000","last_became_peered":"2024-03-07T06:51:15.199322+0000","last_unstale":"2024-03-07T06:51:16.192585+0000","last_undegraded":"2024-03-07T06:51:16.192585+0000","last_fullsized":"2024-03-07T06:51:16.192585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:11.295266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415528+0000","last_change":"2024-03-07T06:51:15.201141+0000","last_active":"2024-03-07T06:51:17.415528+0000","last_peered":"2024-03-07T06:51:17.415528+0000","last_clean":"2024-03-07T06:51:17.415528+0000","last_became_active":"2024-03-07T06:51:15.200965+0000","last_became_peered":"2024-03-07T06:51:15.200965+0000","last_unstale":"2024-03-07T06:51:17.415528+0000","last_undegraded":"2024-03-07T06:51:17.415528+0000","last_fullsized":"2024-03-07T06:51:17.415528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412986+0000","last_change":"2024-03-07T06:51:15.191043+0000","last_active":"2024-03-07T06:51:17.412986+0000","last_peered":"2024-03-07T06:51:17.412986+0000","last_clean":"2024-03-07T06:51:17.412986+0000","last_became_active":"2024-03-07T06:51:15.190777+0000","last_became_peered":"2024-03-07T06:51:15.190777+0000","last_unstale":"2024-03-07T06:51:17.412986+0000","last_undegraded":"2024-03-07T06:51:17.412986+0000","last_fullsized":"2024-03-07T06:51:17.412986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205186+0000","last_change":"2024-03-07T06:51:15.178908+0000","last_active":"2024-03-07T06:51:16.205186+0000","last_peered":"2024-03-07T06:51:16.205186+0000","last_clean":"2024-03-07T06:51:16.205186+0000","last_became_active":"2024-03-07T06:51:15.178387+0000","last_became_peered":"2024-03-07T06:51:15.178387+0000","last_unstale":"2024-03-07T06:51:16.205186+0000","last_undegraded":"2024-03-07T06:51:16.205186+0000","last_fullsized":"2024-03-07T06:51:16.205186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414392+0000","last_change":"2024-03-07T06:51:15.195007+0000","last_active":"2024-03-07T06:51:17.414392+0000","last_peered":"2024-03-07T06:51:17.414392+0000","last_clean":"2024-03-07T06:51:17.414392+0000","last_became_active":"2024-03-07T06:51:15.194624+0000","last_became_peered":"2024-03-07T06:51:15.194624+0000","last_unstale":"2024-03-07T06:51:17.414392+0000","last_undegraded":"2024-03-07T06:51:17.414392+0000","last_fullsized":"2024-03-07T06:51:17.414392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.416108+0000","last_change":"2024-03-07T06:51:15.206405+0000","last_active":"2024-03-07T06:51:17.416108+0000","last_peered":"2024-03-07T06:51:17.416108+0000","last_clean":"2024-03-07T06:51:17.416108+0000","last_became_active":"2024-03-07T06:51:15.205493+0000","last_became_peered":"2024-03-07T06:51:15.205493+0000","last_unstale":"2024-03-07T06:51:17.416108+0000","last_undegraded":"2024-03-07T06:51:17.416108+0000","last_fullsized":"2024-03-07T06:51:17.416108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:43:58.927044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413141+0000","last_change":"2024-03-07T06:51:15.195307+0000","last_active":"2024-03-07T06:51:17.413141+0000","last_peered":"2024-03-07T06:51:17.413141+0000","last_clean":"2024-03-07T06:51:17.413141+0000","last_became_active":"2024-03-07T06:51:15.194990+0000","last_became_peered":"2024-03-07T06:51:15.194990+0000","last_unstale":"2024-03-07T06:51:17.413141+0000","last_undegraded":"2024-03-07T06:51:17.413141+0000","last_fullsized":"2024-03-07T06:51:17.413141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:11.295266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189287+0000","last_change":"2024-03-07T06:51:15.197914+0000","last_active":"2024-03-07T06:51:16.189287+0000","last_peered":"2024-03-07T06:51:16.189287+0000","last_clean":"2024-03-07T06:51:16.189287+0000","last_became_active":"2024-03-07T06:51:15.196938+0000","last_became_peered":"2024-03-07T06:51:15.196938+0000","last_unstale":"2024-03-07T06:51:16.189287+0000","last_undegraded":"2024-03-07T06:51:16.189287+0000","last_fullsized":"2024-03-07T06:51:16.189287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:56.465649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192765+0000","last_change":"2024-03-07T06:51:15.199775+0000","last_active":"2024-03-07T06:51:16.192765+0000","last_peered":"2024-03-07T06:51:16.192765+0000","last_clean":"2024-03-07T06:51:16.192765+0000","last_became_active":"2024-03-07T06:51:15.199447+0000","last_became_peered":"2024-03-07T06:51:15.199447+0000","last_unstale":"2024-03-07T06:51:16.192765+0000","last_undegraded":"2024-03-07T06:51:16.192765+0000","last_fullsized":"2024-03-07T06:51:16.192765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413641+0000","last_change":"2024-03-07T06:51:15.191968+0000","last_active":"2024-03-07T06:51:17.413641+0000","last_peered":"2024-03-07T06:51:17.413641+0000","last_clean":"2024-03-07T06:51:17.413641+0000","last_became_active":"2024-03-07T06:51:15.191587+0000","last_became_peered":"2024-03-07T06:51:15.191587+0000","last_unstale":"2024-03-07T06:51:17.413641+0000","last_undegraded":"2024-03-07T06:51:17.413641+0000","last_fullsized":"2024-03-07T06:51:17.413641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415294+0000","last_change":"2024-03-07T06:51:15.206834+0000","last_active":"2024-03-07T06:51:17.415294+0000","last_peered":"2024-03-07T06:51:17.415294+0000","last_clean":"2024-03-07T06:51:17.415294+0000","last_became_active":"2024-03-07T06:51:15.198815+0000","last_became_peered":"2024-03-07T06:51:15.198815+0000","last_unstale":"2024-03-07T06:51:17.415294+0000","last_undegraded":"2024-03-07T06:51:17.415294+0000","last_fullsized":"2024-03-07T06:51:17.415294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415387+0000","last_change":"2024-03-07T06:51:15.203302+0000","last_active":"2024-03-07T06:51:17.415387+0000","last_peered":"2024-03-07T06:51:17.415387+0000","last_clean":"2024-03-07T06:51:17.415387+0000","last_became_active":"2024-03-07T06:51:15.203041+0000","last_became_peered":"2024-03-07T06:51:15.203041+0000","last_unstale":"2024-03-07T06:51:17.415387+0000","last_undegraded":"2024-03-07T06:51:17.415387+0000","last_fullsized":"2024-03-07T06:51:17.415387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414478+0000","last_change":"2024-03-07T06:51:15.195865+0000","last_active":"2024-03-07T06:51:17.414478+0000","last_peered":"2024-03-07T06:51:17.414478+0000","last_clean":"2024-03-07T06:51:17.414478+0000","last_became_active":"2024-03-07T06:51:15.194043+0000","last_became_peered":"2024-03-07T06:51:15.194043+0000","last_unstale":"2024-03-07T06:51:17.414478+0000","last_undegraded":"2024-03-07T06:51:17.414478+0000","last_fullsized":"2024-03-07T06:51:17.414478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415922+0000","last_change":"2024-03-07T06:51:15.203857+0000","last_active":"2024-03-07T06:51:17.415922+0000","last_peered":"2024-03-07T06:51:17.415922+0000","last_clean":"2024-03-07T06:51:17.415922+0000","last_became_active":"2024-03-07T06:51:15.203562+0000","last_became_peered":"2024-03-07T06:51:15.203562+0000","last_unstale":"2024-03-07T06:51:17.415922+0000","last_undegraded":"2024-03-07T06:51:17.415922+0000","last_fullsized":"2024-03-07T06:51:17.415922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:34:40.365080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193687+0000","last_change":"2024-03-07T06:51:15.210400+0000","last_active":"2024-03-07T06:51:16.193687+0000","last_peered":"2024-03-07T06:51:16.193687+0000","last_clean":"2024-03-07T06:51:16.193687+0000","last_became_active":"2024-03-07T06:51:15.209692+0000","last_became_peered":"2024-03-07T06:51:15.209692+0000","last_unstale":"2024-03-07T06:51:16.193687+0000","last_undegraded":"2024-03-07T06:51:16.193687+0000","last_fullsized":"2024-03-07T06:51:16.193687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205792+0000","last_change":"2024-03-07T06:51:15.190106+0000","last_active":"2024-03-07T06:51:16.205792+0000","last_peered":"2024-03-07T06:51:16.205792+0000","last_clean":"2024-03-07T06:51:16.205792+0000","last_became_active":"2024-03-07T06:51:15.189498+0000","last_became_peered":"2024-03-07T06:51:15.189498+0000","last_unstale":"2024-03-07T06:51:16.205792+0000","last_undegraded":"2024-03-07T06:51:16.205792+0000","last_fullsized":"2024-03-07T06:51:16.205792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415671+0000","last_change":"2024-03-07T06:51:15.203374+0000","last_active":"2024-03-07T06:51:17.415671+0000","last_peered":"2024-03-07T06:51:17.415671+0000","last_clean":"2024-03-07T06:51:17.415671+0000","last_became_active":"2024-03-07T06:51:15.202849+0000","last_became_peered":"2024-03-07T06:51:15.202849+0000","last_unstale":"2024-03-07T06:51:17.415671+0000","last_undegraded":"2024-03-07T06:51:17.415671+0000","last_fullsized":"2024-03-07T06:51:17.415671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413542+0000","last_change":"2024-03-07T06:51:15.200244+0000","last_active":"2024-03-07T06:51:17.413542+0000","last_peered":"2024-03-07T06:51:17.413542+0000","last_clean":"2024-03-07T06:51:17.413542+0000","last_became_active":"2024-03-07T06:51:15.198483+0000","last_became_peered":"2024-03-07T06:51:15.198483+0000","last_unstale":"2024-03-07T06:51:17.413542+0000","last_undegraded":"2024-03-07T06:51:17.413542+0000","last_fullsized":"2024-03-07T06:51:17.413542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415577+0000","last_change":"2024-03-07T06:51:15.195170+0000","last_active":"2024-03-07T06:51:17.415577+0000","last_peered":"2024-03-07T06:51:17.415577+0000","last_clean":"2024-03-07T06:51:17.415577+0000","last_became_active":"2024-03-07T06:51:15.194900+0000","last_became_peered":"2024-03-07T06:51:15.194900+0000","last_unstale":"2024-03-07T06:51:17.415577+0000","last_undegraded":"2024-03-07T06:51:17.415577+0000","last_fullsized":"2024-03-07T06:51:17.415577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:44:23.301530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414305+0000","last_change":"2024-03-07T06:51:15.197411+0000","last_active":"2024-03-07T06:51:17.414305+0000","last_peered":"2024-03-07T06:51:17.414305+0000","last_clean":"2024-03-07T06:51:17.414305+0000","last_became_active":"2024-03-07T06:51:15.194699+0000","last_became_peered":"2024-03-07T06:51:15.194699+0000","last_unstale":"2024-03-07T06:51:17.414305+0000","last_undegraded":"2024-03-07T06:51:17.414305+0000","last_fullsized":"2024-03-07T06:51:17.414305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:43.574428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.195220+0000","last_change":"2024-03-07T06:51:15.201331+0000","last_active":"2024-03-07T06:51:16.195220+0000","last_peered":"2024-03-07T06:51:16.195220+0000","last_clean":"2024-03-07T06:51:16.195220+0000","last_became_active":"2024-03-07T06:51:15.200758+0000","last_became_peered":"2024-03-07T06:51:15.200758+0000","last_unstale":"2024-03-07T06:51:16.195220+0000","last_undegraded":"2024-03-07T06:51:16.195220+0000","last_fullsized":"2024-03-07T06:51:16.195220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192348+0000","last_change":"2024-03-07T06:51:15.180977+0000","last_active":"2024-03-07T06:51:16.192348+0000","last_peered":"2024-03-07T06:51:16.192348+0000","last_clean":"2024-03-07T06:51:16.192348+0000","last_became_active":"2024-03-07T06:51:15.180359+0000","last_became_peered":"2024-03-07T06:51:15.180359+0000","last_unstale":"2024-03-07T06:51:16.192348+0000","last_undegraded":"2024-03-07T06:51:16.192348+0000","last_fullsized":"2024-03-07T06:51:16.192348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:10.776380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415597+0000","last_change":"2024-03-07T06:51:14.171537+0000","last_active":"2024-03-07T06:51:17.415597+0000","last_peered":"2024-03-07T06:51:17.415597+0000","last_clean":"2024-03-07T06:51:17.415597+0000","last_became_active":"2024-03-07T06:51:14.171105+0000","last_became_peered":"2024-03-07T06:51:14.171105+0000","last_unstale":"2024-03-07T06:51:17.415597+0000","last_undegraded":"2024-03-07T06:51:17.415597+0000","last_fullsized":"2024-03-07T06:51:17.415597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413743+0000","last_change":"2024-03-07T06:51:15.198729+0000","last_active":"2024-03-07T06:51:17.413743+0000","last_peered":"2024-03-07T06:51:17.413743+0000","last_clean":"2024-03-07T06:51:17.413743+0000","last_became_active":"2024-03-07T06:51:15.198412+0000","last_became_peered":"2024-03-07T06:51:15.198412+0000","last_unstale":"2024-03-07T06:51:17.413743+0000","last_undegraded":"2024-03-07T06:51:17.413743+0000","last_fullsized":"2024-03-07T06:51:17.413743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192432+0000","last_change":"2024-03-07T06:51:14.172967+0000","last_active":"2024-03-07T06:51:16.192432+0000","last_peered":"2024-03-07T06:51:16.192432+0000","last_clean":"2024-03-07T06:51:16.192432+0000","last_became_active":"2024-03-07T06:51:14.172706+0000","last_became_peered":"2024-03-07T06:51:14.172706+0000","last_unstale":"2024-03-07T06:51:16.192432+0000","last_undegraded":"2024-03-07T06:51:16.192432+0000","last_fullsized":"2024-03-07T06:51:16.192432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415155+0000","last_change":"2024-03-07T06:51:14.167651+0000","last_active":"2024-03-07T06:51:17.415155+0000","last_peered":"2024-03-07T06:51:17.415155+0000","last_clean":"2024-03-07T06:51:17.415155+0000","last_became_active":"2024-03-07T06:51:14.166984+0000","last_became_peered":"2024-03-07T06:51:14.166984+0000","last_unstale":"2024-03-07T06:51:17.415155+0000","last_undegraded":"2024-03-07T06:51:17.415155+0000","last_fullsized":"2024-03-07T06:51:17.415155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415067+0000","last_change":"2024-03-07T06:51:15.202906+0000","last_active":"2024-03-07T06:51:17.415067+0000","last_peered":"2024-03-07T06:51:17.415067+0000","last_clean":"2024-03-07T06:51:17.415067+0000","last_became_active":"2024-03-07T06:51:15.200219+0000","last_became_peered":"2024-03-07T06:51:15.200219+0000","last_unstale":"2024-03-07T06:51:17.415067+0000","last_undegraded":"2024-03-07T06:51:17.415067+0000","last_fullsized":"2024-03-07T06:51:17.415067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414528+0000","last_change":"2024-03-07T06:51:14.175653+0000","last_active":"2024-03-07T06:51:17.414528+0000","last_peered":"2024-03-07T06:51:17.414528+0000","last_clean":"2024-03-07T06:51:17.414528+0000","last_became_active":"2024-03-07T06:51:14.175295+0000","last_became_peered":"2024-03-07T06:51:14.175295+0000","last_unstale":"2024-03-07T06:51:17.414528+0000","last_undegraded":"2024-03-07T06:51:17.414528+0000","last_fullsized":"2024-03-07T06:51:17.414528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205033+0000","last_change":"2024-03-07T06:51:15.190580+0000","last_active":"2024-03-07T06:51:16.205033+0000","last_peered":"2024-03-07T06:51:16.205033+0000","last_clean":"2024-03-07T06:51:16.205033+0000","last_became_active":"2024-03-07T06:51:15.189190+0000","last_became_peered":"2024-03-07T06:51:15.189190+0000","last_unstale":"2024-03-07T06:51:16.205033+0000","last_undegraded":"2024-03-07T06:51:16.205033+0000","last_fullsized":"2024-03-07T06:51:16.205033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193565+0000","last_change":"2024-03-07T06:51:14.173746+0000","last_active":"2024-03-07T06:51:16.193565+0000","last_peered":"2024-03-07T06:51:16.193565+0000","last_clean":"2024-03-07T06:51:16.193565+0000","last_became_active":"2024-03-07T06:51:14.173285+0000","last_became_peered":"2024-03-07T06:51:14.173285+0000","last_unstale":"2024-03-07T06:51:16.193565+0000","last_undegraded":"2024-03-07T06:51:16.193565+0000","last_fullsized":"2024-03-07T06:51:16.193565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415198+0000","last_change":"2024-03-07T06:51:15.215525+0000","last_active":"2024-03-07T06:51:17.415198+0000","last_peered":"2024-03-07T06:51:17.415198+0000","last_clean":"2024-03-07T06:51:17.415198+0000","last_became_active":"2024-03-07T06:51:15.215050+0000","last_became_peered":"2024-03-07T06:51:15.215050+0000","last_unstale":"2024-03-07T06:51:17.415198+0000","last_undegraded":"2024-03-07T06:51:17.415198+0000","last_fullsized":"2024-03-07T06:51:17.415198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:22:38.973224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414959+0000","last_change":"2024-03-07T06:51:14.168633+0000","last_active":"2024-03-07T06:51:17.414959+0000","last_peered":"2024-03-07T06:51:17.414959+0000","last_clean":"2024-03-07T06:51:17.414959+0000","last_became_active":"2024-03-07T06:51:14.168293+0000","last_became_peered":"2024-03-07T06:51:14.168293+0000","last_unstale":"2024-03-07T06:51:17.414959+0000","last_undegraded":"2024-03-07T06:51:17.414959+0000","last_fullsized":"2024-03-07T06:51:17.414959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412761+0000","last_change":"2024-03-07T06:51:15.208732+0000","last_active":"2024-03-07T06:51:17.412761+0000","last_peered":"2024-03-07T06:51:17.412761+0000","last_clean":"2024-03-07T06:51:17.412761+0000","last_became_active":"2024-03-07T06:51:15.208472+0000","last_became_peered":"2024-03-07T06:51:15.208472+0000","last_unstale":"2024-03-07T06:51:17.412761+0000","last_undegraded":"2024-03-07T06:51:17.412761+0000","last_fullsized":"2024-03-07T06:51:17.412761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205600+0000","last_change":"2024-03-07T06:51:14.171631+0000","last_active":"2024-03-07T06:51:16.205600+0000","last_peered":"2024-03-07T06:51:16.205600+0000","last_clean":"2024-03-07T06:51:16.205600+0000","last_became_active":"2024-03-07T06:51:14.171058+0000","last_became_peered":"2024-03-07T06:51:14.171058+0000","last_unstale":"2024-03-07T06:51:16.205600+0000","last_undegraded":"2024-03-07T06:51:16.205600+0000","last_fullsized":"2024-03-07T06:51:16.205600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193446+0000","last_change":"2024-03-07T06:51:15.195194+0000","last_active":"2024-03-07T06:51:16.193446+0000","last_peered":"2024-03-07T06:51:16.193446+0000","last_clean":"2024-03-07T06:51:16.193446+0000","last_became_active":"2024-03-07T06:51:15.191103+0000","last_became_peered":"2024-03-07T06:51:15.191103+0000","last_unstale":"2024-03-07T06:51:16.193446+0000","last_undegraded":"2024-03-07T06:51:16.193446+0000","last_fullsized":"2024-03-07T06:51:16.193446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:40:51.935140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192876+0000","last_change":"2024-03-07T06:51:14.166640+0000","last_active":"2024-03-07T06:51:16.192876+0000","last_peered":"2024-03-07T06:51:16.192876+0000","last_clean":"2024-03-07T06:51:16.192876+0000","last_became_active":"2024-03-07T06:51:14.166329+0000","last_became_peered":"2024-03-07T06:51:14.166329+0000","last_unstale":"2024-03-07T06:51:16.192876+0000","last_undegraded":"2024-03-07T06:51:16.192876+0000","last_fullsized":"2024-03-07T06:51:16.192876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188809+0000","last_change":"2024-03-07T06:51:15.193638+0000","last_active":"2024-03-07T06:51:16.188809+0000","last_peered":"2024-03-07T06:51:16.188809+0000","last_clean":"2024-03-07T06:51:16.188809+0000","last_became_active":"2024-03-07T06:51:15.193354+0000","last_became_peered":"2024-03-07T06:51:15.193354+0000","last_unstale":"2024-03-07T06:51:16.188809+0000","last_undegraded":"2024-03-07T06:51:16.188809+0000","last_fullsized":"2024-03-07T06:51:16.188809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413807+0000","last_change":"2024-03-07T06:51:14.167158+0000","last_active":"2024-03-07T06:51:17.413807+0000","last_peered":"2024-03-07T06:51:17.413807+0000","last_clean":"2024-03-07T06:51:17.413807+0000","last_became_active":"2024-03-07T06:51:14.166835+0000","last_became_peered":"2024-03-07T06:51:14.166835+0000","last_unstale":"2024-03-07T06:51:17.413807+0000","last_undegraded":"2024-03-07T06:51:17.413807+0000","last_fullsized":"2024-03-07T06:51:17.413807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:38:27.101103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413891+0000","last_change":"2024-03-07T06:51:15.199375+0000","last_active":"2024-03-07T06:51:17.413891+0000","last_peered":"2024-03-07T06:51:17.413891+0000","last_clean":"2024-03-07T06:51:17.413891+0000","last_became_active":"2024-03-07T06:51:15.199060+0000","last_became_peered":"2024-03-07T06:51:15.199060+0000","last_unstale":"2024-03-07T06:51:17.413891+0000","last_undegraded":"2024-03-07T06:51:17.413891+0000","last_fullsized":"2024-03-07T06:51:17.413891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415423+0000","last_change":"2024-03-07T06:51:14.170514+0000","last_active":"2024-03-07T06:51:17.415423+0000","last_peered":"2024-03-07T06:51:17.415423+0000","last_clean":"2024-03-07T06:51:17.415423+0000","last_became_active":"2024-03-07T06:51:14.169542+0000","last_became_peered":"2024-03-07T06:51:14.169542+0000","last_unstale":"2024-03-07T06:51:17.415423+0000","last_undegraded":"2024-03-07T06:51:17.415423+0000","last_fullsized":"2024-03-07T06:51:17.415423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191234+0000","last_change":"2024-03-07T06:51:15.208081+0000","last_active":"2024-03-07T06:51:16.191234+0000","last_peered":"2024-03-07T06:51:16.191234+0000","last_clean":"2024-03-07T06:51:16.191234+0000","last_became_active":"2024-03-07T06:51:15.207758+0000","last_became_peered":"2024-03-07T06:51:15.207758+0000","last_unstale":"2024-03-07T06:51:16.191234+0000","last_undegraded":"2024-03-07T06:51:16.191234+0000","last_fullsized":"2024-03-07T06:51:16.191234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:43:16.081728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191798+0000","last_change":"2024-03-07T06:51:14.168470+0000","last_active":"2024-03-07T06:51:16.191798+0000","last_peered":"2024-03-07T06:51:16.191798+0000","last_clean":"2024-03-07T06:51:16.191798+0000","last_became_active":"2024-03-07T06:51:14.168190+0000","last_became_peered":"2024-03-07T06:51:14.168190+0000","last_unstale":"2024-03-07T06:51:16.191798+0000","last_undegraded":"2024-03-07T06:51:16.191798+0000","last_fullsized":"2024-03-07T06:51:16.191798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192778+0000","last_change":"2024-03-07T06:51:15.201105+0000","last_active":"2024-03-07T06:51:16.192778+0000","last_peered":"2024-03-07T06:51:16.192778+0000","last_clean":"2024-03-07T06:51:16.192778+0000","last_became_active":"2024-03-07T06:51:15.200404+0000","last_became_peered":"2024-03-07T06:51:15.200404+0000","last_unstale":"2024-03-07T06:51:16.192778+0000","last_undegraded":"2024-03-07T06:51:16.192778+0000","last_fullsized":"2024-03-07T06:51:16.192778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412017+0000","last_change":"2024-03-07T06:51:14.154100+0000","last_active":"2024-03-07T06:51:17.412017+0000","last_peered":"2024-03-07T06:51:17.412017+0000","last_clean":"2024-03-07T06:51:17.412017+0000","last_became_active":"2024-03-07T06:51:14.152648+0000","last_became_peered":"2024-03-07T06:51:14.152648+0000","last_unstale":"2024-03-07T06:51:17.412017+0000","last_undegraded":"2024-03-07T06:51:17.412017+0000","last_fullsized":"2024-03-07T06:51:17.412017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189886+0000","last_change":"2024-03-07T06:51:15.203942+0000","last_active":"2024-03-07T06:51:16.189886+0000","last_peered":"2024-03-07T06:51:16.189886+0000","last_clean":"2024-03-07T06:51:16.189886+0000","last_became_active":"2024-03-07T06:51:15.203671+0000","last_became_peered":"2024-03-07T06:51:15.203671+0000","last_unstale":"2024-03-07T06:51:16.189886+0000","last_undegraded":"2024-03-07T06:51:16.189886+0000","last_fullsized":"2024-03-07T06:51:16.189886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:02:55.966834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193630+0000","last_change":"2024-03-07T06:51:14.164166+0000","last_active":"2024-03-07T06:51:16.193630+0000","last_peered":"2024-03-07T06:51:16.193630+0000","last_clean":"2024-03-07T06:51:16.193630+0000","last_became_active":"2024-03-07T06:51:14.163821+0000","last_became_peered":"2024-03-07T06:51:14.163821+0000","last_unstale":"2024-03-07T06:51:16.193630+0000","last_undegraded":"2024-03-07T06:51:16.193630+0000","last_fullsized":"2024-03-07T06:51:16.193630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:38:27.101103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190408+0000","last_change":"2024-03-07T06:51:15.207392+0000","last_active":"2024-03-07T06:51:16.190408+0000","last_peered":"2024-03-07T06:51:16.190408+0000","last_clean":"2024-03-07T06:51:16.190408+0000","last_became_active":"2024-03-07T06:51:15.203049+0000","last_became_peered":"2024-03-07T06:51:15.203049+0000","last_unstale":"2024-03-07T06:51:16.190408+0000","last_undegraded":"2024-03-07T06:51:16.190408+0000","last_fullsized":"2024-03-07T06:51:16.190408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:43:58.927044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413191+0000","last_change":"2024-03-07T06:51:14.162546+0000","last_active":"2024-03-07T06:51:17.413191+0000","last_peered":"2024-03-07T06:51:17.413191+0000","last_clean":"2024-03-07T06:51:17.413191+0000","last_became_active":"2024-03-07T06:51:14.161082+0000","last_became_peered":"2024-03-07T06:51:14.161082+0000","last_unstale":"2024-03-07T06:51:17.413191+0000","last_undegraded":"2024-03-07T06:51:17.413191+0000","last_fullsized":"2024-03-07T06:51:17.413191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190725+0000","last_change":"2024-03-07T06:51:15.208036+0000","last_active":"2024-03-07T06:51:16.190725+0000","last_peered":"2024-03-07T06:51:16.190725+0000","last_clean":"2024-03-07T06:51:16.190725+0000","last_became_active":"2024-03-07T06:51:15.206837+0000","last_became_peered":"2024-03-07T06:51:15.206837+0000","last_unstale":"2024-03-07T06:51:16.190725+0000","last_undegraded":"2024-03-07T06:51:16.190725+0000","last_fullsized":"2024-03-07T06:51:16.190725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:55:33.816984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205890+0000","last_change":"2024-03-07T06:51:14.168546+0000","last_active":"2024-03-07T06:51:16.205890+0000","last_peered":"2024-03-07T06:51:16.205890+0000","last_clean":"2024-03-07T06:51:16.205890+0000","last_became_active":"2024-03-07T06:51:14.168238+0000","last_became_peered":"2024-03-07T06:51:14.168238+0000","last_unstale":"2024-03-07T06:51:16.205890+0000","last_undegraded":"2024-03-07T06:51:16.205890+0000","last_fullsized":"2024-03-07T06:51:16.205890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415347+0000","last_change":"2024-03-07T06:51:15.204629+0000","last_active":"2024-03-07T06:51:17.415347+0000","last_peered":"2024-03-07T06:51:17.415347+0000","last_clean":"2024-03-07T06:51:17.415347+0000","last_became_active":"2024-03-07T06:51:15.203615+0000","last_became_peered":"2024-03-07T06:51:15.203615+0000","last_unstale":"2024-03-07T06:51:17.415347+0000","last_undegraded":"2024-03-07T06:51:17.415347+0000","last_fullsized":"2024-03-07T06:51:17.415347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:38:28.416311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412767+0000","last_change":"2024-03-07T06:51:14.170179+0000","last_active":"2024-03-07T06:51:17.412767+0000","last_peered":"2024-03-07T06:51:17.412767+0000","last_clean":"2024-03-07T06:51:17.412767+0000","last_became_active":"2024-03-07T06:51:14.169797+0000","last_became_peered":"2024-03-07T06:51:14.169797+0000","last_unstale":"2024-03-07T06:51:17.412767+0000","last_undegraded":"2024-03-07T06:51:17.412767+0000","last_fullsized":"2024-03-07T06:51:17.412767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188955+0000","last_change":"2024-03-07T06:51:15.212961+0000","last_active":"2024-03-07T06:51:16.188955+0000","last_peered":"2024-03-07T06:51:16.188955+0000","last_clean":"2024-03-07T06:51:16.188955+0000","last_became_active":"2024-03-07T06:51:15.212691+0000","last_became_peered":"2024-03-07T06:51:15.212691+0000","last_unstale":"2024-03-07T06:51:16.188955+0000","last_undegraded":"2024-03-07T06:51:16.188955+0000","last_fullsized":"2024-03-07T06:51:16.188955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:38:28.416311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188951+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:51:16.188951+0000","last_peered":"2024-03-07T06:51:16.188951+0000","last_clean":"2024-03-07T06:51:16.188951+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:51:16.188951+0000","last_undegraded":"2024-03-07T06:51:16.188951+0000","last_fullsized":"2024-03-07T06:51:16.188951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414355+0000","last_change":"2024-03-07T06:51:14.172051+0000","last_active":"2024-03-07T06:51:17.414355+0000","last_peered":"2024-03-07T06:51:17.414355+0000","last_clean":"2024-03-07T06:51:17.414355+0000","last_became_active":"2024-03-07T06:51:14.170736+0000","last_became_peered":"2024-03-07T06:51:14.170736+0000","last_unstale":"2024-03-07T06:51:17.414355+0000","last_undegraded":"2024-03-07T06:51:17.414355+0000","last_fullsized":"2024-03-07T06:51:17.414355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415537+0000","last_change":"2024-03-07T06:51:15.208661+0000","last_active":"2024-03-07T06:51:17.415537+0000","last_peered":"2024-03-07T06:51:17.415537+0000","last_clean":"2024-03-07T06:51:17.415537+0000","last_became_active":"2024-03-07T06:51:15.208394+0000","last_became_peered":"2024-03-07T06:51:15.208394+0000","last_unstale":"2024-03-07T06:51:17.415537+0000","last_undegraded":"2024-03-07T06:51:17.415537+0000","last_fullsized":"2024-03-07T06:51:17.415537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188640+0000","last_change":"2024-03-07T06:51:14.163185+0000","last_active":"2024-03-07T06:51:16.188640+0000","last_peered":"2024-03-07T06:51:16.188640+0000","last_clean":"2024-03-07T06:51:16.188640+0000","last_became_active":"2024-03-07T06:51:14.162494+0000","last_became_peered":"2024-03-07T06:51:14.162494+0000","last_unstale":"2024-03-07T06:51:16.188640+0000","last_undegraded":"2024-03-07T06:51:16.188640+0000","last_fullsized":"2024-03-07T06:51:16.188640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412672+0000","last_change":"2024-03-07T06:51:15.193638+0000","last_active":"2024-03-07T06:51:17.412672+0000","last_peered":"2024-03-07T06:51:17.412672+0000","last_clean":"2024-03-07T06:51:17.412672+0000","last_became_active":"2024-03-07T06:51:15.193379+0000","last_became_peered":"2024-03-07T06:51:15.193379+0000","last_unstale":"2024-03-07T06:51:17.412672+0000","last_undegraded":"2024-03-07T06:51:17.412672+0000","last_fullsized":"2024-03-07T06:51:17.412672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:16:10.967500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412927+0000","last_change":"2024-03-07T06:51:14.163793+0000","last_active":"2024-03-07T06:51:17.412927+0000","last_peered":"2024-03-07T06:51:17.412927+0000","last_clean":"2024-03-07T06:51:17.412927+0000","last_became_active":"2024-03-07T06:51:14.162059+0000","last_became_peered":"2024-03-07T06:51:14.162059+0000","last_unstale":"2024-03-07T06:51:17.412927+0000","last_undegraded":"2024-03-07T06:51:17.412927+0000","last_fullsized":"2024-03-07T06:51:17.412927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205257+0000","last_change":"2024-03-07T06:51:15.187522+0000","last_active":"2024-03-07T06:51:16.205257+0000","last_peered":"2024-03-07T06:51:16.205257+0000","last_clean":"2024-03-07T06:51:16.205257+0000","last_became_active":"2024-03-07T06:51:15.179882+0000","last_became_peered":"2024-03-07T06:51:15.179882+0000","last_unstale":"2024-03-07T06:51:16.205257+0000","last_undegraded":"2024-03-07T06:51:16.205257+0000","last_fullsized":"2024-03-07T06:51:16.205257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415629+0000","last_change":"2024-03-07T06:51:14.164766+0000","last_active":"2024-03-07T06:51:17.415629+0000","last_peered":"2024-03-07T06:51:17.415629+0000","last_clean":"2024-03-07T06:51:17.415629+0000","last_became_active":"2024-03-07T06:51:14.164397+0000","last_became_peered":"2024-03-07T06:51:14.164397+0000","last_unstale":"2024-03-07T06:51:17.415629+0000","last_undegraded":"2024-03-07T06:51:17.415629+0000","last_fullsized":"2024-03-07T06:51:17.415629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192589+0000","last_change":"2024-03-07T06:51:15.190608+0000","last_active":"2024-03-07T06:51:16.192589+0000","last_peered":"2024-03-07T06:51:16.192589+0000","last_clean":"2024-03-07T06:51:16.192589+0000","last_became_active":"2024-03-07T06:51:15.190326+0000","last_became_peered":"2024-03-07T06:51:15.190326+0000","last_unstale":"2024-03-07T06:51:16.192589+0000","last_undegraded":"2024-03-07T06:51:16.192589+0000","last_fullsized":"2024-03-07T06:51:16.192589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:43.574428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205896+0000","last_change":"2024-03-07T06:51:14.164402+0000","last_active":"2024-03-07T06:51:16.205896+0000","last_peered":"2024-03-07T06:51:16.205896+0000","last_clean":"2024-03-07T06:51:16.205896+0000","last_became_active":"2024-03-07T06:51:14.164081+0000","last_became_peered":"2024-03-07T06:51:14.164081+0000","last_unstale":"2024-03-07T06:51:16.205896+0000","last_undegraded":"2024-03-07T06:51:16.205896+0000","last_fullsized":"2024-03-07T06:51:16.205896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415903+0000","last_change":"2024-03-07T06:51:15.208063+0000","last_active":"2024-03-07T06:51:17.415903+0000","last_peered":"2024-03-07T06:51:17.415903+0000","last_clean":"2024-03-07T06:51:17.415903+0000","last_became_active":"2024-03-07T06:51:15.207782+0000","last_became_peered":"2024-03-07T06:51:15.207782+0000","last_unstale":"2024-03-07T06:51:17.415903+0000","last_undegraded":"2024-03-07T06:51:17.415903+0000","last_fullsized":"2024-03-07T06:51:17.415903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:02:55.966834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189333+0000","last_change":"2024-03-07T06:51:14.156560+0000","last_active":"2024-03-07T06:51:16.189333+0000","last_peered":"2024-03-07T06:51:16.189333+0000","last_clean":"2024-03-07T06:51:16.189333+0000","last_became_active":"2024-03-07T06:51:14.156040+0000","last_became_peered":"2024-03-07T06:51:14.156040+0000","last_unstale":"2024-03-07T06:51:16.189333+0000","last_undegraded":"2024-03-07T06:51:16.189333+0000","last_fullsized":"2024-03-07T06:51:16.189333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:07:44.222048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205750+0000","last_change":"2024-03-07T06:51:15.180806+0000","last_active":"2024-03-07T06:51:16.205750+0000","last_peered":"2024-03-07T06:51:16.205750+0000","last_clean":"2024-03-07T06:51:16.205750+0000","last_became_active":"2024-03-07T06:51:15.180402+0000","last_became_peered":"2024-03-07T06:51:15.180402+0000","last_unstale":"2024-03-07T06:51:16.205750+0000","last_undegraded":"2024-03-07T06:51:16.205750+0000","last_fullsized":"2024-03-07T06:51:16.205750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205976+0000","last_change":"2024-03-07T06:51:14.168110+0000","last_active":"2024-03-07T06:51:16.205976+0000","last_peered":"2024-03-07T06:51:16.205976+0000","last_clean":"2024-03-07T06:51:16.205976+0000","last_became_active":"2024-03-07T06:51:14.167759+0000","last_became_peered":"2024-03-07T06:51:14.167759+0000","last_unstale":"2024-03-07T06:51:16.205976+0000","last_undegraded":"2024-03-07T06:51:16.205976+0000","last_fullsized":"2024-03-07T06:51:16.205976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.416085+0000","last_change":"2024-03-07T06:51:15.199832+0000","last_active":"2024-03-07T06:51:17.416085+0000","last_peered":"2024-03-07T06:51:17.416085+0000","last_clean":"2024-03-07T06:51:17.416085+0000","last_became_active":"2024-03-07T06:51:15.199572+0000","last_became_peered":"2024-03-07T06:51:15.199572+0000","last_unstale":"2024-03-07T06:51:17.416085+0000","last_undegraded":"2024-03-07T06:51:17.416085+0000","last_fullsized":"2024-03-07T06:51:17.416085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193650+0000","last_change":"2024-03-07T06:51:14.173549+0000","last_active":"2024-03-07T06:51:16.193650+0000","last_peered":"2024-03-07T06:51:16.193650+0000","last_clean":"2024-03-07T06:51:16.193650+0000","last_became_active":"2024-03-07T06:51:14.173251+0000","last_became_peered":"2024-03-07T06:51:14.173251+0000","last_unstale":"2024-03-07T06:51:16.193650+0000","last_undegraded":"2024-03-07T06:51:16.193650+0000","last_fullsized":"2024-03-07T06:51:16.193650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413748+0000","last_change":"2024-03-07T06:51:15.195178+0000","last_active":"2024-03-07T06:51:17.413748+0000","last_peered":"2024-03-07T06:51:17.413748+0000","last_clean":"2024-03-07T06:51:17.413748+0000","last_became_active":"2024-03-07T06:51:15.194907+0000","last_became_peered":"2024-03-07T06:51:15.194907+0000","last_unstale":"2024-03-07T06:51:17.413748+0000","last_undegraded":"2024-03-07T06:51:17.413748+0000","last_fullsized":"2024-03-07T06:51:17.413748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188590+0000","last_change":"2024-03-07T06:51:14.156566+0000","last_active":"2024-03-07T06:51:16.188590+0000","last_peered":"2024-03-07T06:51:16.188590+0000","last_clean":"2024-03-07T06:51:16.188590+0000","last_became_active":"2024-03-07T06:51:14.156261+0000","last_became_peered":"2024-03-07T06:51:14.156261+0000","last_unstale":"2024-03-07T06:51:16.188590+0000","last_undegraded":"2024-03-07T06:51:16.188590+0000","last_fullsized":"2024-03-07T06:51:16.188590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193424+0000","last_change":"2024-03-07T06:51:15.183065+0000","last_active":"2024-03-07T06:51:16.193424+0000","last_peered":"2024-03-07T06:51:16.193424+0000","last_clean":"2024-03-07T06:51:16.193424+0000","last_became_active":"2024-03-07T06:51:15.182189+0000","last_became_peered":"2024-03-07T06:51:15.182189+0000","last_unstale":"2024-03-07T06:51:16.193424+0000","last_undegraded":"2024-03-07T06:51:16.193424+0000","last_fullsized":"2024-03-07T06:51:16.193424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:05:21.883987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205102+0000","last_change":"2024-03-07T06:51:14.169705+0000","last_active":"2024-03-07T06:51:16.205102+0000","last_peered":"2024-03-07T06:51:16.205102+0000","last_clean":"2024-03-07T06:51:16.205102+0000","last_became_active":"2024-03-07T06:51:14.169444+0000","last_became_peered":"2024-03-07T06:51:14.169444+0000","last_unstale":"2024-03-07T06:51:16.205102+0000","last_undegraded":"2024-03-07T06:51:16.205102+0000","last_fullsized":"2024-03-07T06:51:16.205102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415220+0000","last_change":"2024-03-07T06:51:15.197549+0000","last_active":"2024-03-07T06:51:17.415220+0000","last_peered":"2024-03-07T06:51:17.415220+0000","last_clean":"2024-03-07T06:51:17.415220+0000","last_became_active":"2024-03-07T06:51:15.195020+0000","last_became_peered":"2024-03-07T06:51:15.195020+0000","last_unstale":"2024-03-07T06:51:17.415220+0000","last_undegraded":"2024-03-07T06:51:17.415220+0000","last_fullsized":"2024-03-07T06:51:17.415220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205415+0000","last_change":"2024-03-07T06:51:14.170410+0000","last_active":"2024-03-07T06:51:16.205415+0000","last_peered":"2024-03-07T06:51:16.205415+0000","last_clean":"2024-03-07T06:51:16.205415+0000","last_became_active":"2024-03-07T06:51:14.170043+0000","last_became_peered":"2024-03-07T06:51:14.170043+0000","last_unstale":"2024-03-07T06:51:16.205415+0000","last_undegraded":"2024-03-07T06:51:16.205415+0000","last_fullsized":"2024-03-07T06:51:16.205415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415636+0000","last_change":"2024-03-07T06:51:15.193076+0000","last_active":"2024-03-07T06:51:17.415636+0000","last_peered":"2024-03-07T06:51:17.415636+0000","last_clean":"2024-03-07T06:51:17.415636+0000","last_became_active":"2024-03-07T06:51:15.192700+0000","last_became_peered":"2024-03-07T06:51:15.192700+0000","last_unstale":"2024-03-07T06:51:17.415636+0000","last_undegraded":"2024-03-07T06:51:17.415636+0000","last_fullsized":"2024-03-07T06:51:17.415636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191169+0000","last_change":"2024-03-07T06:51:14.173220+0000","last_active":"2024-03-07T06:51:16.191169+0000","last_peered":"2024-03-07T06:51:16.191169+0000","last_clean":"2024-03-07T06:51:16.191169+0000","last_became_active":"2024-03-07T06:51:14.169687+0000","last_became_peered":"2024-03-07T06:51:14.169687+0000","last_unstale":"2024-03-07T06:51:16.191169+0000","last_undegraded":"2024-03-07T06:51:16.191169+0000","last_fullsized":"2024-03-07T06:51:16.191169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:02:40.568300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192059+0000","last_change":"2024-03-07T06:51:15.205150+0000","last_active":"2024-03-07T06:51:16.192059+0000","last_peered":"2024-03-07T06:51:16.192059+0000","last_clean":"2024-03-07T06:51:16.192059+0000","last_became_active":"2024-03-07T06:51:15.204435+0000","last_became_peered":"2024-03-07T06:51:15.204435+0000","last_unstale":"2024-03-07T06:51:16.192059+0000","last_undegraded":"2024-03-07T06:51:16.192059+0000","last_fullsized":"2024-03-07T06:51:16.192059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415532+0000","last_change":"2024-03-07T06:51:15.214214+0000","last_active":"2024-03-07T06:51:17.415532+0000","last_peered":"2024-03-07T06:51:17.415532+0000","last_clean":"2024-03-07T06:51:17.415532+0000","last_became_active":"2024-03-07T06:51:15.209505+0000","last_became_peered":"2024-03-07T06:51:15.209505+0000","last_unstale":"2024-03-07T06:51:17.415532+0000","last_undegraded":"2024-03-07T06:51:17.415532+0000","last_fullsized":"2024-03-07T06:51:17.415532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:56.465649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188962+0000","last_change":"2024-03-07T06:51:14.172650+0000","last_active":"2024-03-07T06:51:16.188962+0000","last_peered":"2024-03-07T06:51:16.188962+0000","last_clean":"2024-03-07T06:51:16.188962+0000","last_became_active":"2024-03-07T06:51:14.170441+0000","last_became_peered":"2024-03-07T06:51:14.170441+0000","last_unstale":"2024-03-07T06:51:16.188962+0000","last_undegraded":"2024-03-07T06:51:16.188962+0000","last_fullsized":"2024-03-07T06:51:16.188962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:05:20.871811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415022+0000","last_change":"2024-03-07T06:51:15.204124+0000","last_active":"2024-03-07T06:51:17.415022+0000","last_peered":"2024-03-07T06:51:17.415022+0000","last_clean":"2024-03-07T06:51:17.415022+0000","last_became_active":"2024-03-07T06:51:15.203713+0000","last_became_peered":"2024-03-07T06:51:15.203713+0000","last_unstale":"2024-03-07T06:51:17.415022+0000","last_undegraded":"2024-03-07T06:51:17.415022+0000","last_fullsized":"2024-03-07T06:51:17.415022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:34:40.365080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189015+0000","last_change":"2024-03-07T06:51:14.167422+0000","last_active":"2024-03-07T06:51:16.189015+0000","last_peered":"2024-03-07T06:51:16.189015+0000","last_clean":"2024-03-07T06:51:16.189015+0000","last_became_active":"2024-03-07T06:51:14.167152+0000","last_became_peered":"2024-03-07T06:51:14.167152+0000","last_unstale":"2024-03-07T06:51:16.189015+0000","last_undegraded":"2024-03-07T06:51:16.189015+0000","last_fullsized":"2024-03-07T06:51:16.189015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:40:50.922964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412609+0000","last_change":"2024-03-07T06:51:15.210848+0000","last_active":"2024-03-07T06:51:17.412609+0000","last_peered":"2024-03-07T06:51:17.412609+0000","last_clean":"2024-03-07T06:51:17.412609+0000","last_became_active":"2024-03-07T06:51:15.210526+0000","last_became_peered":"2024-03-07T06:51:15.210526+0000","last_unstale":"2024-03-07T06:51:17.412609+0000","last_undegraded":"2024-03-07T06:51:17.412609+0000","last_fullsized":"2024-03-07T06:51:17.412609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:40:57.934235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415818+0000","last_change":"2024-03-07T06:51:14.163311+0000","last_active":"2024-03-07T06:51:17.415818+0000","last_peered":"2024-03-07T06:51:17.415818+0000","last_clean":"2024-03-07T06:51:17.415818+0000","last_became_active":"2024-03-07T06:51:14.161332+0000","last_became_peered":"2024-03-07T06:51:14.161332+0000","last_unstale":"2024-03-07T06:51:17.415818+0000","last_undegraded":"2024-03-07T06:51:17.415818+0000","last_fullsized":"2024-03-07T06:51:17.415818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415995+0000","last_change":"2024-03-07T06:51:15.200486+0000","last_active":"2024-03-07T06:51:17.415995+0000","last_peered":"2024-03-07T06:51:17.415995+0000","last_clean":"2024-03-07T06:51:17.415995+0000","last_became_active":"2024-03-07T06:51:15.200171+0000","last_became_peered":"2024-03-07T06:51:15.200171+0000","last_unstale":"2024-03-07T06:51:17.415995+0000","last_undegraded":"2024-03-07T06:51:17.415995+0000","last_fullsized":"2024-03-07T06:51:17.415995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:22:38.973224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412810+0000","last_change":"2024-03-07T06:51:14.169164+0000","last_active":"2024-03-07T06:51:17.412810+0000","last_peered":"2024-03-07T06:51:17.412810+0000","last_clean":"2024-03-07T06:51:17.412810+0000","last_became_active":"2024-03-07T06:51:14.168662+0000","last_became_peered":"2024-03-07T06:51:14.168662+0000","last_unstale":"2024-03-07T06:51:17.412810+0000","last_undegraded":"2024-03-07T06:51:17.412810+0000","last_fullsized":"2024-03-07T06:51:17.412810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:51:30.126 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:51:30.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.367+0000 7f9123200640 1 Processor -- start 2024-03-07T06:51:30.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.367+0000 7f9123200640 1 -- start start 2024-03-07T06:51:30.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.368+0000 7f9123200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f912415aaf0 0x7f912406e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:30.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.368+0000 7f9123200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9124072830 0x7f9124072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:30.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.368+0000 7f9123200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912415b5e0 con 0x7f91240cd4c0 2024-03-07T06:51:30.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.368+0000 7f9123200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91240731e0 con 0x7f9124072830 2024-03-07T06:51:30.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.368+0000 7f9123200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912406e850 con 0x7f912415aaf0 2024-03-07T06:51:30.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.369+0000 7f9122800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9124072830 0x7f9124072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:30.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.369+0000 7f911be00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f91240cd4c0 0x7f912415aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:44210/0 (socket says 172.21.15.133:44210) 2024-03-07T06:51:30.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.369+0000 7f911be00640 1 -- 172.21.15.133:0/1391691865 learned_addr learned my addr 172.21.15.133:0/1391691865 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:30.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.369+0000 7f9121e00640 1 --2- 172.21.15.133:0/1391691865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f912415aaf0 0x7f912406e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:30.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.369+0000 7f9122800640 1 -- 172.21.15.133:0/1391691865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f912415aaf0 msgr2=0x7f912406e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:30.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.369+0000 7f9122800640 1 --2- 172.21.15.133:0/1391691865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f912415aaf0 0x7f912406e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.369+0000 7f9122800640 1 -- 172.21.15.133:0/1391691865 >> v1:172.21.15.133:6789/0 conn(0x7f91240cd4c0 legacy=0x7f912415aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:30.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.369+0000 7f9122800640 1 -- 172.21.15.133:0/1391691865 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912406eb30 con 0x7f9124072830 2024-03-07T06:51:30.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.369+0000 7f9121400640 1 -- 172.21.15.133:0/1391691865 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 595569303 0 0) 0x7f912415b5e0 con 0x7f91240cd4c0 2024-03-07T06:51:30.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.370+0000 7f9122800640 1 --2- 172.21.15.133:0/1391691865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9124072830 0x7f9124072ca0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f911c009d90 tx=0x7f911c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575678a1febb2503 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:30.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.370+0000 7f9121400640 1 -- 172.21.15.133:0/1391691865 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c039070 con 0x7f9124072830 2024-03-07T06:51:30.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.370+0000 7f9121400640 1 -- 172.21.15.133:0/1391691865 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c02cc50 con 0x7f9124072830 2024-03-07T06:51:30.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.370+0000 7f9121400640 1 -- 172.21.15.133:0/1391691865 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c034440 con 0x7f9124072830 2024-03-07T06:51:30.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.371+0000 7f9123200640 1 -- 172.21.15.133:0/1391691865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9124072830 msgr2=0x7f9124072ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:30.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.371+0000 7f9123200640 1 --2- 172.21.15.133:0/1391691865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9124072830 0x7f9124072ca0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f911c009d90 tx=0x7f911c02bea0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.371+0000 7f9123200640 1 -- 172.21.15.133:0/1391691865 shutdown_connections 2024-03-07T06:51:30.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.371+0000 7f9123200640 1 --2- 172.21.15.133:0/1391691865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9124072830 0x7f9124072ca0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.371+0000 7f9123200640 1 --2- 172.21.15.133:0/1391691865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f912415aaf0 0x7f912406e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.371+0000 7f9123200640 1 -- 172.21.15.133:0/1391691865 >> 172.21.15.133:0/1391691865 conn(0x7f91240b2e90 msgr2=0x7f91240b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:30.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.371+0000 7f9123200640 1 -- 172.21.15.133:0/1391691865 shutdown_connections 2024-03-07T06:51:30.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.371+0000 7f9123200640 1 -- 172.21.15.133:0/1391691865 wait complete. 2024-03-07T06:51:30.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.373+0000 7f9123200640 1 Processor -- start 2024-03-07T06:51:30.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.373+0000 7f9123200640 1 -- start start 2024-03-07T06:51:30.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9123200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9124072830 0x7f912414d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:30.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9123200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91240cd4c0 0x7f91241544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:30.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9123200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912415aaf0 0x7f9124150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:30.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9123200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912406e560 con 0x7f912415aaf0 2024-03-07T06:51:30.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9123200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91240cdaa0 con 0x7f91240cd4c0 2024-03-07T06:51:30.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9123200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912406f0f0 con 0x7f9124072830 2024-03-07T06:51:30.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9122800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912415aaf0 0x7f9124150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:30.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9122800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912415aaf0 0x7f9124150310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38878/0 (socket says 172.21.15.133:38878) 2024-03-07T06:51:30.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9122800640 1 -- 172.21.15.133:0/3780203510 learned_addr learned my addr 172.21.15.133:0/3780203510 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:30.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f911be00640 1 --2- 172.21.15.133:0/3780203510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91240cd4c0 0x7f91241544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:30.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9121e00640 1 --2- 172.21.15.133:0/3780203510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9124072830 0x7f912414d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:30.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9122800640 1 -- 172.21.15.133:0/3780203510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9124072830 msgr2=0x7f912414d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:30.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9122800640 1 --2- 172.21.15.133:0/3780203510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9124072830 0x7f912414d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9122800640 1 -- 172.21.15.133:0/3780203510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91240cd4c0 msgr2=0x7f91241544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:30.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9122800640 1 --2- 172.21.15.133:0/3780203510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91240cd4c0 0x7f91241544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.374+0000 7f9122800640 1 -- 172.21.15.133:0/3780203510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912414da30 con 0x7f912415aaf0 2024-03-07T06:51:30.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.375+0000 7f9122800640 1 --2- 172.21.15.133:0/3780203510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912415aaf0 0x7f9124150310 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f911c009d90 tx=0x7f911c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:30.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.375+0000 7f911a000640 1 -- 172.21.15.133:0/3780203510 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c039050 con 0x7f912415aaf0 2024-03-07T06:51:30.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.375+0000 7f911a000640 1 -- 172.21.15.133:0/3780203510 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c045070 con 0x7f912415aaf0 2024-03-07T06:51:30.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.375+0000 7f911a000640 1 -- 172.21.15.133:0/3780203510 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c03e610 con 0x7f912415aaf0 2024-03-07T06:51:30.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.375+0000 7f9123200640 1 -- 172.21.15.133:0/3780203510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91240ae020 con 0x7f912415aaf0 2024-03-07T06:51:30.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.375+0000 7f9123200640 1 -- 172.21.15.133:0/3780203510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91240ae5e0 con 0x7f912415aaf0 2024-03-07T06:51:30.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.376+0000 7f9123200640 1 -- 172.21.15.133:0/3780203510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9124062c90 con 0x7f912415aaf0 2024-03-07T06:51:30.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.377+0000 7f911a000640 1 -- 172.21.15.133:0/3780203510 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f911c03e7b0 con 0x7f912415aaf0 2024-03-07T06:51:30.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.378+0000 7f911a000640 1 --2- 172.21.15.133:0/3780203510 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f910006b500 0x7f910006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:30.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.378+0000 7f911a000640 1 -- 172.21.15.133:0/3780203510 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f911c0acce0 con 0x7f912415aaf0 2024-03-07T06:51:30.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.378+0000 7f9121e00640 1 --2- 172.21.15.133:0/3780203510 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f910006b500 0x7f910006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:30.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.379+0000 7f9121e00640 1 --2- 172.21.15.133:0/3780203510 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f910006b500 0x7f910006d9c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9114006fd0 tx=0x7f9114008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:30.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.382+0000 7f911a000640 1 -- 172.21.15.133:0/3780203510 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f911c032e20 con 0x7f912415aaf0 2024-03-07T06:51:30.559 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.559+0000 7f9123200640 1 -- 172.21.15.133:0/3780203510 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f912406daa0 con 0x7f910006b500 2024-03-07T06:51:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.578+0000 7f911a000640 1 -- 172.21.15.133:0/3780203510 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251837 (secure 0 0 0) 0x7f912406daa0 con 0x7f910006b500 2024-03-07T06:51:30.582 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:30.583 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:51:30.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.583+0000 7f9123200640 1 -- 172.21.15.133:0/3780203510 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f910006b500 msgr2=0x7f910006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:30.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.583+0000 7f9123200640 1 --2- 172.21.15.133:0/3780203510 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f910006b500 0x7f910006d9c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9114006fd0 tx=0x7f9114008040 comp rx=0 tx=0).stop 2024-03-07T06:51:30.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.583+0000 7f9123200640 1 -- 172.21.15.133:0/3780203510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912415aaf0 msgr2=0x7f9124150310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:30.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.583+0000 7f9123200640 1 --2- 172.21.15.133:0/3780203510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912415aaf0 0x7f9124150310 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f911c009d90 tx=0x7f911c004a40 comp rx=0 tx=0).stop 2024-03-07T06:51:30.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.584+0000 7f9123200640 1 -- 172.21.15.133:0/3780203510 shutdown_connections 2024-03-07T06:51:30.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.584+0000 7f9123200640 1 --2- 172.21.15.133:0/3780203510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912415aaf0 0x7f9124150310 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.584+0000 7f9123200640 1 --2- 172.21.15.133:0/3780203510 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f910006b500 0x7f910006d9c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.584+0000 7f9123200640 1 --2- 172.21.15.133:0/3780203510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91240cd4c0 0x7f91241544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.584+0000 7f9123200640 1 --2- 172.21.15.133:0/3780203510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9124072830 0x7f912414d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.584+0000 7f9123200640 1 -- 172.21.15.133:0/3780203510 >> 172.21.15.133:0/3780203510 conn(0x7f91240b2e90 msgr2=0x7f912406dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:30.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.584+0000 7f9123200640 1 -- 172.21.15.133:0/3780203510 shutdown_connections 2024-03-07T06:51:30.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.584+0000 7f9123200640 1 -- 172.21.15.133:0/3780203510 wait complete. 2024-03-07T06:51:30.599 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-03-07T06:51:30.475604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219088,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755632,"statfs":{"total":773094113280,"available":772869767168,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005013"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190674+0000","last_change":"2024-03-07T06:51:15.202253+0000","last_active":"2024-03-07T06:51:16.190674+0000","last_peered":"2024-03-07T06:51:16.190674+0000","last_clean":"2024-03-07T06:51:16.190674+0000","last_became_active":"2024-03-07T06:51:15.201848+0000","last_became_peered":"2024-03-07T06:51:15.201848+0000","last_unstale":"2024-03-07T06:51:16.190674+0000","last_undegraded":"2024-03-07T06:51:16.190674+0000","last_fullsized":"2024-03-07T06:51:16.190674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:41:54.645832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205980+0000","last_change":"2024-03-07T06:51:15.188301+0000","last_active":"2024-03-07T06:51:16.205980+0000","last_peered":"2024-03-07T06:51:16.205980+0000","last_clean":"2024-03-07T06:51:16.205980+0000","last_became_active":"2024-03-07T06:51:15.188021+0000","last_became_peered":"2024-03-07T06:51:15.188021+0000","last_unstale":"2024-03-07T06:51:16.205980+0000","last_undegraded":"2024-03-07T06:51:16.205980+0000","last_fullsized":"2024-03-07T06:51:16.205980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412687+0000","last_change":"2024-03-07T06:51:15.188296+0000","last_active":"2024-03-07T06:51:17.412687+0000","last_peered":"2024-03-07T06:51:17.412687+0000","last_clean":"2024-03-07T06:51:17.412687+0000","last_became_active":"2024-03-07T06:51:15.187954+0000","last_became_peered":"2024-03-07T06:51:15.187954+0000","last_unstale":"2024-03-07T06:51:17.412687+0000","last_undegraded":"2024-03-07T06:51:17.412687+0000","last_fullsized":"2024-03-07T06:51:17.412687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414882+0000","last_change":"2024-03-07T06:51:15.200894+0000","last_active":"2024-03-07T06:51:17.414882+0000","last_peered":"2024-03-07T06:51:17.414882+0000","last_clean":"2024-03-07T06:51:17.414882+0000","last_became_active":"2024-03-07T06:51:15.200617+0000","last_became_peered":"2024-03-07T06:51:15.200617+0000","last_unstale":"2024-03-07T06:51:17.414882+0000","last_undegraded":"2024-03-07T06:51:17.414882+0000","last_fullsized":"2024-03-07T06:51:17.414882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190864+0000","last_change":"2024-03-07T06:51:15.189383+0000","last_active":"2024-03-07T06:51:16.190864+0000","last_peered":"2024-03-07T06:51:16.190864+0000","last_clean":"2024-03-07T06:51:16.190864+0000","last_became_active":"2024-03-07T06:51:15.188192+0000","last_became_peered":"2024-03-07T06:51:15.188192+0000","last_unstale":"2024-03-07T06:51:16.190864+0000","last_undegraded":"2024-03-07T06:51:16.190864+0000","last_fullsized":"2024-03-07T06:51:16.190864+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:07:16.800545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414731+0000","last_change":"2024-03-07T06:51:15.195882+0000","last_active":"2024-03-07T06:51:17.414731+0000","last_peered":"2024-03-07T06:51:17.414731+0000","last_clean":"2024-03-07T06:51:17.414731+0000","last_became_active":"2024-03-07T06:51:15.195572+0000","last_became_peered":"2024-03-07T06:51:15.195572+0000","last_unstale":"2024-03-07T06:51:17.414731+0000","last_undegraded":"2024-03-07T06:51:17.414731+0000","last_fullsized":"2024-03-07T06:51:17.414731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189735+0000","last_change":"2024-03-07T06:51:15.190861+0000","last_active":"2024-03-07T06:51:16.189735+0000","last_peered":"2024-03-07T06:51:16.189735+0000","last_clean":"2024-03-07T06:51:16.189735+0000","last_became_active":"2024-03-07T06:51:15.190546+0000","last_became_peered":"2024-03-07T06:51:15.190546+0000","last_unstale":"2024-03-07T06:51:16.189735+0000","last_undegraded":"2024-03-07T06:51:16.189735+0000","last_fullsized":"2024-03-07T06:51:16.189735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:44:23.301530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415109+0000","last_change":"2024-03-07T06:51:15.202655+0000","last_active":"2024-03-07T06:51:17.415109+0000","last_peered":"2024-03-07T06:51:17.415109+0000","last_clean":"2024-03-07T06:51:17.415109+0000","last_became_active":"2024-03-07T06:51:15.197943+0000","last_became_peered":"2024-03-07T06:51:15.197943+0000","last_unstale":"2024-03-07T06:51:17.415109+0000","last_undegraded":"2024-03-07T06:51:17.415109+0000","last_fullsized":"2024-03-07T06:51:17.415109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190628+0000","last_change":"2024-03-07T06:51:15.209333+0000","last_active":"2024-03-07T06:51:16.190628+0000","last_peered":"2024-03-07T06:51:16.190628+0000","last_clean":"2024-03-07T06:51:16.190628+0000","last_became_active":"2024-03-07T06:51:15.209034+0000","last_became_peered":"2024-03-07T06:51:15.209034+0000","last_unstale":"2024-03-07T06:51:16.190628+0000","last_undegraded":"2024-03-07T06:51:16.190628+0000","last_fullsized":"2024-03-07T06:51:16.190628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:31:37.292017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193770+0000","last_change":"2024-03-07T06:51:15.196320+0000","last_active":"2024-03-07T06:51:16.193770+0000","last_peered":"2024-03-07T06:51:16.193770+0000","last_clean":"2024-03-07T06:51:16.193770+0000","last_became_active":"2024-03-07T06:51:15.196029+0000","last_became_peered":"2024-03-07T06:51:15.196029+0000","last_unstale":"2024-03-07T06:51:16.193770+0000","last_undegraded":"2024-03-07T06:51:16.193770+0000","last_fullsized":"2024-03-07T06:51:16.193770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192627+0000","last_change":"2024-03-07T06:51:15.212042+0000","last_active":"2024-03-07T06:51:16.192627+0000","last_peered":"2024-03-07T06:51:16.192627+0000","last_clean":"2024-03-07T06:51:16.192627+0000","last_became_active":"2024-03-07T06:51:15.211737+0000","last_became_peered":"2024-03-07T06:51:15.211737+0000","last_unstale":"2024-03-07T06:51:16.192627+0000","last_undegraded":"2024-03-07T06:51:16.192627+0000","last_fullsized":"2024-03-07T06:51:16.192627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192585+0000","last_change":"2024-03-07T06:51:15.200183+0000","last_active":"2024-03-07T06:51:16.192585+0000","last_peered":"2024-03-07T06:51:16.192585+0000","last_clean":"2024-03-07T06:51:16.192585+0000","last_became_active":"2024-03-07T06:51:15.199322+0000","last_became_peered":"2024-03-07T06:51:15.199322+0000","last_unstale":"2024-03-07T06:51:16.192585+0000","last_undegraded":"2024-03-07T06:51:16.192585+0000","last_fullsized":"2024-03-07T06:51:16.192585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:11.295266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415528+0000","last_change":"2024-03-07T06:51:15.201141+0000","last_active":"2024-03-07T06:51:17.415528+0000","last_peered":"2024-03-07T06:51:17.415528+0000","last_clean":"2024-03-07T06:51:17.415528+0000","last_became_active":"2024-03-07T06:51:15.200965+0000","last_became_peered":"2024-03-07T06:51:15.200965+0000","last_unstale":"2024-03-07T06:51:17.415528+0000","last_undegraded":"2024-03-07T06:51:17.415528+0000","last_fullsized":"2024-03-07T06:51:17.415528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412986+0000","last_change":"2024-03-07T06:51:15.191043+0000","last_active":"2024-03-07T06:51:17.412986+0000","last_peered":"2024-03-07T06:51:17.412986+0000","last_clean":"2024-03-07T06:51:17.412986+0000","last_became_active":"2024-03-07T06:51:15.190777+0000","last_became_peered":"2024-03-07T06:51:15.190777+0000","last_unstale":"2024-03-07T06:51:17.412986+0000","last_undegraded":"2024-03-07T06:51:17.412986+0000","last_fullsized":"2024-03-07T06:51:17.412986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205186+0000","last_change":"2024-03-07T06:51:15.178908+0000","last_active":"2024-03-07T06:51:16.205186+0000","last_peered":"2024-03-07T06:51:16.205186+0000","last_clean":"2024-03-07T06:51:16.205186+0000","last_became_active":"2024-03-07T06:51:15.178387+0000","last_became_peered":"2024-03-07T06:51:15.178387+0000","last_unstale":"2024-03-07T06:51:16.205186+0000","last_undegraded":"2024-03-07T06:51:16.205186+0000","last_fullsized":"2024-03-07T06:51:16.205186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414392+0000","last_change":"2024-03-07T06:51:15.195007+0000","last_active":"2024-03-07T06:51:17.414392+0000","last_peered":"2024-03-07T06:51:17.414392+0000","last_clean":"2024-03-07T06:51:17.414392+0000","last_became_active":"2024-03-07T06:51:15.194624+0000","last_became_peered":"2024-03-07T06:51:15.194624+0000","last_unstale":"2024-03-07T06:51:17.414392+0000","last_undegraded":"2024-03-07T06:51:17.414392+0000","last_fullsized":"2024-03-07T06:51:17.414392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.416108+0000","last_change":"2024-03-07T06:51:15.206405+0000","last_active":"2024-03-07T06:51:17.416108+0000","last_peered":"2024-03-07T06:51:17.416108+0000","last_clean":"2024-03-07T06:51:17.416108+0000","last_became_active":"2024-03-07T06:51:15.205493+0000","last_became_peered":"2024-03-07T06:51:15.205493+0000","last_unstale":"2024-03-07T06:51:17.416108+0000","last_undegraded":"2024-03-07T06:51:17.416108+0000","last_fullsized":"2024-03-07T06:51:17.416108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:43:58.927044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413141+0000","last_change":"2024-03-07T06:51:15.195307+0000","last_active":"2024-03-07T06:51:17.413141+0000","last_peered":"2024-03-07T06:51:17.413141+0000","last_clean":"2024-03-07T06:51:17.413141+0000","last_became_active":"2024-03-07T06:51:15.194990+0000","last_became_peered":"2024-03-07T06:51:15.194990+0000","last_unstale":"2024-03-07T06:51:17.413141+0000","last_undegraded":"2024-03-07T06:51:17.413141+0000","last_fullsized":"2024-03-07T06:51:17.413141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:11.295266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189287+0000","last_change":"2024-03-07T06:51:15.197914+0000","last_active":"2024-03-07T06:51:16.189287+0000","last_peered":"2024-03-07T06:51:16.189287+0000","last_clean":"2024-03-07T06:51:16.189287+0000","last_became_active":"2024-03-07T06:51:15.196938+0000","last_became_peered":"2024-03-07T06:51:15.196938+0000","last_unstale":"2024-03-07T06:51:16.189287+0000","last_undegraded":"2024-03-07T06:51:16.189287+0000","last_fullsized":"2024-03-07T06:51:16.189287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:56.465649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192765+0000","last_change":"2024-03-07T06:51:15.199775+0000","last_active":"2024-03-07T06:51:16.192765+0000","last_peered":"2024-03-07T06:51:16.192765+0000","last_clean":"2024-03-07T06:51:16.192765+0000","last_became_active":"2024-03-07T06:51:15.199447+0000","last_became_peered":"2024-03-07T06:51:15.199447+0000","last_unstale":"2024-03-07T06:51:16.192765+0000","last_undegraded":"2024-03-07T06:51:16.192765+0000","last_fullsized":"2024-03-07T06:51:16.192765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413641+0000","last_change":"2024-03-07T06:51:15.191968+0000","last_active":"2024-03-07T06:51:17.413641+0000","last_peered":"2024-03-07T06:51:17.413641+0000","last_clean":"2024-03-07T06:51:17.413641+0000","last_became_active":"2024-03-07T06:51:15.191587+0000","last_became_peered":"2024-03-07T06:51:15.191587+0000","last_unstale":"2024-03-07T06:51:17.413641+0000","last_undegraded":"2024-03-07T06:51:17.413641+0000","last_fullsized":"2024-03-07T06:51:17.413641+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415294+0000","last_change":"2024-03-07T06:51:15.206834+0000","last_active":"2024-03-07T06:51:17.415294+0000","last_peered":"2024-03-07T06:51:17.415294+0000","last_clean":"2024-03-07T06:51:17.415294+0000","last_became_active":"2024-03-07T06:51:15.198815+0000","last_became_peered":"2024-03-07T06:51:15.198815+0000","last_unstale":"2024-03-07T06:51:17.415294+0000","last_undegraded":"2024-03-07T06:51:17.415294+0000","last_fullsized":"2024-03-07T06:51:17.415294+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415387+0000","last_change":"2024-03-07T06:51:15.203302+0000","last_active":"2024-03-07T06:51:17.415387+0000","last_peered":"2024-03-07T06:51:17.415387+0000","last_clean":"2024-03-07T06:51:17.415387+0000","last_became_active":"2024-03-07T06:51:15.203041+0000","last_became_peered":"2024-03-07T06:51:15.203041+0000","last_unstale":"2024-03-07T06:51:17.415387+0000","last_undegraded":"2024-03-07T06:51:17.415387+0000","last_fullsized":"2024-03-07T06:51:17.415387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414478+0000","last_change":"2024-03-07T06:51:15.195865+0000","last_active":"2024-03-07T06:51:17.414478+0000","last_peered":"2024-03-07T06:51:17.414478+0000","last_clean":"2024-03-07T06:51:17.414478+0000","last_became_active":"2024-03-07T06:51:15.194043+0000","last_became_peered":"2024-03-07T06:51:15.194043+0000","last_unstale":"2024-03-07T06:51:17.414478+0000","last_undegraded":"2024-03-07T06:51:17.414478+0000","last_fullsized":"2024-03-07T06:51:17.414478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415922+0000","last_change":"2024-03-07T06:51:15.203857+0000","last_active":"2024-03-07T06:51:17.415922+0000","last_peered":"2024-03-07T06:51:17.415922+0000","last_clean":"2024-03-07T06:51:17.415922+0000","last_became_active":"2024-03-07T06:51:15.203562+0000","last_became_peered":"2024-03-07T06:51:15.203562+0000","last_unstale":"2024-03-07T06:51:17.415922+0000","last_undegraded":"2024-03-07T06:51:17.415922+0000","last_fullsized":"2024-03-07T06:51:17.415922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:34:40.365080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193687+0000","last_change":"2024-03-07T06:51:15.210400+0000","last_active":"2024-03-07T06:51:16.193687+0000","last_peered":"2024-03-07T06:51:16.193687+0000","last_clean":"2024-03-07T06:51:16.193687+0000","last_became_active":"2024-03-07T06:51:15.209692+0000","last_became_peered":"2024-03-07T06:51:15.209692+0000","last_unstale":"2024-03-07T06:51:16.193687+0000","last_undegraded":"2024-03-07T06:51:16.193687+0000","last_fullsized":"2024-03-07T06:51:16.193687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205792+0000","last_change":"2024-03-07T06:51:15.190106+0000","last_active":"2024-03-07T06:51:16.205792+0000","last_peered":"2024-03-07T06:51:16.205792+0000","last_clean":"2024-03-07T06:51:16.205792+0000","last_became_active":"2024-03-07T06:51:15.189498+0000","last_became_peered":"2024-03-07T06:51:15.189498+0000","last_unstale":"2024-03-07T06:51:16.205792+0000","last_undegraded":"2024-03-07T06:51:16.205792+0000","last_fullsized":"2024-03-07T06:51:16.205792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415671+0000","last_change":"2024-03-07T06:51:15.203374+0000","last_active":"2024-03-07T06:51:17.415671+0000","last_peered":"2024-03-07T06:51:17.415671+0000","last_clean":"2024-03-07T06:51:17.415671+0000","last_became_active":"2024-03-07T06:51:15.202849+0000","last_became_peered":"2024-03-07T06:51:15.202849+0000","last_unstale":"2024-03-07T06:51:17.415671+0000","last_undegraded":"2024-03-07T06:51:17.415671+0000","last_fullsized":"2024-03-07T06:51:17.415671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413542+0000","last_change":"2024-03-07T06:51:15.200244+0000","last_active":"2024-03-07T06:51:17.413542+0000","last_peered":"2024-03-07T06:51:17.413542+0000","last_clean":"2024-03-07T06:51:17.413542+0000","last_became_active":"2024-03-07T06:51:15.198483+0000","last_became_peered":"2024-03-07T06:51:15.198483+0000","last_unstale":"2024-03-07T06:51:17.413542+0000","last_undegraded":"2024-03-07T06:51:17.413542+0000","last_fullsized":"2024-03-07T06:51:17.413542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:31:51.836751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415577+0000","last_change":"2024-03-07T06:51:15.195170+0000","last_active":"2024-03-07T06:51:17.415577+0000","last_peered":"2024-03-07T06:51:17.415577+0000","last_clean":"2024-03-07T06:51:17.415577+0000","last_became_active":"2024-03-07T06:51:15.194900+0000","last_became_peered":"2024-03-07T06:51:15.194900+0000","last_unstale":"2024-03-07T06:51:17.415577+0000","last_undegraded":"2024-03-07T06:51:17.415577+0000","last_fullsized":"2024-03-07T06:51:17.415577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:44:23.301530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414305+0000","last_change":"2024-03-07T06:51:15.197411+0000","last_active":"2024-03-07T06:51:17.414305+0000","last_peered":"2024-03-07T06:51:17.414305+0000","last_clean":"2024-03-07T06:51:17.414305+0000","last_became_active":"2024-03-07T06:51:15.194699+0000","last_became_peered":"2024-03-07T06:51:15.194699+0000","last_unstale":"2024-03-07T06:51:17.414305+0000","last_undegraded":"2024-03-07T06:51:17.414305+0000","last_fullsized":"2024-03-07T06:51:17.414305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:43.574428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.195220+0000","last_change":"2024-03-07T06:51:15.201331+0000","last_active":"2024-03-07T06:51:16.195220+0000","last_peered":"2024-03-07T06:51:16.195220+0000","last_clean":"2024-03-07T06:51:16.195220+0000","last_became_active":"2024-03-07T06:51:15.200758+0000","last_became_peered":"2024-03-07T06:51:15.200758+0000","last_unstale":"2024-03-07T06:51:16.195220+0000","last_undegraded":"2024-03-07T06:51:16.195220+0000","last_fullsized":"2024-03-07T06:51:16.195220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192348+0000","last_change":"2024-03-07T06:51:15.180977+0000","last_active":"2024-03-07T06:51:16.192348+0000","last_peered":"2024-03-07T06:51:16.192348+0000","last_clean":"2024-03-07T06:51:16.192348+0000","last_became_active":"2024-03-07T06:51:15.180359+0000","last_became_peered":"2024-03-07T06:51:15.180359+0000","last_unstale":"2024-03-07T06:51:16.192348+0000","last_undegraded":"2024-03-07T06:51:16.192348+0000","last_fullsized":"2024-03-07T06:51:16.192348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:10.776380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415597+0000","last_change":"2024-03-07T06:51:14.171537+0000","last_active":"2024-03-07T06:51:17.415597+0000","last_peered":"2024-03-07T06:51:17.415597+0000","last_clean":"2024-03-07T06:51:17.415597+0000","last_became_active":"2024-03-07T06:51:14.171105+0000","last_became_peered":"2024-03-07T06:51:14.171105+0000","last_unstale":"2024-03-07T06:51:17.415597+0000","last_undegraded":"2024-03-07T06:51:17.415597+0000","last_fullsized":"2024-03-07T06:51:17.415597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413743+0000","last_change":"2024-03-07T06:51:15.198729+0000","last_active":"2024-03-07T06:51:17.413743+0000","last_peered":"2024-03-07T06:51:17.413743+0000","last_clean":"2024-03-07T06:51:17.413743+0000","last_became_active":"2024-03-07T06:51:15.198412+0000","last_became_peered":"2024-03-07T06:51:15.198412+0000","last_unstale":"2024-03-07T06:51:17.413743+0000","last_undegraded":"2024-03-07T06:51:17.413743+0000","last_fullsized":"2024-03-07T06:51:17.413743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:13.570395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192432+0000","last_change":"2024-03-07T06:51:14.172967+0000","last_active":"2024-03-07T06:51:16.192432+0000","last_peered":"2024-03-07T06:51:16.192432+0000","last_clean":"2024-03-07T06:51:16.192432+0000","last_became_active":"2024-03-07T06:51:14.172706+0000","last_became_peered":"2024-03-07T06:51:14.172706+0000","last_unstale":"2024-03-07T06:51:16.192432+0000","last_undegraded":"2024-03-07T06:51:16.192432+0000","last_fullsized":"2024-03-07T06:51:16.192432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415155+0000","last_change":"2024-03-07T06:51:14.167651+0000","last_active":"2024-03-07T06:51:17.415155+0000","last_peered":"2024-03-07T06:51:17.415155+0000","last_clean":"2024-03-07T06:51:17.415155+0000","last_became_active":"2024-03-07T06:51:14.166984+0000","last_became_peered":"2024-03-07T06:51:14.166984+0000","last_unstale":"2024-03-07T06:51:17.415155+0000","last_undegraded":"2024-03-07T06:51:17.415155+0000","last_fullsized":"2024-03-07T06:51:17.415155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415067+0000","last_change":"2024-03-07T06:51:15.202906+0000","last_active":"2024-03-07T06:51:17.415067+0000","last_peered":"2024-03-07T06:51:17.415067+0000","last_clean":"2024-03-07T06:51:17.415067+0000","last_became_active":"2024-03-07T06:51:15.200219+0000","last_became_peered":"2024-03-07T06:51:15.200219+0000","last_unstale":"2024-03-07T06:51:17.415067+0000","last_undegraded":"2024-03-07T06:51:17.415067+0000","last_fullsized":"2024-03-07T06:51:17.415067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414528+0000","last_change":"2024-03-07T06:51:14.175653+0000","last_active":"2024-03-07T06:51:17.414528+0000","last_peered":"2024-03-07T06:51:17.414528+0000","last_clean":"2024-03-07T06:51:17.414528+0000","last_became_active":"2024-03-07T06:51:14.175295+0000","last_became_peered":"2024-03-07T06:51:14.175295+0000","last_unstale":"2024-03-07T06:51:17.414528+0000","last_undegraded":"2024-03-07T06:51:17.414528+0000","last_fullsized":"2024-03-07T06:51:17.414528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205033+0000","last_change":"2024-03-07T06:51:15.190580+0000","last_active":"2024-03-07T06:51:16.205033+0000","last_peered":"2024-03-07T06:51:16.205033+0000","last_clean":"2024-03-07T06:51:16.205033+0000","last_became_active":"2024-03-07T06:51:15.189190+0000","last_became_peered":"2024-03-07T06:51:15.189190+0000","last_unstale":"2024-03-07T06:51:16.205033+0000","last_undegraded":"2024-03-07T06:51:16.205033+0000","last_fullsized":"2024-03-07T06:51:16.205033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:03:29.053842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193565+0000","last_change":"2024-03-07T06:51:14.173746+0000","last_active":"2024-03-07T06:51:16.193565+0000","last_peered":"2024-03-07T06:51:16.193565+0000","last_clean":"2024-03-07T06:51:16.193565+0000","last_became_active":"2024-03-07T06:51:14.173285+0000","last_became_peered":"2024-03-07T06:51:14.173285+0000","last_unstale":"2024-03-07T06:51:16.193565+0000","last_undegraded":"2024-03-07T06:51:16.193565+0000","last_fullsized":"2024-03-07T06:51:16.193565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415198+0000","last_change":"2024-03-07T06:51:15.215525+0000","last_active":"2024-03-07T06:51:17.415198+0000","last_peered":"2024-03-07T06:51:17.415198+0000","last_clean":"2024-03-07T06:51:17.415198+0000","last_became_active":"2024-03-07T06:51:15.215050+0000","last_became_peered":"2024-03-07T06:51:15.215050+0000","last_unstale":"2024-03-07T06:51:17.415198+0000","last_undegraded":"2024-03-07T06:51:17.415198+0000","last_fullsized":"2024-03-07T06:51:17.415198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:22:38.973224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414959+0000","last_change":"2024-03-07T06:51:14.168633+0000","last_active":"2024-03-07T06:51:17.414959+0000","last_peered":"2024-03-07T06:51:17.414959+0000","last_clean":"2024-03-07T06:51:17.414959+0000","last_became_active":"2024-03-07T06:51:14.168293+0000","last_became_peered":"2024-03-07T06:51:14.168293+0000","last_unstale":"2024-03-07T06:51:17.414959+0000","last_undegraded":"2024-03-07T06:51:17.414959+0000","last_fullsized":"2024-03-07T06:51:17.414959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412761+0000","last_change":"2024-03-07T06:51:15.208732+0000","last_active":"2024-03-07T06:51:17.412761+0000","last_peered":"2024-03-07T06:51:17.412761+0000","last_clean":"2024-03-07T06:51:17.412761+0000","last_became_active":"2024-03-07T06:51:15.208472+0000","last_became_peered":"2024-03-07T06:51:15.208472+0000","last_unstale":"2024-03-07T06:51:17.412761+0000","last_undegraded":"2024-03-07T06:51:17.412761+0000","last_fullsized":"2024-03-07T06:51:17.412761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205600+0000","last_change":"2024-03-07T06:51:14.171631+0000","last_active":"2024-03-07T06:51:16.205600+0000","last_peered":"2024-03-07T06:51:16.205600+0000","last_clean":"2024-03-07T06:51:16.205600+0000","last_became_active":"2024-03-07T06:51:14.171058+0000","last_became_peered":"2024-03-07T06:51:14.171058+0000","last_unstale":"2024-03-07T06:51:16.205600+0000","last_undegraded":"2024-03-07T06:51:16.205600+0000","last_fullsized":"2024-03-07T06:51:16.205600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193446+0000","last_change":"2024-03-07T06:51:15.195194+0000","last_active":"2024-03-07T06:51:16.193446+0000","last_peered":"2024-03-07T06:51:16.193446+0000","last_clean":"2024-03-07T06:51:16.193446+0000","last_became_active":"2024-03-07T06:51:15.191103+0000","last_became_peered":"2024-03-07T06:51:15.191103+0000","last_unstale":"2024-03-07T06:51:16.193446+0000","last_undegraded":"2024-03-07T06:51:16.193446+0000","last_fullsized":"2024-03-07T06:51:16.193446+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:40:51.935140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192876+0000","last_change":"2024-03-07T06:51:14.166640+0000","last_active":"2024-03-07T06:51:16.192876+0000","last_peered":"2024-03-07T06:51:16.192876+0000","last_clean":"2024-03-07T06:51:16.192876+0000","last_became_active":"2024-03-07T06:51:14.166329+0000","last_became_peered":"2024-03-07T06:51:14.166329+0000","last_unstale":"2024-03-07T06:51:16.192876+0000","last_undegraded":"2024-03-07T06:51:16.192876+0000","last_fullsized":"2024-03-07T06:51:16.192876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188809+0000","last_change":"2024-03-07T06:51:15.193638+0000","last_active":"2024-03-07T06:51:16.188809+0000","last_peered":"2024-03-07T06:51:16.188809+0000","last_clean":"2024-03-07T06:51:16.188809+0000","last_became_active":"2024-03-07T06:51:15.193354+0000","last_became_peered":"2024-03-07T06:51:15.193354+0000","last_unstale":"2024-03-07T06:51:16.188809+0000","last_undegraded":"2024-03-07T06:51:16.188809+0000","last_fullsized":"2024-03-07T06:51:16.188809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413807+0000","last_change":"2024-03-07T06:51:14.167158+0000","last_active":"2024-03-07T06:51:17.413807+0000","last_peered":"2024-03-07T06:51:17.413807+0000","last_clean":"2024-03-07T06:51:17.413807+0000","last_became_active":"2024-03-07T06:51:14.166835+0000","last_became_peered":"2024-03-07T06:51:14.166835+0000","last_unstale":"2024-03-07T06:51:17.413807+0000","last_undegraded":"2024-03-07T06:51:17.413807+0000","last_fullsized":"2024-03-07T06:51:17.413807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:38:27.101103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413891+0000","last_change":"2024-03-07T06:51:15.199375+0000","last_active":"2024-03-07T06:51:17.413891+0000","last_peered":"2024-03-07T06:51:17.413891+0000","last_clean":"2024-03-07T06:51:17.413891+0000","last_became_active":"2024-03-07T06:51:15.199060+0000","last_became_peered":"2024-03-07T06:51:15.199060+0000","last_unstale":"2024-03-07T06:51:17.413891+0000","last_undegraded":"2024-03-07T06:51:17.413891+0000","last_fullsized":"2024-03-07T06:51:17.413891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415423+0000","last_change":"2024-03-07T06:51:14.170514+0000","last_active":"2024-03-07T06:51:17.415423+0000","last_peered":"2024-03-07T06:51:17.415423+0000","last_clean":"2024-03-07T06:51:17.415423+0000","last_became_active":"2024-03-07T06:51:14.169542+0000","last_became_peered":"2024-03-07T06:51:14.169542+0000","last_unstale":"2024-03-07T06:51:17.415423+0000","last_undegraded":"2024-03-07T06:51:17.415423+0000","last_fullsized":"2024-03-07T06:51:17.415423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191234+0000","last_change":"2024-03-07T06:51:15.208081+0000","last_active":"2024-03-07T06:51:16.191234+0000","last_peered":"2024-03-07T06:51:16.191234+0000","last_clean":"2024-03-07T06:51:16.191234+0000","last_became_active":"2024-03-07T06:51:15.207758+0000","last_became_peered":"2024-03-07T06:51:15.207758+0000","last_unstale":"2024-03-07T06:51:16.191234+0000","last_undegraded":"2024-03-07T06:51:16.191234+0000","last_fullsized":"2024-03-07T06:51:16.191234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:43:16.081728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191798+0000","last_change":"2024-03-07T06:51:14.168470+0000","last_active":"2024-03-07T06:51:16.191798+0000","last_peered":"2024-03-07T06:51:16.191798+0000","last_clean":"2024-03-07T06:51:16.191798+0000","last_became_active":"2024-03-07T06:51:14.168190+0000","last_became_peered":"2024-03-07T06:51:14.168190+0000","last_unstale":"2024-03-07T06:51:16.191798+0000","last_undegraded":"2024-03-07T06:51:16.191798+0000","last_fullsized":"2024-03-07T06:51:16.191798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192778+0000","last_change":"2024-03-07T06:51:15.201105+0000","last_active":"2024-03-07T06:51:16.192778+0000","last_peered":"2024-03-07T06:51:16.192778+0000","last_clean":"2024-03-07T06:51:16.192778+0000","last_became_active":"2024-03-07T06:51:15.200404+0000","last_became_peered":"2024-03-07T06:51:15.200404+0000","last_unstale":"2024-03-07T06:51:16.192778+0000","last_undegraded":"2024-03-07T06:51:16.192778+0000","last_fullsized":"2024-03-07T06:51:16.192778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412017+0000","last_change":"2024-03-07T06:51:14.154100+0000","last_active":"2024-03-07T06:51:17.412017+0000","last_peered":"2024-03-07T06:51:17.412017+0000","last_clean":"2024-03-07T06:51:17.412017+0000","last_became_active":"2024-03-07T06:51:14.152648+0000","last_became_peered":"2024-03-07T06:51:14.152648+0000","last_unstale":"2024-03-07T06:51:17.412017+0000","last_undegraded":"2024-03-07T06:51:17.412017+0000","last_fullsized":"2024-03-07T06:51:17.412017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189886+0000","last_change":"2024-03-07T06:51:15.203942+0000","last_active":"2024-03-07T06:51:16.189886+0000","last_peered":"2024-03-07T06:51:16.189886+0000","last_clean":"2024-03-07T06:51:16.189886+0000","last_became_active":"2024-03-07T06:51:15.203671+0000","last_became_peered":"2024-03-07T06:51:15.203671+0000","last_unstale":"2024-03-07T06:51:16.189886+0000","last_undegraded":"2024-03-07T06:51:16.189886+0000","last_fullsized":"2024-03-07T06:51:16.189886+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:02:55.966834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193630+0000","last_change":"2024-03-07T06:51:14.164166+0000","last_active":"2024-03-07T06:51:16.193630+0000","last_peered":"2024-03-07T06:51:16.193630+0000","last_clean":"2024-03-07T06:51:16.193630+0000","last_became_active":"2024-03-07T06:51:14.163821+0000","last_became_peered":"2024-03-07T06:51:14.163821+0000","last_unstale":"2024-03-07T06:51:16.193630+0000","last_undegraded":"2024-03-07T06:51:16.193630+0000","last_fullsized":"2024-03-07T06:51:16.193630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:38:27.101103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190408+0000","last_change":"2024-03-07T06:51:15.207392+0000","last_active":"2024-03-07T06:51:16.190408+0000","last_peered":"2024-03-07T06:51:16.190408+0000","last_clean":"2024-03-07T06:51:16.190408+0000","last_became_active":"2024-03-07T06:51:15.203049+0000","last_became_peered":"2024-03-07T06:51:15.203049+0000","last_unstale":"2024-03-07T06:51:16.190408+0000","last_undegraded":"2024-03-07T06:51:16.190408+0000","last_fullsized":"2024-03-07T06:51:16.190408+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:43:58.927044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413191+0000","last_change":"2024-03-07T06:51:14.162546+0000","last_active":"2024-03-07T06:51:17.413191+0000","last_peered":"2024-03-07T06:51:17.413191+0000","last_clean":"2024-03-07T06:51:17.413191+0000","last_became_active":"2024-03-07T06:51:14.161082+0000","last_became_peered":"2024-03-07T06:51:14.161082+0000","last_unstale":"2024-03-07T06:51:17.413191+0000","last_undegraded":"2024-03-07T06:51:17.413191+0000","last_fullsized":"2024-03-07T06:51:17.413191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.190725+0000","last_change":"2024-03-07T06:51:15.208036+0000","last_active":"2024-03-07T06:51:16.190725+0000","last_peered":"2024-03-07T06:51:16.190725+0000","last_clean":"2024-03-07T06:51:16.190725+0000","last_became_active":"2024-03-07T06:51:15.206837+0000","last_became_peered":"2024-03-07T06:51:15.206837+0000","last_unstale":"2024-03-07T06:51:16.190725+0000","last_undegraded":"2024-03-07T06:51:16.190725+0000","last_fullsized":"2024-03-07T06:51:16.190725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:55:33.816984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205890+0000","last_change":"2024-03-07T06:51:14.168546+0000","last_active":"2024-03-07T06:51:16.205890+0000","last_peered":"2024-03-07T06:51:16.205890+0000","last_clean":"2024-03-07T06:51:16.205890+0000","last_became_active":"2024-03-07T06:51:14.168238+0000","last_became_peered":"2024-03-07T06:51:14.168238+0000","last_unstale":"2024-03-07T06:51:16.205890+0000","last_undegraded":"2024-03-07T06:51:16.205890+0000","last_fullsized":"2024-03-07T06:51:16.205890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415347+0000","last_change":"2024-03-07T06:51:15.204629+0000","last_active":"2024-03-07T06:51:17.415347+0000","last_peered":"2024-03-07T06:51:17.415347+0000","last_clean":"2024-03-07T06:51:17.415347+0000","last_became_active":"2024-03-07T06:51:15.203615+0000","last_became_peered":"2024-03-07T06:51:15.203615+0000","last_unstale":"2024-03-07T06:51:17.415347+0000","last_undegraded":"2024-03-07T06:51:17.415347+0000","last_fullsized":"2024-03-07T06:51:17.415347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:38:28.416311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412767+0000","last_change":"2024-03-07T06:51:14.170179+0000","last_active":"2024-03-07T06:51:17.412767+0000","last_peered":"2024-03-07T06:51:17.412767+0000","last_clean":"2024-03-07T06:51:17.412767+0000","last_became_active":"2024-03-07T06:51:14.169797+0000","last_became_peered":"2024-03-07T06:51:14.169797+0000","last_unstale":"2024-03-07T06:51:17.412767+0000","last_undegraded":"2024-03-07T06:51:17.412767+0000","last_fullsized":"2024-03-07T06:51:17.412767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188955+0000","last_change":"2024-03-07T06:51:15.212961+0000","last_active":"2024-03-07T06:51:16.188955+0000","last_peered":"2024-03-07T06:51:16.188955+0000","last_clean":"2024-03-07T06:51:16.188955+0000","last_became_active":"2024-03-07T06:51:15.212691+0000","last_became_peered":"2024-03-07T06:51:15.212691+0000","last_unstale":"2024-03-07T06:51:16.188955+0000","last_undegraded":"2024-03-07T06:51:16.188955+0000","last_fullsized":"2024-03-07T06:51:16.188955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:38:28.416311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188951+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:51:16.188951+0000","last_peered":"2024-03-07T06:51:16.188951+0000","last_clean":"2024-03-07T06:51:16.188951+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:51:16.188951+0000","last_undegraded":"2024-03-07T06:51:16.188951+0000","last_fullsized":"2024-03-07T06:51:16.188951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.414355+0000","last_change":"2024-03-07T06:51:14.172051+0000","last_active":"2024-03-07T06:51:17.414355+0000","last_peered":"2024-03-07T06:51:17.414355+0000","last_clean":"2024-03-07T06:51:17.414355+0000","last_became_active":"2024-03-07T06:51:14.170736+0000","last_became_peered":"2024-03-07T06:51:14.170736+0000","last_unstale":"2024-03-07T06:51:17.414355+0000","last_undegraded":"2024-03-07T06:51:17.414355+0000","last_fullsized":"2024-03-07T06:51:17.414355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415537+0000","last_change":"2024-03-07T06:51:15.208661+0000","last_active":"2024-03-07T06:51:17.415537+0000","last_peered":"2024-03-07T06:51:17.415537+0000","last_clean":"2024-03-07T06:51:17.415537+0000","last_became_active":"2024-03-07T06:51:15.208394+0000","last_became_peered":"2024-03-07T06:51:15.208394+0000","last_unstale":"2024-03-07T06:51:17.415537+0000","last_undegraded":"2024-03-07T06:51:17.415537+0000","last_fullsized":"2024-03-07T06:51:17.415537+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:37.904679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188640+0000","last_change":"2024-03-07T06:51:14.163185+0000","last_active":"2024-03-07T06:51:16.188640+0000","last_peered":"2024-03-07T06:51:16.188640+0000","last_clean":"2024-03-07T06:51:16.188640+0000","last_became_active":"2024-03-07T06:51:14.162494+0000","last_became_peered":"2024-03-07T06:51:14.162494+0000","last_unstale":"2024-03-07T06:51:16.188640+0000","last_undegraded":"2024-03-07T06:51:16.188640+0000","last_fullsized":"2024-03-07T06:51:16.188640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:42.562252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412672+0000","last_change":"2024-03-07T06:51:15.193638+0000","last_active":"2024-03-07T06:51:17.412672+0000","last_peered":"2024-03-07T06:51:17.412672+0000","last_clean":"2024-03-07T06:51:17.412672+0000","last_became_active":"2024-03-07T06:51:15.193379+0000","last_became_peered":"2024-03-07T06:51:15.193379+0000","last_unstale":"2024-03-07T06:51:17.412672+0000","last_undegraded":"2024-03-07T06:51:17.412672+0000","last_fullsized":"2024-03-07T06:51:17.412672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:16:10.967500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412927+0000","last_change":"2024-03-07T06:51:14.163793+0000","last_active":"2024-03-07T06:51:17.412927+0000","last_peered":"2024-03-07T06:51:17.412927+0000","last_clean":"2024-03-07T06:51:17.412927+0000","last_became_active":"2024-03-07T06:51:14.162059+0000","last_became_peered":"2024-03-07T06:51:14.162059+0000","last_unstale":"2024-03-07T06:51:17.412927+0000","last_undegraded":"2024-03-07T06:51:17.412927+0000","last_fullsized":"2024-03-07T06:51:17.412927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:02:48.000556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205257+0000","last_change":"2024-03-07T06:51:15.187522+0000","last_active":"2024-03-07T06:51:16.205257+0000","last_peered":"2024-03-07T06:51:16.205257+0000","last_clean":"2024-03-07T06:51:16.205257+0000","last_became_active":"2024-03-07T06:51:15.179882+0000","last_became_peered":"2024-03-07T06:51:15.179882+0000","last_unstale":"2024-03-07T06:51:16.205257+0000","last_undegraded":"2024-03-07T06:51:16.205257+0000","last_fullsized":"2024-03-07T06:51:16.205257+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415629+0000","last_change":"2024-03-07T06:51:14.164766+0000","last_active":"2024-03-07T06:51:17.415629+0000","last_peered":"2024-03-07T06:51:17.415629+0000","last_clean":"2024-03-07T06:51:17.415629+0000","last_became_active":"2024-03-07T06:51:14.164397+0000","last_became_peered":"2024-03-07T06:51:14.164397+0000","last_unstale":"2024-03-07T06:51:17.415629+0000","last_undegraded":"2024-03-07T06:51:17.415629+0000","last_fullsized":"2024-03-07T06:51:17.415629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192589+0000","last_change":"2024-03-07T06:51:15.190608+0000","last_active":"2024-03-07T06:51:16.192589+0000","last_peered":"2024-03-07T06:51:16.192589+0000","last_clean":"2024-03-07T06:51:16.192589+0000","last_became_active":"2024-03-07T06:51:15.190326+0000","last_became_peered":"2024-03-07T06:51:15.190326+0000","last_unstale":"2024-03-07T06:51:16.192589+0000","last_undegraded":"2024-03-07T06:51:16.192589+0000","last_fullsized":"2024-03-07T06:51:16.192589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:57:43.574428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205896+0000","last_change":"2024-03-07T06:51:14.164402+0000","last_active":"2024-03-07T06:51:16.205896+0000","last_peered":"2024-03-07T06:51:16.205896+0000","last_clean":"2024-03-07T06:51:16.205896+0000","last_became_active":"2024-03-07T06:51:14.164081+0000","last_became_peered":"2024-03-07T06:51:14.164081+0000","last_unstale":"2024-03-07T06:51:16.205896+0000","last_undegraded":"2024-03-07T06:51:16.205896+0000","last_fullsized":"2024-03-07T06:51:16.205896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415903+0000","last_change":"2024-03-07T06:51:15.208063+0000","last_active":"2024-03-07T06:51:17.415903+0000","last_peered":"2024-03-07T06:51:17.415903+0000","last_clean":"2024-03-07T06:51:17.415903+0000","last_became_active":"2024-03-07T06:51:15.207782+0000","last_became_peered":"2024-03-07T06:51:15.207782+0000","last_unstale":"2024-03-07T06:51:17.415903+0000","last_undegraded":"2024-03-07T06:51:17.415903+0000","last_fullsized":"2024-03-07T06:51:17.415903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:02:55.966834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189333+0000","last_change":"2024-03-07T06:51:14.156560+0000","last_active":"2024-03-07T06:51:16.189333+0000","last_peered":"2024-03-07T06:51:16.189333+0000","last_clean":"2024-03-07T06:51:16.189333+0000","last_became_active":"2024-03-07T06:51:14.156040+0000","last_became_peered":"2024-03-07T06:51:14.156040+0000","last_unstale":"2024-03-07T06:51:16.189333+0000","last_undegraded":"2024-03-07T06:51:16.189333+0000","last_fullsized":"2024-03-07T06:51:16.189333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:07:44.222048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205750+0000","last_change":"2024-03-07T06:51:15.180806+0000","last_active":"2024-03-07T06:51:16.205750+0000","last_peered":"2024-03-07T06:51:16.205750+0000","last_clean":"2024-03-07T06:51:16.205750+0000","last_became_active":"2024-03-07T06:51:15.180402+0000","last_became_peered":"2024-03-07T06:51:15.180402+0000","last_unstale":"2024-03-07T06:51:16.205750+0000","last_undegraded":"2024-03-07T06:51:16.205750+0000","last_fullsized":"2024-03-07T06:51:16.205750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:38:05.514621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205976+0000","last_change":"2024-03-07T06:51:14.168110+0000","last_active":"2024-03-07T06:51:16.205976+0000","last_peered":"2024-03-07T06:51:16.205976+0000","last_clean":"2024-03-07T06:51:16.205976+0000","last_became_active":"2024-03-07T06:51:14.167759+0000","last_became_peered":"2024-03-07T06:51:14.167759+0000","last_unstale":"2024-03-07T06:51:16.205976+0000","last_undegraded":"2024-03-07T06:51:16.205976+0000","last_fullsized":"2024-03-07T06:51:16.205976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:03.695138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.416085+0000","last_change":"2024-03-07T06:51:15.199832+0000","last_active":"2024-03-07T06:51:17.416085+0000","last_peered":"2024-03-07T06:51:17.416085+0000","last_clean":"2024-03-07T06:51:17.416085+0000","last_became_active":"2024-03-07T06:51:15.199572+0000","last_became_peered":"2024-03-07T06:51:15.199572+0000","last_unstale":"2024-03-07T06:51:17.416085+0000","last_undegraded":"2024-03-07T06:51:17.416085+0000","last_fullsized":"2024-03-07T06:51:17.416085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:41:44.654410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193650+0000","last_change":"2024-03-07T06:51:14.173549+0000","last_active":"2024-03-07T06:51:16.193650+0000","last_peered":"2024-03-07T06:51:16.193650+0000","last_clean":"2024-03-07T06:51:16.193650+0000","last_became_active":"2024-03-07T06:51:14.173251+0000","last_became_peered":"2024-03-07T06:51:14.173251+0000","last_unstale":"2024-03-07T06:51:16.193650+0000","last_undegraded":"2024-03-07T06:51:16.193650+0000","last_fullsized":"2024-03-07T06:51:16.193650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:55:09.764204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.413748+0000","last_change":"2024-03-07T06:51:15.195178+0000","last_active":"2024-03-07T06:51:17.413748+0000","last_peered":"2024-03-07T06:51:17.413748+0000","last_clean":"2024-03-07T06:51:17.413748+0000","last_became_active":"2024-03-07T06:51:15.194907+0000","last_became_peered":"2024-03-07T06:51:15.194907+0000","last_unstale":"2024-03-07T06:51:17.413748+0000","last_undegraded":"2024-03-07T06:51:17.413748+0000","last_fullsized":"2024-03-07T06:51:17.413748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188590+0000","last_change":"2024-03-07T06:51:14.156566+0000","last_active":"2024-03-07T06:51:16.188590+0000","last_peered":"2024-03-07T06:51:16.188590+0000","last_clean":"2024-03-07T06:51:16.188590+0000","last_became_active":"2024-03-07T06:51:14.156261+0000","last_became_peered":"2024-03-07T06:51:14.156261+0000","last_unstale":"2024-03-07T06:51:16.188590+0000","last_undegraded":"2024-03-07T06:51:16.188590+0000","last_fullsized":"2024-03-07T06:51:16.188590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:07:10.283090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.193424+0000","last_change":"2024-03-07T06:51:15.183065+0000","last_active":"2024-03-07T06:51:16.193424+0000","last_peered":"2024-03-07T06:51:16.193424+0000","last_clean":"2024-03-07T06:51:16.193424+0000","last_became_active":"2024-03-07T06:51:15.182189+0000","last_became_peered":"2024-03-07T06:51:15.182189+0000","last_unstale":"2024-03-07T06:51:16.193424+0000","last_undegraded":"2024-03-07T06:51:16.193424+0000","last_fullsized":"2024-03-07T06:51:16.193424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:05:21.883987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205102+0000","last_change":"2024-03-07T06:51:14.169705+0000","last_active":"2024-03-07T06:51:16.205102+0000","last_peered":"2024-03-07T06:51:16.205102+0000","last_clean":"2024-03-07T06:51:16.205102+0000","last_became_active":"2024-03-07T06:51:14.169444+0000","last_became_peered":"2024-03-07T06:51:14.169444+0000","last_unstale":"2024-03-07T06:51:16.205102+0000","last_undegraded":"2024-03-07T06:51:16.205102+0000","last_fullsized":"2024-03-07T06:51:16.205102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:52:34.546532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415220+0000","last_change":"2024-03-07T06:51:15.197549+0000","last_active":"2024-03-07T06:51:17.415220+0000","last_peered":"2024-03-07T06:51:17.415220+0000","last_clean":"2024-03-07T06:51:17.415220+0000","last_became_active":"2024-03-07T06:51:15.195020+0000","last_became_peered":"2024-03-07T06:51:15.195020+0000","last_unstale":"2024-03-07T06:51:17.415220+0000","last_undegraded":"2024-03-07T06:51:17.415220+0000","last_fullsized":"2024-03-07T06:51:17.415220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:55:58.007185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.205415+0000","last_change":"2024-03-07T06:51:14.170410+0000","last_active":"2024-03-07T06:51:16.205415+0000","last_peered":"2024-03-07T06:51:16.205415+0000","last_clean":"2024-03-07T06:51:16.205415+0000","last_became_active":"2024-03-07T06:51:14.170043+0000","last_became_peered":"2024-03-07T06:51:14.170043+0000","last_unstale":"2024-03-07T06:51:16.205415+0000","last_undegraded":"2024-03-07T06:51:16.205415+0000","last_fullsized":"2024-03-07T06:51:16.205415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:26:52.619164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415636+0000","last_change":"2024-03-07T06:51:15.193076+0000","last_active":"2024-03-07T06:51:17.415636+0000","last_peered":"2024-03-07T06:51:17.415636+0000","last_clean":"2024-03-07T06:51:17.415636+0000","last_became_active":"2024-03-07T06:51:15.192700+0000","last_became_peered":"2024-03-07T06:51:15.192700+0000","last_unstale":"2024-03-07T06:51:17.415636+0000","last_undegraded":"2024-03-07T06:51:17.415636+0000","last_fullsized":"2024-03-07T06:51:17.415636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:31:20.324076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.191169+0000","last_change":"2024-03-07T06:51:14.173220+0000","last_active":"2024-03-07T06:51:16.191169+0000","last_peered":"2024-03-07T06:51:16.191169+0000","last_clean":"2024-03-07T06:51:16.191169+0000","last_became_active":"2024-03-07T06:51:14.169687+0000","last_became_peered":"2024-03-07T06:51:14.169687+0000","last_unstale":"2024-03-07T06:51:16.191169+0000","last_undegraded":"2024-03-07T06:51:16.191169+0000","last_fullsized":"2024-03-07T06:51:16.191169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:02:40.568300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.192059+0000","last_change":"2024-03-07T06:51:15.205150+0000","last_active":"2024-03-07T06:51:16.192059+0000","last_peered":"2024-03-07T06:51:16.192059+0000","last_clean":"2024-03-07T06:51:16.192059+0000","last_became_active":"2024-03-07T06:51:15.204435+0000","last_became_peered":"2024-03-07T06:51:15.204435+0000","last_unstale":"2024-03-07T06:51:16.192059+0000","last_undegraded":"2024-03-07T06:51:16.192059+0000","last_fullsized":"2024-03-07T06:51:16.192059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:44.556661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415532+0000","last_change":"2024-03-07T06:51:15.214214+0000","last_active":"2024-03-07T06:51:17.415532+0000","last_peered":"2024-03-07T06:51:17.415532+0000","last_clean":"2024-03-07T06:51:17.415532+0000","last_became_active":"2024-03-07T06:51:15.209505+0000","last_became_peered":"2024-03-07T06:51:15.209505+0000","last_unstale":"2024-03-07T06:51:17.415532+0000","last_undegraded":"2024-03-07T06:51:17.415532+0000","last_fullsized":"2024-03-07T06:51:17.415532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:04:56.465649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.188962+0000","last_change":"2024-03-07T06:51:14.172650+0000","last_active":"2024-03-07T06:51:16.188962+0000","last_peered":"2024-03-07T06:51:16.188962+0000","last_clean":"2024-03-07T06:51:16.188962+0000","last_became_active":"2024-03-07T06:51:14.170441+0000","last_became_peered":"2024-03-07T06:51:14.170441+0000","last_unstale":"2024-03-07T06:51:16.188962+0000","last_undegraded":"2024-03-07T06:51:16.188962+0000","last_fullsized":"2024-03-07T06:51:16.188962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:05:20.871811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415022+0000","last_change":"2024-03-07T06:51:15.204124+0000","last_active":"2024-03-07T06:51:17.415022+0000","last_peered":"2024-03-07T06:51:17.415022+0000","last_clean":"2024-03-07T06:51:17.415022+0000","last_became_active":"2024-03-07T06:51:15.203713+0000","last_became_peered":"2024-03-07T06:51:15.203713+0000","last_unstale":"2024-03-07T06:51:17.415022+0000","last_undegraded":"2024-03-07T06:51:17.415022+0000","last_fullsized":"2024-03-07T06:51:17.415022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:34:40.365080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:16.189015+0000","last_change":"2024-03-07T06:51:14.167422+0000","last_active":"2024-03-07T06:51:16.189015+0000","last_peered":"2024-03-07T06:51:16.189015+0000","last_clean":"2024-03-07T06:51:16.189015+0000","last_became_active":"2024-03-07T06:51:14.167152+0000","last_became_peered":"2024-03-07T06:51:14.167152+0000","last_unstale":"2024-03-07T06:51:16.189015+0000","last_undegraded":"2024-03-07T06:51:16.189015+0000","last_fullsized":"2024-03-07T06:51:16.189015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:40:50.922964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412609+0000","last_change":"2024-03-07T06:51:15.210848+0000","last_active":"2024-03-07T06:51:17.412609+0000","last_peered":"2024-03-07T06:51:17.412609+0000","last_clean":"2024-03-07T06:51:17.412609+0000","last_became_active":"2024-03-07T06:51:15.210526+0000","last_became_peered":"2024-03-07T06:51:15.210526+0000","last_unstale":"2024-03-07T06:51:17.412609+0000","last_undegraded":"2024-03-07T06:51:17.412609+0000","last_fullsized":"2024-03-07T06:51:17.412609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:40:57.934235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415818+0000","last_change":"2024-03-07T06:51:14.163311+0000","last_active":"2024-03-07T06:51:17.415818+0000","last_peered":"2024-03-07T06:51:17.415818+0000","last_clean":"2024-03-07T06:51:17.415818+0000","last_became_active":"2024-03-07T06:51:14.161332+0000","last_became_peered":"2024-03-07T06:51:14.161332+0000","last_unstale":"2024-03-07T06:51:17.415818+0000","last_undegraded":"2024-03-07T06:51:17.415818+0000","last_fullsized":"2024-03-07T06:51:17.415818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.415995+0000","last_change":"2024-03-07T06:51:15.200486+0000","last_active":"2024-03-07T06:51:17.415995+0000","last_peered":"2024-03-07T06:51:17.415995+0000","last_clean":"2024-03-07T06:51:17.415995+0000","last_became_active":"2024-03-07T06:51:15.200171+0000","last_became_peered":"2024-03-07T06:51:15.200171+0000","last_unstale":"2024-03-07T06:51:17.415995+0000","last_undegraded":"2024-03-07T06:51:17.415995+0000","last_fullsized":"2024-03-07T06:51:17.415995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:14.122791+0000","last_clean_scrub_stamp":"2024-03-07T06:51:14.122791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:22:38.973224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-07T06:51:17.412810+0000","last_change":"2024-03-07T06:51:14.169164+0000","last_active":"2024-03-07T06:51:17.412810+0000","last_peered":"2024-03-07T06:51:17.412810+0000","last_clean":"2024-03-07T06:51:17.412810+0000","last_became_active":"2024-03-07T06:51:14.168662+0000","last_became_peered":"2024-03-07T06:51:14.168662+0000","last_unstale":"2024-03-07T06:51:17.412810+0000","last_undegraded":"2024-03-07T06:51:17.412810+0000","last_fullsized":"2024-03-07T06:51:17.412810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:13.110616+0000","last_clean_scrub_stamp":"2024-03-07T06:51:13.110616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:30:11.789614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:51:30.604 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-07T06:51:30.604 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-07T06:51:30.604 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-07T06:51:30.604 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-07T06:51:30.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.840+0000 7fb777e00640 1 Processor -- start 2024-03-07T06:51:30.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.841+0000 7fb777e00640 1 -- start start 2024-03-07T06:51:30.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.842+0000 7fb777e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7780cd4c0 0x7fb77815aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:30.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.842+0000 7fb777e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb778072830 0x7fb778072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:30.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.842+0000 7fb777e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb77806e850 con 0x7fb77815aaf0 2024-03-07T06:51:30.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.842+0000 7fb777e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7780731e0 con 0x7fb778072830 2024-03-07T06:51:30.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.842+0000 7fb777e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77806eb30 con 0x7fb7780cd4c0 2024-03-07T06:51:30.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.843+0000 7fb76fe00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7780cd4c0 0x7fb77815aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:30.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.843+0000 7fb777400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb778072830 0x7fb778072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:30.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.843+0000 7fb777400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb778072830 0x7fb778072ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:52900/0 (socket says 172.21.15.133:52900) 2024-03-07T06:51:30.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.843+0000 7fb777400640 1 -- 172.21.15.133:0/3298002388 learned_addr learned my addr 172.21.15.133:0/3298002388 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:30.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.844+0000 7fb777400640 1 -- 172.21.15.133:0/3298002388 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7780cd4c0 msgr2=0x7fb77815aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:30.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.844+0000 7fb777400640 1 --2- 172.21.15.133:0/3298002388 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7780cd4c0 0x7fb77815aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.844+0000 7fb777400640 1 -- 172.21.15.133:0/3298002388 >> v1:172.21.15.161:6789/0 conn(0x7fb77815aaf0 legacy=0x7fb77806e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:30.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.844+0000 7fb777400640 1 -- 172.21.15.133:0/3298002388 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77806ee10 con 0x7fb778072830 2024-03-07T06:51:30.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.845+0000 7fb777400640 1 --2- 172.21.15.133:0/3298002388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb778072830 0x7fb778072ca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb770009e50 tx=0x7fb77002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0dd9497031bcd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:30.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.845+0000 7fb776000640 1 -- 172.21.15.133:0/3298002388 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77002cb00 con 0x7fb778072830 2024-03-07T06:51:30.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.845+0000 7fb776000640 1 -- 172.21.15.133:0/3298002388 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77002cca0 con 0x7fb778072830 2024-03-07T06:51:30.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.845+0000 7fb776000640 1 -- 172.21.15.133:0/3298002388 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770034610 con 0x7fb778072830 2024-03-07T06:51:30.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.846+0000 7fb777e00640 1 -- 172.21.15.133:0/3298002388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb778072830 msgr2=0x7fb778072ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:30.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.846+0000 7fb777e00640 1 --2- 172.21.15.133:0/3298002388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb778072830 0x7fb778072ca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb770009e50 tx=0x7fb77002bf60 comp rx=0 tx=0).stop 2024-03-07T06:51:30.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.846+0000 7fb777e00640 1 -- 172.21.15.133:0/3298002388 shutdown_connections 2024-03-07T06:51:30.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.846+0000 7fb777e00640 1 --2- 172.21.15.133:0/3298002388 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb778072830 0x7fb778072ca0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.846+0000 7fb777e00640 1 --2- 172.21.15.133:0/3298002388 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7780cd4c0 0x7fb77815aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.846+0000 7fb777e00640 1 -- 172.21.15.133:0/3298002388 >> 172.21.15.133:0/3298002388 conn(0x7fb7780b2e90 msgr2=0x7fb7780b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:30.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.846+0000 7fb777e00640 1 -- 172.21.15.133:0/3298002388 shutdown_connections 2024-03-07T06:51:30.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.846+0000 7fb777e00640 1 -- 172.21.15.133:0/3298002388 wait complete. 2024-03-07T06:51:30.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.847+0000 7fb777e00640 1 Processor -- start 2024-03-07T06:51:30.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.848+0000 7fb777e00640 1 -- start start 2024-03-07T06:51:30.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.848+0000 7fb777e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb778072830 0x7fb778147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:30.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.848+0000 7fb777e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7780cd4c0 0x7fb778151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:30.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.848+0000 7fb777e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb77815aaf0 0x7fb7780add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:30.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.848+0000 7fb777e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77815b410 con 0x7fb77815aaf0 2024-03-07T06:51:30.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.848+0000 7fb777e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77806f0f0 con 0x7fb778072830 2024-03-07T06:51:30.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.848+0000 7fb777e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77815b590 con 0x7fb7780cd4c0 2024-03-07T06:51:30.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.849+0000 7fb777400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb77815aaf0 0x7fb7780add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:30.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.849+0000 7fb777400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb77815aaf0 0x7fb7780add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55842/0 (socket says 172.21.15.133:55842) 2024-03-07T06:51:30.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.849+0000 7fb777400640 1 -- 172.21.15.133:0/1970241341 learned_addr learned my addr 172.21.15.133:0/1970241341 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:30.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.849+0000 7fb776a00640 1 --2- 172.21.15.133:0/1970241341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb778072830 0x7fb778147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:30.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.849+0000 7fb76fe00640 1 --2- 172.21.15.133:0/1970241341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7780cd4c0 0x7fb778151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:30.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.849+0000 7fb777400640 1 -- 172.21.15.133:0/1970241341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7780cd4c0 msgr2=0x7fb778151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:30.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.849+0000 7fb777400640 1 --2- 172.21.15.133:0/1970241341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7780cd4c0 0x7fb778151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.849+0000 7fb777400640 1 -- 172.21.15.133:0/1970241341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb778072830 msgr2=0x7fb778147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:30.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.849+0000 7fb777400640 1 --2- 172.21.15.133:0/1970241341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb778072830 0x7fb778147d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:30.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.849+0000 7fb777400640 1 -- 172.21.15.133:0/1970241341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7780b0390 con 0x7fb77815aaf0 2024-03-07T06:51:30.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.849+0000 7fb777400640 1 --2- 172.21.15.133:0/1970241341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb77815aaf0 0x7fb7780add90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb770036040 tx=0x7fb770004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:30.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.850+0000 7fb76ea00640 1 -- 172.21.15.133:0/1970241341 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770043020 con 0x7fb77815aaf0 2024-03-07T06:51:30.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.850+0000 7fb76ea00640 1 -- 172.21.15.133:0/1970241341 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77002ce50 con 0x7fb77815aaf0 2024-03-07T06:51:30.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.850+0000 7fb76ea00640 1 -- 172.21.15.133:0/1970241341 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770009510 con 0x7fb77815aaf0 2024-03-07T06:51:30.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.850+0000 7fb777e00640 1 -- 172.21.15.133:0/1970241341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7781f8200 con 0x7fb77815aaf0 2024-03-07T06:51:30.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.850+0000 7fb777e00640 1 -- 172.21.15.133:0/1970241341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7780b0930 con 0x7fb77815aaf0 2024-03-07T06:51:30.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.851+0000 7fb777e00640 1 -- 172.21.15.133:0/1970241341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7780b0b30 con 0x7fb77815aaf0 2024-03-07T06:51:30.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.852+0000 7fb76ea00640 1 -- 172.21.15.133:0/1970241341 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb770046020 con 0x7fb77815aaf0 2024-03-07T06:51:30.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.853+0000 7fb76ea00640 1 --2- 172.21.15.133:0/1970241341 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb75406b4d0 0x7fb75406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:30.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.853+0000 7fb76ea00640 1 -- 172.21.15.133:0/1970241341 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7700ac6f0 con 0x7fb77815aaf0 2024-03-07T06:51:30.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.854+0000 7fb776a00640 1 --2- 172.21.15.133:0/1970241341 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb75406b4d0 0x7fb75406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:30.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.855+0000 7fb776a00640 1 --2- 172.21.15.133:0/1970241341 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb75406b4d0 0x7fb75406d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb768000f60 tx=0x7fb768009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:30.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:30.858+0000 7fb76ea00640 1 -- 172.21.15.133:0/1970241341 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb77004d030 con 0x7fb77815aaf0 2024-03-07T06:51:31.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.075+0000 7fb777e00640 1 -- 172.21.15.133:0/1970241341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb778062c90 con 0x7fb77815aaf0 2024-03-07T06:51:31.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.075+0000 7fb76ea00640 1 -- 172.21.15.133:0/1970241341 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb77003c350 con 0x7fb77815aaf0 2024-03-07T06:51:31.075 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:31.075 INFO:teuthology.orchestra.run.smithi133.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-07T06:51:31.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.078+0000 7fb777e00640 1 -- 172.21.15.133:0/1970241341 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb75406b4d0 msgr2=0x7fb75406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:31.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.078+0000 7fb777e00640 1 --2- 172.21.15.133:0/1970241341 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb75406b4d0 0x7fb75406d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb768000f60 tx=0x7fb768009040 comp rx=0 tx=0).stop 2024-03-07T06:51:31.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.078+0000 7fb777e00640 1 -- 172.21.15.133:0/1970241341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb77815aaf0 msgr2=0x7fb7780add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:31.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.078+0000 7fb777e00640 1 --2- 172.21.15.133:0/1970241341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb77815aaf0 0x7fb7780add90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb770036040 tx=0x7fb770004a40 comp rx=0 tx=0).stop 2024-03-07T06:51:31.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.078+0000 7fb777e00640 1 -- 172.21.15.133:0/1970241341 shutdown_connections 2024-03-07T06:51:31.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.078+0000 7fb777e00640 1 --2- 172.21.15.133:0/1970241341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb77815aaf0 0x7fb7780add90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.078+0000 7fb777e00640 1 --2- 172.21.15.133:0/1970241341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7780cd4c0 0x7fb778151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.078+0000 7fb777e00640 1 --2- 172.21.15.133:0/1970241341 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb75406b4d0 0x7fb75406d990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.078+0000 7fb777e00640 1 --2- 172.21.15.133:0/1970241341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb778072830 0x7fb778147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.078+0000 7fb777e00640 1 -- 172.21.15.133:0/1970241341 >> 172.21.15.133:0/1970241341 conn(0x7fb7780b2e90 msgr2=0x7fb7780cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:31.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.078+0000 7fb777e00640 1 -- 172.21.15.133:0/1970241341 shutdown_connections 2024-03-07T06:51:31.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.079+0000 7fb777e00640 1 -- 172.21.15.133:0/1970241341 wait complete. 2024-03-07T06:51:31.095 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-07T06:51:31.096 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:31.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.305+0000 7f09d7800640 1 Processor -- start 2024-03-07T06:51:31.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.305+0000 7f09d7800640 1 -- start start 2024-03-07T06:51:31.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.306+0000 7f09d7800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f09d015fbb0 0x7f09d00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:31.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.306+0000 7f09d7800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09d0072950 0x7f09d00cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:31.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.306+0000 7f09d7800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d00731b0 con 0x7f09d015ff90 2024-03-07T06:51:31.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.306+0000 7f09d7800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d0072f60 con 0x7f09d015fbb0 2024-03-07T06:51:31.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.306+0000 7f09d7800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d0073490 con 0x7f09d0072950 2024-03-07T06:51:31.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.307+0000 7f09d4e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f09d015fbb0 0x7f09d00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:31.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.307+0000 7f09d4e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f09d015fbb0 0x7f09d00cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:52922/0 (socket says 172.21.15.133:52922) 2024-03-07T06:51:31.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.307+0000 7f09d4e00640 1 -- 172.21.15.133:0/3254084069 learned_addr learned my addr 172.21.15.133:0/3254084069 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:31.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.307+0000 7f09cfe00640 1 --2- 172.21.15.133:0/3254084069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09d0072950 0x7f09d00cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:31.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.308+0000 7f09cfe00640 1 -- 172.21.15.133:0/3254084069 >> v2:172.21.15.161:3300/0 conn(0x7f09d015fbb0 msgr2=0x7f09d00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:31.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.308+0000 7f09cfe00640 1 --2- 172.21.15.133:0/3254084069 >> v2:172.21.15.161:3300/0 conn(0x7f09d015fbb0 0x7f09d00cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.308+0000 7f09cfe00640 1 -- 172.21.15.133:0/3254084069 >> v1:172.21.15.161:6789/0 conn(0x7f09d015ff90 legacy=0x7f09d00acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:31.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.308+0000 7f09cfe00640 1 -- 172.21.15.133:0/3254084069 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d0072150 con 0x7f09d0072950 2024-03-07T06:51:31.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.308+0000 7f09d4e00640 1 --2- 172.21.15.133:0/3254084069 >> v2:172.21.15.161:3300/0 conn(0x7f09d015fbb0 0x7f09d00cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T06:51:31.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.309+0000 7f09cfe00640 1 --2- 172.21.15.133:0/3254084069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09d0072950 0x7f09d00cdee0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f09d00730e0 tx=0x7f09bc02b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61985650aa35d11f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:31.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.309+0000 7f09cf400640 1 -- 172.21.15.133:0/3254084069 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc031770 con 0x7f09d0072950 2024-03-07T06:51:31.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.310+0000 7f09cf400640 1 -- 172.21.15.133:0/3254084069 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc037070 con 0x7f09d0072950 2024-03-07T06:51:31.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.310+0000 7f09cf400640 1 -- 172.21.15.133:0/3254084069 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc031b30 con 0x7f09d0072950 2024-03-07T06:51:31.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.310+0000 7f09d7800640 1 -- 172.21.15.133:0/3254084069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09d0072950 msgr2=0x7f09d00cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:31.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.310+0000 7f09d7800640 1 --2- 172.21.15.133:0/3254084069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09d0072950 0x7f09d00cdee0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f09d00730e0 tx=0x7f09bc02b810 comp rx=0 tx=0).stop 2024-03-07T06:51:31.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.310+0000 7f09d7800640 1 -- 172.21.15.133:0/3254084069 shutdown_connections 2024-03-07T06:51:31.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.310+0000 7f09d7800640 1 --2- 172.21.15.133:0/3254084069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09d0072950 0x7f09d00cdee0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.310+0000 7f09d7800640 1 --2- 172.21.15.133:0/3254084069 >> v2:172.21.15.161:3300/0 conn(0x7f09d015fbb0 0x7f09d00cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.310+0000 7f09d7800640 1 -- 172.21.15.133:0/3254084069 >> 172.21.15.133:0/3254084069 conn(0x7f09d00b3120 msgr2=0x7f09d00b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:31.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.310+0000 7f09d7800640 1 -- 172.21.15.133:0/3254084069 shutdown_connections 2024-03-07T06:51:31.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.310+0000 7f09d7800640 1 -- 172.21.15.133:0/3254084069 wait complete. 2024-03-07T06:51:31.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.312+0000 7f09d7800640 1 Processor -- start 2024-03-07T06:51:31.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.312+0000 7f09d7800640 1 -- start start 2024-03-07T06:51:31.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.313+0000 7f09d7800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d015fbb0 0x7f09d006e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:31.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.313+0000 7f09d7800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09d015ff90 0x7f09d015c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:31.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.313+0000 7f09d7800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09d015c860 0x7f09d014d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:31.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.313+0000 7f09d7800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00b1a00 con 0x7f09d015fbb0 2024-03-07T06:51:31.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.313+0000 7f09d7800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d0160980 con 0x7f09d015ff90 2024-03-07T06:51:31.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.313+0000 7f09d7800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d01603e0 con 0x7f09d015c860 2024-03-07T06:51:31.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.313+0000 7f09d4e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d015fbb0 0x7f09d006e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.313+0000 7f09d4e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d015fbb0 0x7f09d006e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55856/0 (socket says 172.21.15.133:55856) 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.313+0000 7f09d4e00640 1 -- 172.21.15.133:0/2351089137 learned_addr learned my addr 172.21.15.133:0/2351089137 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.314+0000 7f09cfe00640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09d015ff90 0x7f09d015c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.314+0000 7f09d4e00640 1 -- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09d015c860 msgr2=0x7f09d014d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.314+0000 7f09d5800640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09d015c860 0x7f09d014d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.314+0000 7f09d4e00640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09d015c860 0x7f09d014d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.314+0000 7f09d4e00640 1 -- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09d015ff90 msgr2=0x7f09d015c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.314+0000 7f09d4e00640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09d015ff90 0x7f09d015c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.314+0000 7f09d4e00640 1 -- 172.21.15.133:0/2351089137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d006e7a0 con 0x7f09d015fbb0 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.314+0000 7f09cfe00640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09d015ff90 0x7f09d015c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.314+0000 7f09d4e00640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d015fbb0 0x7f09d006e030 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f09c000ea00 tx=0x7f09c000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.314+0000 7f09ccc00640 1 -- 172.21.15.133:0/2351089137 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0017070 con 0x7f09d015fbb0 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.314+0000 7f09ccc00640 1 -- 172.21.15.133:0/2351089137 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c000ca60 con 0x7f09d015fbb0 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.314+0000 7f09ccc00640 1 -- 172.21.15.133:0/2351089137 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c000ce10 con 0x7f09d015fbb0 2024-03-07T06:51:31.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.314+0000 7f09d7800640 1 -- 172.21.15.133:0/2351089137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d006ea80 con 0x7f09d015fbb0 2024-03-07T06:51:31.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.314+0000 7f09d7800640 1 -- 172.21.15.133:0/2351089137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d006f040 con 0x7f09d015fbb0 2024-03-07T06:51:31.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.316+0000 7f09d7800640 1 -- 172.21.15.133:0/2351089137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d015cc70 con 0x7f09d015fbb0 2024-03-07T06:51:31.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.316+0000 7f09ccc00640 1 -- 172.21.15.133:0/2351089137 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f09c001a070 con 0x7f09d015fbb0 2024-03-07T06:51:31.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.317+0000 7f09ccc00640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f09b006b500 0x7f09b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:31.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.317+0000 7f09ccc00640 1 -- 172.21.15.133:0/2351089137 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09c008a4a0 con 0x7f09d015fbb0 2024-03-07T06:51:31.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.318+0000 7f09cfe00640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f09b006b500 0x7f09b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:31.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.319+0000 7f09cfe00640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f09b006b500 0x7f09b006d9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f09bc036000 tx=0x7f09bc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:31.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.322+0000 7f09ccc00640 1 -- 172.21.15.133:0/2351089137 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f09d015cc70 con 0x7f09d015fbb0 2024-03-07T06:51:31.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.519+0000 7f09d7800640 1 -- 172.21.15.133:0/2351089137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09d0062c90 con 0x7f09d015fbb0 2024-03-07T06:51:31.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.521+0000 7f09ccc00640 1 -- 172.21.15.133:0/2351089137 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f09c0059320 con 0x7f09d015fbb0 2024-03-07T06:51:31.521 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:31.522 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":21,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:16.174650+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:31.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.524+0000 7f09d7800640 1 -- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f09b006b500 msgr2=0x7f09b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:31.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.524+0000 7f09d7800640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f09b006b500 0x7f09b006d9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f09bc036000 tx=0x7f09bc008030 comp rx=0 tx=0).stop 2024-03-07T06:51:31.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.524+0000 7f09d7800640 1 -- 172.21.15.133:0/2351089137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d015fbb0 msgr2=0x7f09d006e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:31.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.524+0000 7f09d7800640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d015fbb0 0x7f09d006e030 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f09c000ea00 tx=0x7f09c000eec0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.524+0000 7f09d7800640 1 -- 172.21.15.133:0/2351089137 shutdown_connections 2024-03-07T06:51:31.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.524+0000 7f09d7800640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f09d015c860 0x7f09d014d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.524+0000 7f09d7800640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f09b006b500 0x7f09b006d9c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.524+0000 7f09d7800640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f09d015ff90 0x7f09d015c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.524+0000 7f09d7800640 1 --2- 172.21.15.133:0/2351089137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09d015fbb0 0x7f09d006e030 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.524+0000 7f09d7800640 1 -- 172.21.15.133:0/2351089137 >> 172.21.15.133:0/2351089137 conn(0x7f09d00b3120 msgr2=0x7f09d015e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:31.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.524+0000 7f09d7800640 1 -- 172.21.15.133:0/2351089137 shutdown_connections 2024-03-07T06:51:31.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.524+0000 7f09d7800640 1 -- 172.21.15.133:0/2351089137 wait complete. 2024-03-07T06:51:31.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:51:12.858333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:51:13.615348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:51:31.541 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:51:31.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.751+0000 7fc9a2e00640 1 Processor -- start 2024-03-07T06:51:31.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.752+0000 7fc9a2e00640 1 -- start start 2024-03-07T06:51:31.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.753+0000 7fc9a2e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc99c15e250 0x7fc99c0b9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:31.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.753+0000 7fc9a2e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc99c0d2160 0x7fc99c0d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:31.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.753+0000 7fc9a2e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99c0b7f80 con 0x7fc99c071ca0 2024-03-07T06:51:31.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.753+0000 7fc9a2e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99c0d3b20 con 0x7fc99c15e250 2024-03-07T06:51:31.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.753+0000 7fc9a2e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c070c00 con 0x7fc99c0d2160 2024-03-07T06:51:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.754+0000 7fc9a1a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc99c15e250 0x7fc99c0b9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.754+0000 7fc9a2400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc99c0d2160 0x7fc99c0d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.754+0000 7fc9a1a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc99c15e250 0x7fc99c0b9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:52942/0 (socket says 172.21.15.133:52942) 2024-03-07T06:51:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.754+0000 7fc9a1a00640 1 -- 172.21.15.133:0/3626679765 learned_addr learned my addr 172.21.15.133:0/3626679765 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.754+0000 7fc9a1a00640 1 -- 172.21.15.133:0/3626679765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc99c0d2160 msgr2=0x7fc99c0d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.754+0000 7fc9a1a00640 1 --2- 172.21.15.133:0/3626679765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc99c0d2160 0x7fc99c0d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.754+0000 7fc9a1a00640 1 -- 172.21.15.133:0/3626679765 >> v1:172.21.15.161:6789/0 conn(0x7fc99c071ca0 legacy=0x7fc99c0703b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.754+0000 7fc9a1a00640 1 -- 172.21.15.133:0/3626679765 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c070ee0 con 0x7fc99c15e250 2024-03-07T06:51:31.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.756+0000 7fc9a1a00640 1 --2- 172.21.15.133:0/3626679765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc99c15e250 0x7fc99c0b9a90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009870 tx=0x7fc98c02ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe494c7e8358af09 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:31.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.756+0000 7fc99be00640 1 -- 172.21.15.133:0/3626679765 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c02f830 con 0x7fc99c15e250 2024-03-07T06:51:31.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.756+0000 7fc99be00640 1 -- 172.21.15.133:0/3626679765 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c02f9d0 con 0x7fc99c15e250 2024-03-07T06:51:31.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.756+0000 7fc99be00640 1 -- 172.21.15.133:0/3626679765 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c02fda0 con 0x7fc99c15e250 2024-03-07T06:51:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.757+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3626679765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc99c15e250 msgr2=0x7fc99c0b9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.757+0000 7fc9a2e00640 1 --2- 172.21.15.133:0/3626679765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc99c15e250 0x7fc99c0b9a90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc98c009870 tx=0x7fc98c02ba10 comp rx=0 tx=0).stop 2024-03-07T06:51:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.757+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3626679765 shutdown_connections 2024-03-07T06:51:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.757+0000 7fc9a2e00640 1 --2- 172.21.15.133:0/3626679765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc99c0d2160 0x7fc99c0d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.757+0000 7fc9a2e00640 1 --2- 172.21.15.133:0/3626679765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc99c15e250 0x7fc99c0b9a90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.757+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3626679765 >> 172.21.15.133:0/3626679765 conn(0x7fc99c0d3e40 msgr2=0x7fc99c0d4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.757+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3626679765 shutdown_connections 2024-03-07T06:51:31.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.757+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3626679765 wait complete. 2024-03-07T06:51:31.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.759+0000 7fc9a2e00640 1 Processor -- start 2024-03-07T06:51:31.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.759+0000 7fc9a2e00640 1 -- start start 2024-03-07T06:51:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a2e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99c071ca0 0x7fc99c0cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a2e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc99c0d2160 0x7fc99c15a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a2e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc99c15e250 0x7fc99c15e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a2e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c071780 con 0x7fc99c071ca0 2024-03-07T06:51:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a2e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0714a0 con 0x7fc99c0d2160 2024-03-07T06:51:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a2e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0d2b10 con 0x7fc99c15e250 2024-03-07T06:51:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a1a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99c071ca0 0x7fc99c0cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a1a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99c071ca0 0x7fc99c0cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55872/0 (socket says 172.21.15.133:55872) 2024-03-07T06:51:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a1a00640 1 -- 172.21.15.133:0/3539685305 learned_addr learned my addr 172.21.15.133:0/3539685305 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a2400640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc99c15e250 0x7fc99c15e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:31.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a1000640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc99c0d2160 0x7fc99c15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a1a00640 1 -- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc99c15e250 msgr2=0x7fc99c15e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a1a00640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc99c15e250 0x7fc99c15e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a1a00640 1 -- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc99c0d2160 msgr2=0x7fc99c15a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a1a00640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc99c0d2160 0x7fc99c15a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a1a00640 1 -- 172.21.15.133:0/3539685305 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c1f3c90 con 0x7fc99c071ca0 2024-03-07T06:51:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a2400640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc99c15e250 0x7fc99c15e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.760+0000 7fc9a1000640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc99c0d2160 0x7fc99c15a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.761+0000 7fc9a1a00640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99c071ca0 0x7fc99c0cc470 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc98c000c00 tx=0x7fc98c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.761+0000 7fc999600640 1 -- 172.21.15.133:0/3539685305 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c037c40 con 0x7fc99c071ca0 2024-03-07T06:51:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.761+0000 7fc999600640 1 -- 172.21.15.133:0/3539685305 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c046070 con 0x7fc99c071ca0 2024-03-07T06:51:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.761+0000 7fc999600640 1 -- 172.21.15.133:0/3539685305 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c041400 con 0x7fc99c071ca0 2024-03-07T06:51:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.761+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3539685305 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99c1f3f70 con 0x7fc99c071ca0 2024-03-07T06:51:31.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.761+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3539685305 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99c1f4530 con 0x7fc99c071ca0 2024-03-07T06:51:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.762+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3539685305 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99c1f4910 con 0x7fc99c071ca0 2024-03-07T06:51:31.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.763+0000 7fc999600640 1 -- 172.21.15.133:0/3539685305 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc98c040070 con 0x7fc99c071ca0 2024-03-07T06:51:31.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.764+0000 7fc999600640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc97c06b4d0 0x7fc97c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:31.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.765+0000 7fc999600640 1 -- 172.21.15.133:0/3539685305 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc98c0ac720 con 0x7fc99c071ca0 2024-03-07T06:51:31.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.765+0000 7fc9a1000640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc97c06b4d0 0x7fc97c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:31.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.766+0000 7fc9a1000640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc97c06b4d0 0x7fc97c06d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0cdf10 tx=0x7fc988009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:31.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.770+0000 7fc999600640 1 -- 172.21.15.133:0/3539685305 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc99c1f4910 con 0x7fc99c071ca0 2024-03-07T06:51:31.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.965+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3539685305 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc99c062c90 con 0x7fc99c071ca0 2024-03-07T06:51:31.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.966+0000 7fc999600640 1 -- 172.21.15.133:0/3539685305 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc98c033030 con 0x7fc99c071ca0 2024-03-07T06:51:31.966 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:51:31.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.968+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc97c06b4d0 msgr2=0x7fc97c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.968+0000 7fc9a2e00640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc97c06b4d0 0x7fc97c06d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0cdf10 tx=0x7fc988009040 comp rx=0 tx=0).stop 2024-03-07T06:51:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.968+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3539685305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99c071ca0 msgr2=0x7fc99c0cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.968+0000 7fc9a2e00640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99c071ca0 0x7fc99c0cc470 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc98c000c00 tx=0x7fc98c004730 comp rx=0 tx=0).stop 2024-03-07T06:51:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.968+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3539685305 shutdown_connections 2024-03-07T06:51:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.968+0000 7fc9a2e00640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc99c15e250 0x7fc99c15e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.968+0000 7fc9a2e00640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc97c06b4d0 0x7fc97c06d990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.968+0000 7fc9a2e00640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc99c0d2160 0x7fc99c15a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.968+0000 7fc9a2e00640 1 --2- 172.21.15.133:0/3539685305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc99c071ca0 0x7fc99c0cc470 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.968+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3539685305 >> 172.21.15.133:0/3539685305 conn(0x7fc99c0d3e40 msgr2=0x7fc99c0b4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.969+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3539685305 shutdown_connections 2024-03-07T06:51:31.969 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:31.969+0000 7fc9a2e00640 1 -- 172.21.15.133:0/3539685305 wait complete. 2024-03-07T06:51:31.988 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:51:32.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.201+0000 7ff328a00640 1 Processor -- start 2024-03-07T06:51:32.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.201+0000 7ff328a00640 1 -- start start 2024-03-07T06:51:32.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.202+0000 7ff328a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3240cf610 0x7ff3240cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:32.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.202+0000 7ff328a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff3240cda20 0x7ff3240719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:32.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.202+0000 7ff328a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32406e150 con 0x7ff324145670 2024-03-07T06:51:32.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.203+0000 7ff328a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3240cfa70 con 0x7ff3240cda20 2024-03-07T06:51:32.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.203+0000 7ff328a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32406ed60 con 0x7ff3240cf610 2024-03-07T06:51:32.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.203+0000 7ff322800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff3240cda20 0x7ff3240719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:32.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.203+0000 7ff319400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3240cf610 0x7ff3240cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:32.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.203+0000 7ff322800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff3240cda20 0x7ff3240719a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:52960/0 (socket says 172.21.15.133:52960) 2024-03-07T06:51:32.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.203+0000 7ff322800640 1 -- 172.21.15.133:0/2648786223 learned_addr learned my addr 172.21.15.133:0/2648786223 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:32.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.204+0000 7ff322800640 1 -- 172.21.15.133:0/2648786223 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3240cf610 msgr2=0x7ff3240cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:32.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.204+0000 7ff322800640 1 --2- 172.21.15.133:0/2648786223 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3240cf610 0x7ff3240cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.204+0000 7ff322800640 1 -- 172.21.15.133:0/2648786223 >> v1:172.21.15.161:6789/0 conn(0x7ff324145670 legacy=0x7ff32415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:32.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.204+0000 7ff322800640 1 -- 172.21.15.133:0/2648786223 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32406e430 con 0x7ff3240cda20 2024-03-07T06:51:32.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.205+0000 7ff322800640 1 --2- 172.21.15.133:0/2648786223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3240cda20 0x7ff3240719a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009e50 tx=0x7ff31c02bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6177e068047dc249 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:32.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.205+0000 7ff321400640 1 -- 172.21.15.133:0/2648786223 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c02ca80 con 0x7ff3240cda20 2024-03-07T06:51:32.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.206+0000 7ff321400640 1 -- 172.21.15.133:0/2648786223 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c02cc20 con 0x7ff3240cda20 2024-03-07T06:51:32.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.206+0000 7ff321400640 1 -- 172.21.15.133:0/2648786223 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c0343f0 con 0x7ff3240cda20 2024-03-07T06:51:32.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.206+0000 7ff328a00640 1 -- 172.21.15.133:0/2648786223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3240cda20 msgr2=0x7ff3240719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:32.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.206+0000 7ff328a00640 1 --2- 172.21.15.133:0/2648786223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3240cda20 0x7ff3240719a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009e50 tx=0x7ff31c02bee0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.206+0000 7ff328a00640 1 -- 172.21.15.133:0/2648786223 shutdown_connections 2024-03-07T06:51:32.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.206+0000 7ff328a00640 1 --2- 172.21.15.133:0/2648786223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3240cda20 0x7ff3240719a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.206+0000 7ff328a00640 1 --2- 172.21.15.133:0/2648786223 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3240cf610 0x7ff3240cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.206+0000 7ff328a00640 1 -- 172.21.15.133:0/2648786223 >> 172.21.15.133:0/2648786223 conn(0x7ff3240b1d30 msgr2=0x7ff3240b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:32.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.206+0000 7ff328a00640 1 -- 172.21.15.133:0/2648786223 shutdown_connections 2024-03-07T06:51:32.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.206+0000 7ff328a00640 1 -- 172.21.15.133:0/2648786223 wait complete. 2024-03-07T06:51:32.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.208+0000 7ff328a00640 1 Processor -- start 2024-03-07T06:51:32.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.209+0000 7ff328a00640 1 -- start start 2024-03-07T06:51:32.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.210+0000 7ff328a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3240cda20 0x7ff32415bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:32.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.210+0000 7ff328a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3240cf610 0x7ff3241f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:32.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.210+0000 7ff328a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff324145670 0x7ff3241f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:32.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.210+0000 7ff328a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff324145a50 con 0x7ff324145670 2024-03-07T06:51:32.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.210+0000 7ff328a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32406e9f0 con 0x7ff3240cf610 2024-03-07T06:51:32.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.210+0000 7ff328a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff324145bd0 con 0x7ff3240cda20 2024-03-07T06:51:32.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.210+0000 7ff322800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff324145670 0x7ff3241f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:32.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.210+0000 7ff322800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff324145670 0x7ff3241f7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55874/0 (socket says 172.21.15.133:55874) 2024-03-07T06:51:32.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.210+0000 7ff322800640 1 -- 172.21.15.133:0/419729185 learned_addr learned my addr 172.21.15.133:0/419729185 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:32.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.210+0000 7ff319400640 1 --2- 172.21.15.133:0/419729185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3240cf610 0x7ff3241f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:32.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.210+0000 7ff321e00640 1 --2- 172.21.15.133:0/419729185 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3240cda20 0x7ff32415bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:32.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.211+0000 7ff322800640 1 -- 172.21.15.133:0/419729185 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3240cda20 msgr2=0x7ff32415bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:32.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.211+0000 7ff322800640 1 --2- 172.21.15.133:0/419729185 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3240cda20 0x7ff32415bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.211+0000 7ff322800640 1 -- 172.21.15.133:0/419729185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3240cf610 msgr2=0x7ff3241f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:32.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.211+0000 7ff322800640 1 --2- 172.21.15.133:0/419729185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3240cf610 0x7ff3241f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.211+0000 7ff322800640 1 -- 172.21.15.133:0/419729185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32415bff0 con 0x7ff324145670 2024-03-07T06:51:32.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.211+0000 7ff322800640 1 --2- 172.21.15.133:0/419729185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff324145670 0x7ff3241f7930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009f70 tx=0x7ff31c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:32.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.211+0000 7ff31aa00640 1 -- 172.21.15.133:0/419729185 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c043020 con 0x7ff324145670 2024-03-07T06:51:32.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.211+0000 7ff31aa00640 1 -- 172.21.15.133:0/419729185 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c03e450 con 0x7ff324145670 2024-03-07T06:51:32.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.211+0000 7ff31aa00640 1 -- 172.21.15.133:0/419729185 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c009510 con 0x7ff324145670 2024-03-07T06:51:32.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.211+0000 7ff328a00640 1 -- 172.21.15.133:0/419729185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3240b4840 con 0x7ff324145670 2024-03-07T06:51:32.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.212+0000 7ff328a00640 1 -- 172.21.15.133:0/419729185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3240af430 con 0x7ff324145670 2024-03-07T06:51:32.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.213+0000 7ff328a00640 1 -- 172.21.15.133:0/419729185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff324062c90 con 0x7ff324145670 2024-03-07T06:51:32.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.214+0000 7ff31aa00640 1 -- 172.21.15.133:0/419729185 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff31c046020 con 0x7ff324145670 2024-03-07T06:51:32.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.214+0000 7ff31aa00640 1 --2- 172.21.15.133:0/419729185 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff30006b4d0 0x7ff30006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:32.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.215+0000 7ff31aa00640 1 -- 172.21.15.133:0/419729185 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff31c0ac9f0 con 0x7ff324145670 2024-03-07T06:51:32.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.215+0000 7ff321e00640 1 --2- 172.21.15.133:0/419729185 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff30006b4d0 0x7ff30006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:32.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.216+0000 7ff321e00640 1 --2- 172.21.15.133:0/419729185 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff30006b4d0 0x7ff30006d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff314000f60 tx=0x7ff314009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:32.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.219+0000 7ff31aa00640 1 -- 172.21.15.133:0/419729185 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff31c04d030 con 0x7ff324145670 2024-03-07T06:51:32.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.416+0000 7ff328a00640 1 -- 172.21.15.133:0/419729185 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff3240cca20 con 0x7ff324145670 2024-03-07T06:51:32.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.417+0000 7ff31aa00640 1 -- 172.21.15.133:0/419729185 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff31c07b860 con 0x7ff324145670 2024-03-07T06:51:32.416 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:51:32.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.419+0000 7ff328a00640 1 -- 172.21.15.133:0/419729185 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff30006b4d0 msgr2=0x7ff30006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:32.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.419+0000 7ff328a00640 1 --2- 172.21.15.133:0/419729185 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff30006b4d0 0x7ff30006d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff314000f60 tx=0x7ff314009040 comp rx=0 tx=0).stop 2024-03-07T06:51:32.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.419+0000 7ff328a00640 1 -- 172.21.15.133:0/419729185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff324145670 msgr2=0x7ff3241f7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:32.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.419+0000 7ff328a00640 1 --2- 172.21.15.133:0/419729185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff324145670 0x7ff3241f7930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff31c009f70 tx=0x7ff31c004a40 comp rx=0 tx=0).stop 2024-03-07T06:51:32.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.419+0000 7ff328a00640 1 -- 172.21.15.133:0/419729185 shutdown_connections 2024-03-07T06:51:32.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.419+0000 7ff328a00640 1 --2- 172.21.15.133:0/419729185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff324145670 0x7ff3241f7930 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.419+0000 7ff328a00640 1 --2- 172.21.15.133:0/419729185 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff30006b4d0 0x7ff30006d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.419+0000 7ff328a00640 1 --2- 172.21.15.133:0/419729185 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3240cf610 0x7ff3241f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.419+0000 7ff328a00640 1 --2- 172.21.15.133:0/419729185 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3240cda20 0x7ff32415bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.419+0000 7ff328a00640 1 -- 172.21.15.133:0/419729185 >> 172.21.15.133:0/419729185 conn(0x7ff3240b1d30 msgr2=0x7ff3240719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:32.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.419+0000 7ff328a00640 1 -- 172.21.15.133:0/419729185 shutdown_connections 2024-03-07T06:51:32.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.420+0000 7ff328a00640 1 -- 172.21.15.133:0/419729185 wait complete. 2024-03-07T06:51:32.439 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:51:32.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.648+0000 7fe6e6600640 1 Processor -- start 2024-03-07T06:51:32.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.648+0000 7fe6e6600640 1 -- start start 2024-03-07T06:51:32.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.649+0000 7fe6e6600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6e00b5900 0x7fe6e006d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:32.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.649+0000 7fe6e6600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe6e00b2c00 0x7fe6e00b3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:32.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.649+0000 7fe6e6600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e00ada50 con 0x7fe6e0147840 2024-03-07T06:51:32.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.649+0000 7fe6e6600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e00b35a0 con 0x7fe6e00b2c00 2024-03-07T06:51:32.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.649+0000 7fe6e6600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e00ae380 con 0x7fe6e00b5900 2024-03-07T06:51:32.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.650+0000 7fe6dfe00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6e00b5900 0x7fe6e006d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:32.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.650+0000 7fe6e5c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe6e00b2c00 0x7fe6e00b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:32.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.650+0000 7fe6e5c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe6e00b2c00 0x7fe6e00b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:52976/0 (socket says 172.21.15.133:52976) 2024-03-07T06:51:32.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.650+0000 7fe6e5c00640 1 -- 172.21.15.133:0/2926473069 learned_addr learned my addr 172.21.15.133:0/2926473069 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:32.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.651+0000 7fe6dfe00640 1 -- 172.21.15.133:0/2926473069 >> v2:172.21.15.161:3300/0 conn(0x7fe6e00b2c00 msgr2=0x7fe6e00b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:32.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.651+0000 7fe6dfe00640 1 --2- 172.21.15.133:0/2926473069 >> v2:172.21.15.161:3300/0 conn(0x7fe6e00b2c00 0x7fe6e00b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.651+0000 7fe6dfe00640 1 -- 172.21.15.133:0/2926473069 >> v1:172.21.15.161:6789/0 conn(0x7fe6e0147840 legacy=0x7fe6e015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:32.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.651+0000 7fe6dfe00640 1 -- 172.21.15.133:0/2926473069 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e00add30 con 0x7fe6e00b5900 2024-03-07T06:51:32.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.652+0000 7fe6dfe00640 1 --2- 172.21.15.133:0/2926473069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6e00b5900 0x7fe6e006d0a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00097a0 tx=0x7fe6d002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc7f040efbcda7e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:32.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.652+0000 7fe6df400640 1 -- 172.21.15.133:0/2926473069 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0035020 con 0x7fe6e00b5900 2024-03-07T06:51:32.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.652+0000 7fe6df400640 1 -- 172.21.15.133:0/2926473069 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d0031810 con 0x7fe6e00b5900 2024-03-07T06:51:32.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.652+0000 7fe6df400640 1 -- 172.21.15.133:0/2926473069 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0031be0 con 0x7fe6e00b5900 2024-03-07T06:51:32.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.653+0000 7fe6e6600640 1 -- 172.21.15.133:0/2926473069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6e00b5900 msgr2=0x7fe6e006d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:32.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.653+0000 7fe6e6600640 1 --2- 172.21.15.133:0/2926473069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6e00b5900 0x7fe6e006d0a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00097a0 tx=0x7fe6d002b8b0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.653+0000 7fe6e6600640 1 -- 172.21.15.133:0/2926473069 shutdown_connections 2024-03-07T06:51:32.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.653+0000 7fe6e6600640 1 --2- 172.21.15.133:0/2926473069 >> v2:172.21.15.161:3300/0 conn(0x7fe6e00b2c00 0x7fe6e00b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.653+0000 7fe6e6600640 1 --2- 172.21.15.133:0/2926473069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6e00b5900 0x7fe6e006d0a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.653+0000 7fe6e6600640 1 -- 172.21.15.133:0/2926473069 >> 172.21.15.133:0/2926473069 conn(0x7fe6e0071860 msgr2=0x7fe6e0071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:32.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.653+0000 7fe6e6600640 1 -- 172.21.15.133:0/2926473069 shutdown_connections 2024-03-07T06:51:32.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.653+0000 7fe6e6600640 1 -- 172.21.15.133:0/2926473069 wait complete. 2024-03-07T06:51:32.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.654+0000 7fe6e6600640 1 Processor -- start 2024-03-07T06:51:32.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.655+0000 7fe6e6600640 1 -- start start 2024-03-07T06:51:32.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.655+0000 7fe6e6600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e00b2c00 0x7fe6e015da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:32.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.655+0000 7fe6e6600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6e00b5900 0x7fe6e00cf300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:32.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.655+0000 7fe6e6600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e0147840 0x7fe6e00cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.655+0000 7fe6e6600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e006d760 con 0x7fe6e00b2c00 2024-03-07T06:51:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.655+0000 7fe6e6600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e006d5e0 con 0x7fe6e0147840 2024-03-07T06:51:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.655+0000 7fe6e6600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e00ae810 con 0x7fe6e00b5900 2024-03-07T06:51:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.656+0000 7fe6e5200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e00b2c00 0x7fe6e015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.656+0000 7fe6e5200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e00b2c00 0x7fe6e015da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55880/0 (socket says 172.21.15.133:55880) 2024-03-07T06:51:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.656+0000 7fe6e5200640 1 -- 172.21.15.133:0/1997195746 learned_addr learned my addr 172.21.15.133:0/1997195746 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.656+0000 7fe6dfe00640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6e00b5900 0x7fe6e00cf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.656+0000 7fe6e5c00640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e0147840 0x7fe6e00cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.656+0000 7fe6e5200640 1 -- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6e00b5900 msgr2=0x7fe6e00cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.656+0000 7fe6e5200640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6e00b5900 0x7fe6e00cf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.656+0000 7fe6e5200640 1 -- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e0147840 msgr2=0x7fe6e00cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.656+0000 7fe6e5200640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e0147840 0x7fe6e00cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.656+0000 7fe6e5200640 1 -- 172.21.15.133:0/1997195746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e015df50 con 0x7fe6e00b2c00 2024-03-07T06:51:32.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.656+0000 7fe6e5c00640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e0147840 0x7fe6e00cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:32.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.656+0000 7fe6dfe00640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6e00b5900 0x7fe6e00cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:32.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.656+0000 7fe6e5200640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e00b2c00 0x7fe6e015da10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400c3c0 tx=0x7fe6d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:32.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.657+0000 7fe6dcc00640 1 -- 172.21.15.133:0/1997195746 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d40052d0 con 0x7fe6e00b2c00 2024-03-07T06:51:32.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.657+0000 7fe6dcc00640 1 -- 172.21.15.133:0/1997195746 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d4005490 con 0x7fe6e00b2c00 2024-03-07T06:51:32.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.657+0000 7fe6dcc00640 1 -- 172.21.15.133:0/1997195746 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4007260 con 0x7fe6e00b2c00 2024-03-07T06:51:32.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.657+0000 7fe6e6600640 1 -- 172.21.15.133:0/1997195746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e00ce260 con 0x7fe6e00b2c00 2024-03-07T06:51:32.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.657+0000 7fe6e6600640 1 -- 172.21.15.133:0/1997195746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e00ce820 con 0x7fe6e00b2c00 2024-03-07T06:51:32.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.658+0000 7fe6e6600640 1 -- 172.21.15.133:0/1997195746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e0062c90 con 0x7fe6e00b2c00 2024-03-07T06:51:32.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.659+0000 7fe6dcc00640 1 -- 172.21.15.133:0/1997195746 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe6d4007400 con 0x7fe6e00b2c00 2024-03-07T06:51:32.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.660+0000 7fe6dcc00640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe6c406b500 0x7fe6c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:32.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.660+0000 7fe6dcc00640 1 -- 172.21.15.133:0/1997195746 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6d4089660 con 0x7fe6e00b2c00 2024-03-07T06:51:32.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.660+0000 7fe6dfe00640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe6c406b500 0x7fe6c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:32.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.661+0000 7fe6dfe00640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe6c406b500 0x7fe6c406d9c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0036000 tx=0x7fe6d0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:32.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.664+0000 7fe6dcc00640 1 -- 172.21.15.133:0/1997195746 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe6d40584d0 con 0x7fe6e00b2c00 2024-03-07T06:51:32.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.856+0000 7fe6e6600640 1 -- 172.21.15.133:0/1997195746 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe6e00b5300 con 0x7fe6e00b2c00 2024-03-07T06:51:32.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.856+0000 7fe6dcc00640 1 -- 172.21.15.133:0/1997195746 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe6d4057dd0 con 0x7fe6e00b2c00 2024-03-07T06:51:32.856 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:51:32.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.859+0000 7fe6e6600640 1 -- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe6c406b500 msgr2=0x7fe6c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:32.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.859+0000 7fe6e6600640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe6c406b500 0x7fe6c406d9c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0036000 tx=0x7fe6d0008030 comp rx=0 tx=0).stop 2024-03-07T06:51:32.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.859+0000 7fe6e6600640 1 -- 172.21.15.133:0/1997195746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e00b2c00 msgr2=0x7fe6e015da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:32.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.859+0000 7fe6e6600640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e00b2c00 0x7fe6e015da10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe6d400c3c0 tx=0x7fe6d400c880 comp rx=0 tx=0).stop 2024-03-07T06:51:32.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.859+0000 7fe6e6600640 1 -- 172.21.15.133:0/1997195746 shutdown_connections 2024-03-07T06:51:32.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.859+0000 7fe6e6600640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe6c406b500 0x7fe6c406d9c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.859+0000 7fe6e6600640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe6e0147840 0x7fe6e00cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.859+0000 7fe6e6600640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6e00b5900 0x7fe6e00cf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.859+0000 7fe6e6600640 1 --2- 172.21.15.133:0/1997195746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e00b2c00 0x7fe6e015da10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:32.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.859+0000 7fe6e6600640 1 -- 172.21.15.133:0/1997195746 >> 172.21.15.133:0/1997195746 conn(0x7fe6e0071860 msgr2=0x7fe6e00aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:32.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.859+0000 7fe6e6600640 1 -- 172.21.15.133:0/1997195746 shutdown_connections 2024-03-07T06:51:32.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:32.860+0000 7fe6e6600640 1 -- 172.21.15.133:0/1997195746 wait complete. 2024-03-07T06:51:32.879 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:51:33.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.075+0000 7ff88c600640 1 Processor -- start 2024-03-07T06:51:33.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.075+0000 7ff88c600640 1 -- start start 2024-03-07T06:51:33.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.077+0000 7ff88c600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff8840af950 0x7ff8840b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.077+0000 7ff88c600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88406f140 con 0x7ff8840ce5f0 2024-03-07T06:51:33.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.077+0000 7ff88c600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff88406f420 con 0x7ff88415b1b0 2024-03-07T06:51:33.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.077+0000 7ff88c600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff884071610 con 0x7ff8840af950 2024-03-07T06:51:33.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.077+0000 7ff889200640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff8840ce5f0 0x7ff884071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46914/0 (socket says 172.21.15.133:46914) 2024-03-07T06:51:33.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.077+0000 7ff889200640 1 -- 172.21.15.133:0/76134209 learned_addr learned my addr 172.21.15.133:0/76134209 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:33.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.077+0000 7ff88a600640 1 --2- 172.21.15.133:0/76134209 >> v2:172.21.15.161:3300/0 conn(0x7ff8840af950 0x7ff8840b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.077+0000 7ff883e00640 1 -- 172.21.15.133:0/76134209 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460443457 0 0) 0x7ff88406f140 con 0x7ff8840ce5f0 2024-03-07T06:51:33.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.077+0000 7ff883e00640 1 -- 172.21.15.133:0/76134209 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff86c003520 con 0x7ff8840ce5f0 2024-03-07T06:51:33.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff883e00640 1 -- 172.21.15.133:0/76134209 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 768152595 0 0) 0x7ff86c003520 con 0x7ff8840ce5f0 2024-03-07T06:51:33.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff883e00640 1 -- 172.21.15.133:0/76134209 >> v2:172.21.15.161:3300/0 conn(0x7ff8840af950 msgr2=0x7ff8840b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff883e00640 1 --2- 172.21.15.133:0/76134209 >> v2:172.21.15.161:3300/0 conn(0x7ff8840af950 0x7ff8840b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff883e00640 1 -- 172.21.15.133:0/76134209 >> v1:172.21.15.161:6789/0 conn(0x7ff88415b1b0 legacy=0x7ff8840cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff883e00640 1 -- 172.21.15.133:0/76134209 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff884071860 con 0x7ff8840ce5f0 2024-03-07T06:51:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff883e00640 1 -- 172.21.15.133:0/76134209 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7ff870001670 con 0x7ff8840ce5f0 2024-03-07T06:51:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff883e00640 1 -- 172.21.15.133:0/76134209 >> v1:172.21.15.133:6789/0 conn(0x7ff8840ce5f0 legacy=0x7ff884071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff883e00640 1 --2- 172.21.15.133:0/76134209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff86c003a00 0x7ff86c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff883e00640 1 --2- 172.21.15.133:0/76134209 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff86c004800 0x7ff86c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff883e00640 1 --2- 172.21.15.133:0/76134209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff86c005210 0x7ff86c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff883e00640 1 -- 172.21.15.133:0/76134209 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff884071610 con 0x7ff86c003a00 2024-03-07T06:51:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff883e00640 1 -- 172.21.15.133:0/76134209 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c00e350 con 0x7ff86c005210 2024-03-07T06:51:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff883e00640 1 -- 172.21.15.133:0/76134209 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86c00e550 con 0x7ff86c004800 2024-03-07T06:51:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff889c00640 1 --2- 172.21.15.133:0/76134209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff86c003a00 0x7ff86c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff889c00640 1 -- 172.21.15.133:0/76134209 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff86c004800 msgr2=0x7ff86c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:33.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff889c00640 1 --2- 172.21.15.133:0/76134209 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff86c004800 0x7ff86c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff889c00640 1 -- 172.21.15.133:0/76134209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff86c005210 msgr2=0x7ff86c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:33.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff889c00640 1 --2- 172.21.15.133:0/76134209 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff86c005210 0x7ff86c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff889c00640 1 -- 172.21.15.133:0/76134209 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86c003520 con 0x7ff86c003a00 2024-03-07T06:51:33.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.078+0000 7ff88a600640 1 -- 172.21.15.133:0/76134209 reap_dead start 2024-03-07T06:51:33.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.079+0000 7ff889c00640 1 --2- 172.21.15.133:0/76134209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff86c003a00 0x7ff86c003e00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff878003db0 tx=0x7ff86c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63046764602fbcb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:33.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.080+0000 7ff883e00640 1 -- 172.21.15.133:0/76134209 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878037070 con 0x7ff86c003a00 2024-03-07T06:51:33.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.080+0000 7ff883e00640 1 -- 172.21.15.133:0/76134209 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff878032600 con 0x7ff86c003a00 2024-03-07T06:51:33.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.080+0000 7ff883e00640 1 -- 172.21.15.133:0/76134209 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff878003980 con 0x7ff86c003a00 2024-03-07T06:51:33.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.080+0000 7ff88c600640 1 -- 172.21.15.133:0/76134209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff86c003a00 msgr2=0x7ff86c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.080+0000 7ff88c600640 1 --2- 172.21.15.133:0/76134209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff86c003a00 0x7ff86c003e00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff878003db0 tx=0x7ff86c005830 comp rx=0 tx=0).stop 2024-03-07T06:51:33.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.080+0000 7ff88c600640 1 -- 172.21.15.133:0/76134209 shutdown_connections 2024-03-07T06:51:33.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.080+0000 7ff88c600640 1 --2- 172.21.15.133:0/76134209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff86c003a00 0x7ff86c003e00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.080+0000 7ff88c600640 1 -- 172.21.15.133:0/76134209 >> 172.21.15.133:0/76134209 conn(0x7ff8840b3120 msgr2=0x7ff8840b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:33.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.080+0000 7ff88c600640 1 -- 172.21.15.133:0/76134209 shutdown_connections 2024-03-07T06:51:33.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.080+0000 7ff88c600640 1 -- 172.21.15.133:0/76134209 wait complete. 2024-03-07T06:51:33.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.082+0000 7ff88c600640 1 Processor -- start 2024-03-07T06:51:33.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.082+0000 7ff88c600640 1 -- start start 2024-03-07T06:51:33.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff88c600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff86c003a00 0x7ff8841439f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff88c600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8840b0480 0x7ff8840b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff88c600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff884156c70 0x7ff884157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff88c600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88415b710 con 0x7ff8840b0480 2024-03-07T06:51:33.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff88c600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88415b590 con 0x7ff884156c70 2024-03-07T06:51:33.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff88c600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8840b3c10 con 0x7ff86c003a00 2024-03-07T06:51:33.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff889200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8840b0480 0x7ff8840b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff889200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8840b0480 0x7ff8840b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55900/0 (socket says 172.21.15.133:55900) 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff889200640 1 -- 172.21.15.133:0/4220784155 learned_addr learned my addr 172.21.15.133:0/4220784155 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff889c00640 1 --2- 172.21.15.133:0/4220784155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff86c003a00 0x7ff8841439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff889200640 1 -- 172.21.15.133:0/4220784155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff86c003a00 msgr2=0x7ff8841439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff889200640 1 --2- 172.21.15.133:0/4220784155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff86c003a00 0x7ff8841439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff889200640 1 -- 172.21.15.133:0/4220784155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff884156c70 msgr2=0x7ff884157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff889200640 1 --2- 172.21.15.133:0/4220784155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff884156c70 0x7ff884157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.083+0000 7ff889200640 1 -- 172.21.15.133:0/4220784155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88415c010 con 0x7ff8840b0480 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.084+0000 7ff889200640 1 --2- 172.21.15.133:0/4220784155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8840b0480 0x7ff8840b0900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff87000c730 tx=0x7ff87000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.084+0000 7ff881600640 1 -- 172.21.15.133:0/4220784155 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870018070 con 0x7ff8840b0480 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.084+0000 7ff881600640 1 -- 172.21.15.133:0/4220784155 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8700072d0 con 0x7ff8840b0480 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.084+0000 7ff881600640 1 -- 172.21.15.133:0/4220784155 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8700047e0 con 0x7ff8840b0480 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.084+0000 7ff88c600640 1 -- 172.21.15.133:0/4220784155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8841579c0 con 0x7ff8840b0480 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.084+0000 7ff88c600640 1 -- 172.21.15.133:0/4220784155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff884157f80 con 0x7ff8840b0480 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.085+0000 7ff88c600640 1 -- 172.21.15.133:0/4220784155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff884062c90 con 0x7ff8840b0480 2024-03-07T06:51:33.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.086+0000 7ff881600640 1 -- 172.21.15.133:0/4220784155 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff870004980 con 0x7ff8840b0480 2024-03-07T06:51:33.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.087+0000 7ff881600640 1 --2- 172.21.15.133:0/4220784155 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff86406b500 0x7ff86406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.087+0000 7ff881600640 1 -- 172.21.15.133:0/4220784155 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff870089650 con 0x7ff8840b0480 2024-03-07T06:51:33.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.088+0000 7ff889c00640 1 --2- 172.21.15.133:0/4220784155 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff86406b500 0x7ff86406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.088+0000 7ff889c00640 1 --2- 172.21.15.133:0/4220784155 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff86406b500 0x7ff86406d9c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff878034000 tx=0x7ff878009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:33.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.091+0000 7ff881600640 1 -- 172.21.15.133:0/4220784155 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff870004c70 con 0x7ff8840b0480 2024-03-07T06:51:33.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.321+0000 7ff88c600640 1 -- 172.21.15.133:0/4220784155 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff88415c1e0 con 0x7ff8840b0480 2024-03-07T06:51:33.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.322+0000 7ff881600640 1 -- 172.21.15.133:0/4220784155 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4983 (secure 0 0 0) 0x7ff8700584d0 con 0x7ff8840b0480 2024-03-07T06:51:33.322 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:33.322 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1116649196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1116649196},{"type":"v1","addr":"172.21.15.161:6835","nonce":1116649196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3485625105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3485625105},{"type":"v1","addr":"172.21.15.133:6835","nonce":3485625105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3866978630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3866978630},{"type":"v1","addr":"172.21.15.133:6833","nonce":3866978630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:51:14.594513+0000","modified":"2024-03-07T06:51:17.675044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2885273912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2885273912},{"type":"v1","addr":"172.21.15.161:6837","nonce":2885273912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-03-07T06:51:33.322 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 8 2024-03-07T06:51:33.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.324+0000 7ff88c600640 1 -- 172.21.15.133:0/4220784155 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff86406b500 msgr2=0x7ff86406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.324+0000 7ff88c600640 1 --2- 172.21.15.133:0/4220784155 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff86406b500 0x7ff86406d9c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff878034000 tx=0x7ff878009030 comp rx=0 tx=0).stop 2024-03-07T06:51:33.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.324+0000 7ff88c600640 1 -- 172.21.15.133:0/4220784155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8840b0480 msgr2=0x7ff8840b0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.324+0000 7ff88c600640 1 --2- 172.21.15.133:0/4220784155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8840b0480 0x7ff8840b0900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff87000c730 tx=0x7ff87000cbf0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.325+0000 7ff88c600640 1 -- 172.21.15.133:0/4220784155 shutdown_connections 2024-03-07T06:51:33.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.325+0000 7ff88c600640 1 --2- 172.21.15.133:0/4220784155 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff86406b500 0x7ff86406d9c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.325+0000 7ff88c600640 1 --2- 172.21.15.133:0/4220784155 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff884156c70 0x7ff884157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.325+0000 7ff88c600640 1 --2- 172.21.15.133:0/4220784155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8840b0480 0x7ff8840b0900 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.325+0000 7ff88c600640 1 --2- 172.21.15.133:0/4220784155 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff86c003a00 0x7ff8841439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.325+0000 7ff88c600640 1 -- 172.21.15.133:0/4220784155 >> 172.21.15.133:0/4220784155 conn(0x7ff8840b3120 msgr2=0x7ff8840afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:33.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.325+0000 7ff88c600640 1 -- 172.21.15.133:0/4220784155 shutdown_connections 2024-03-07T06:51:33.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.325+0000 7ff88c600640 1 -- 172.21.15.133:0/4220784155 wait complete. 2024-03-07T06:51:33.343 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:33.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.538+0000 7f24c9400640 1 Processor -- start 2024-03-07T06:51:33.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.539+0000 7f24c9400640 1 -- start start 2024-03-07T06:51:33.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.540+0000 7f24c9400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f24c415e390 0x7f24c40724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.540+0000 7f24c9400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c4071d20 0x7f24c40ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.540+0000 7f24c9400640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c40cdf80 con 0x7f24c4073050 2024-03-07T06:51:33.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.540+0000 7f24c9400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c40ce4d0 con 0x7f24c415e390 2024-03-07T06:51:33.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.540+0000 7f24c9400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c415e770 con 0x7f24c4071d20 2024-03-07T06:51:33.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.540+0000 7f24b9e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f24c4073050 0x7f24c4071610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46926/0 (socket says 172.21.15.133:46926) 2024-03-07T06:51:33.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.540+0000 7f24b9e00640 1 -- 172.21.15.133:0/3904572604 learned_addr learned my addr 172.21.15.133:0/3904572604 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:33.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c3200640 1 --2- 172.21.15.133:0/3904572604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c4071d20 0x7f24c40ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c2800640 1 --2- 172.21.15.133:0/3904572604 >> v2:172.21.15.161:3300/0 conn(0x7f24c415e390 0x7f24c40724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 -- 172.21.15.133:0/3904572604 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530281834 0 0) 0x7f24c40cdf80 con 0x7f24c4073050 2024-03-07T06:51:33.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 -- 172.21.15.133:0/3904572604 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24a8003520 con 0x7f24c4073050 2024-03-07T06:51:33.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 -- 172.21.15.133:0/3904572604 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2333038633 0 0) 0x7f24a8003520 con 0x7f24c4073050 2024-03-07T06:51:33.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 -- 172.21.15.133:0/3904572604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c4071d20 msgr2=0x7f24c40ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:33.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 --2- 172.21.15.133:0/3904572604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c4071d20 0x7f24c40ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 -- 172.21.15.133:0/3904572604 >> v2:172.21.15.161:3300/0 conn(0x7f24c415e390 msgr2=0x7f24c40724b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:33.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 --2- 172.21.15.133:0/3904572604 >> v2:172.21.15.161:3300/0 conn(0x7f24c415e390 0x7f24c40724b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 -- 172.21.15.133:0/3904572604 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c40b3d90 con 0x7f24c4073050 2024-03-07T06:51:33.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 -- 172.21.15.133:0/3904572604 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f24ac001670 con 0x7f24c4073050 2024-03-07T06:51:33.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 -- 172.21.15.133:0/3904572604 >> v1:172.21.15.133:6789/0 conn(0x7f24c4073050 legacy=0x7f24c4071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 --2- 172.21.15.133:0/3904572604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a80039b0 0x7f24a8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 --2- 172.21.15.133:0/3904572604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a8004800 0x7f24a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 --2- 172.21.15.133:0/3904572604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c4071d20 0x7f24a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 -- 172.21.15.133:0/3904572604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c40ce4d0 con 0x7f24a80039b0 2024-03-07T06:51:33.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 -- 172.21.15.133:0/3904572604 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c415e770 con 0x7f24a8004800 2024-03-07T06:51:33.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.541+0000 7f24c1e00640 1 -- 172.21.15.133:0/3904572604 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a800b9c0 con 0x7f24c4071d20 2024-03-07T06:51:33.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.542+0000 7f24c2800640 1 --2- 172.21.15.133:0/3904572604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a80039b0 0x7f24a8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.542+0000 7f24c3200640 1 --2- 172.21.15.133:0/3904572604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c4071d20 0x7f24a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.542+0000 7f24b9e00640 1 --2- 172.21.15.133:0/3904572604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a8004800 0x7f24a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.542+0000 7f24c2800640 1 -- 172.21.15.133:0/3904572604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c4071d20 msgr2=0x7f24a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:33.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.542+0000 7f24c2800640 1 --2- 172.21.15.133:0/3904572604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c4071d20 0x7f24a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.542+0000 7f24c2800640 1 -- 172.21.15.133:0/3904572604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a8004800 msgr2=0x7f24a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:33.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.542+0000 7f24c2800640 1 --2- 172.21.15.133:0/3904572604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a8004800 0x7f24a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.542+0000 7f24c2800640 1 -- 172.21.15.133:0/3904572604 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a8003520 con 0x7f24a80039b0 2024-03-07T06:51:33.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.542+0000 7f24c3200640 1 --2- 172.21.15.133:0/3904572604 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c4071d20 0x7f24a80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:33.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.542+0000 7f24b9e00640 1 --2- 172.21.15.133:0/3904572604 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24a8004800 0x7f24a8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:33.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.543+0000 7f24c2800640 1 --2- 172.21.15.133:0/3904572604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a80039b0 0x7f24a8003df0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009fc0 tx=0x7f24a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23eed29a98ce9851 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:33.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.543+0000 7f24c1e00640 1 -- 172.21.15.133:0/3904572604 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4039070 con 0x7f24a80039b0 2024-03-07T06:51:33.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.543+0000 7f24c1e00640 1 -- 172.21.15.133:0/3904572604 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b402cc40 con 0x7f24a80039b0 2024-03-07T06:51:33.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.543+0000 7f24c1e00640 1 -- 172.21.15.133:0/3904572604 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4009c10 con 0x7f24a80039b0 2024-03-07T06:51:33.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.544+0000 7f24c9400640 1 -- 172.21.15.133:0/3904572604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a80039b0 msgr2=0x7f24a8003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.544+0000 7f24c9400640 1 --2- 172.21.15.133:0/3904572604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a80039b0 0x7f24a8003df0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009fc0 tx=0x7f24a8005830 comp rx=0 tx=0).stop 2024-03-07T06:51:33.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.544+0000 7f24c3200640 1 -- 172.21.15.133:0/3904572604 reap_dead start 2024-03-07T06:51:33.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.544+0000 7f24c9400640 1 -- 172.21.15.133:0/3904572604 shutdown_connections 2024-03-07T06:51:33.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.544+0000 7f24c9400640 1 -- 172.21.15.133:0/3904572604 >> 172.21.15.133:0/3904572604 conn(0x7f24c40b3120 msgr2=0x7f24c40b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:33.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.544+0000 7f24c9400640 1 -- 172.21.15.133:0/3904572604 shutdown_connections 2024-03-07T06:51:33.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.544+0000 7f24c9400640 1 -- 172.21.15.133:0/3904572604 wait complete. 2024-03-07T06:51:33.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.546+0000 7f24c9400640 1 Processor -- start 2024-03-07T06:51:33.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.546+0000 7f24c9400640 1 -- start start 2024-03-07T06:51:33.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24c9400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a80039b0 0x7f24c41546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24c9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c415e8f0 0x7f24c4147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24c9400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c4157e50 0x7f24c4158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24c9400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c40ae720 con 0x7f24a80039b0 2024-03-07T06:51:33.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24c9400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c40b4070 con 0x7f24c415e8f0 2024-03-07T06:51:33.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24c9400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c40ae8a0 con 0x7f24c4157e50 2024-03-07T06:51:33.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24c2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a80039b0 0x7f24c41546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24c2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a80039b0 0x7f24c41546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55912/0 (socket says 172.21.15.133:55912) 2024-03-07T06:51:33.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24c2800640 1 -- 172.21.15.133:0/4147021724 learned_addr learned my addr 172.21.15.133:0/4147021724 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:33.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24b9e00640 1 --2- 172.21.15.133:0/4147021724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c415e8f0 0x7f24c4147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24c3200640 1 --2- 172.21.15.133:0/4147021724 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c4157e50 0x7f24c4158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24c2800640 1 -- 172.21.15.133:0/4147021724 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c4157e50 msgr2=0x7f24c4158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24c2800640 1 --2- 172.21.15.133:0/4147021724 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c4157e50 0x7f24c4158300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24c2800640 1 -- 172.21.15.133:0/4147021724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c415e8f0 msgr2=0x7f24c4147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.547+0000 7f24c2800640 1 --2- 172.21.15.133:0/4147021724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c415e8f0 0x7f24c4147830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.548+0000 7f24c2800640 1 -- 172.21.15.133:0/4147021724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c415f900 con 0x7f24a80039b0 2024-03-07T06:51:33.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.548+0000 7f24c2800640 1 --2- 172.21.15.133:0/4147021724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a80039b0 0x7f24c41546e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f24b4037040 tx=0x7f24b4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:33.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.548+0000 7f24bb400640 1 -- 172.21.15.133:0/4147021724 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4039050 con 0x7f24a80039b0 2024-03-07T06:51:33.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.548+0000 7f24bb400640 1 -- 172.21.15.133:0/4147021724 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b403a040 con 0x7f24a80039b0 2024-03-07T06:51:33.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.548+0000 7f24c9400640 1 -- 172.21.15.133:0/4147021724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c415f060 con 0x7f24a80039b0 2024-03-07T06:51:33.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.548+0000 7f24bb400640 1 -- 172.21.15.133:0/4147021724 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4003990 con 0x7f24a80039b0 2024-03-07T06:51:33.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.548+0000 7f24c9400640 1 -- 172.21.15.133:0/4147021724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c415f620 con 0x7f24a80039b0 2024-03-07T06:51:33.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.549+0000 7f24c9400640 1 -- 172.21.15.133:0/4147021724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c4062c90 con 0x7f24a80039b0 2024-03-07T06:51:33.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.550+0000 7f24bb400640 1 -- 172.21.15.133:0/4147021724 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f24b4047020 con 0x7f24a80039b0 2024-03-07T06:51:33.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.551+0000 7f24bb400640 1 --2- 172.21.15.133:0/4147021724 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f24a006b500 0x7f24a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.552+0000 7f24bb400640 1 -- 172.21.15.133:0/4147021724 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24b40ac350 con 0x7f24a80039b0 2024-03-07T06:51:33.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.552+0000 7f24b9e00640 1 --2- 172.21.15.133:0/4147021724 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f24a006b500 0x7f24a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.553+0000 7f24b9e00640 1 --2- 172.21.15.133:0/4147021724 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f24a006b500 0x7f24a006d9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f24ac000f60 tx=0x7f24ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:33.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.556+0000 7f24bb400640 1 -- 172.21.15.133:0/4147021724 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f24b407b2e0 con 0x7f24a80039b0 2024-03-07T06:51:33.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.751+0000 7f24c9400640 1 -- 172.21.15.133:0/4147021724 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24c40ceaa0 con 0x7f24a80039b0 2024-03-07T06:51:33.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.752+0000 7f24bb400640 1 -- 172.21.15.133:0/4147021724 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f24b407abe0 con 0x7f24a80039b0 2024-03-07T06:51:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:33.753 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":21,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:16.174650+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:33.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.755+0000 7f24c9400640 1 -- 172.21.15.133:0/4147021724 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f24a006b500 msgr2=0x7f24a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.755+0000 7f24c9400640 1 --2- 172.21.15.133:0/4147021724 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f24a006b500 0x7f24a006d9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f24ac000f60 tx=0x7f24ac009040 comp rx=0 tx=0).stop 2024-03-07T06:51:33.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.755+0000 7f24c9400640 1 -- 172.21.15.133:0/4147021724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a80039b0 msgr2=0x7f24c41546e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.755+0000 7f24c9400640 1 --2- 172.21.15.133:0/4147021724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a80039b0 0x7f24c41546e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f24b4037040 tx=0x7f24b4009750 comp rx=0 tx=0).stop 2024-03-07T06:51:33.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.755+0000 7f24c9400640 1 -- 172.21.15.133:0/4147021724 shutdown_connections 2024-03-07T06:51:33.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.755+0000 7f24c9400640 1 --2- 172.21.15.133:0/4147021724 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c4157e50 0x7f24c4158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.755+0000 7f24c9400640 1 --2- 172.21.15.133:0/4147021724 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f24a006b500 0x7f24a006d9c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.755+0000 7f24c9400640 1 --2- 172.21.15.133:0/4147021724 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c415e8f0 0x7f24c4147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.755+0000 7f24c9400640 1 --2- 172.21.15.133:0/4147021724 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24a80039b0 0x7f24c41546e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.755+0000 7f24c9400640 1 -- 172.21.15.133:0/4147021724 >> 172.21.15.133:0/4147021724 conn(0x7f24c40b3120 msgr2=0x7f24c40b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:33.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.755+0000 7f24c9400640 1 -- 172.21.15.133:0/4147021724 shutdown_connections 2024-03-07T06:51:33.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.755+0000 7f24c9400640 1 -- 172.21.15.133:0/4147021724 wait complete. 2024-03-07T06:51:33.773 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:51:12.858333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:51:13.615348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:51:33.774 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:51:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.981+0000 7f6f85400640 1 Processor -- start 2024-03-07T06:51:33.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.982+0000 7f6f85400640 1 -- start start 2024-03-07T06:51:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.983+0000 7f6f85400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6f80145670 0x7f6f800cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.983+0000 7f6f85400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f8015cf90 0x7f6f800cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.983+0000 7f6f85400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6f800cda20 0x7f6f8015fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.983+0000 7f6f85400640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8006ed60 con 0x7f6f800cda20 2024-03-07T06:51:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.983+0000 7f6f85400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f8006e150 con 0x7f6f80145670 2024-03-07T06:51:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.983+0000 7f6f85400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8006e430 con 0x7f6f8015cf90 2024-03-07T06:51:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.983+0000 7f6f7f200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6f800cda20 0x7f6f8015fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.983+0000 7f6f7f200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6f800cda20 0x7f6f8015fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55922/0 (socket says 172.21.15.133:55922) 2024-03-07T06:51:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.983+0000 7f6f7f200640 1 -- 172.21.15.133:0/363754067 learned_addr learned my addr 172.21.15.133:0/363754067 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.984+0000 7f6f7e800640 1 --2- 172.21.15.133:0/363754067 >> v2:172.21.15.161:3300/0 conn(0x7f6f80145670 0x7f6f800cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.984+0000 7f6f7f200640 1 -- 172.21.15.133:0/363754067 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f8015cf90 msgr2=0x7f6f800cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:33.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.984+0000 7f6f7f200640 1 --2- 172.21.15.133:0/363754067 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f8015cf90 0x7f6f800cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.984+0000 7f6f7f200640 1 -- 172.21.15.133:0/363754067 >> v2:172.21.15.161:3300/0 conn(0x7f6f80145670 msgr2=0x7f6f800cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.984+0000 7f6f7f200640 1 --2- 172.21.15.133:0/363754067 >> v2:172.21.15.161:3300/0 conn(0x7f6f80145670 0x7f6f800cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.984+0000 7f6f7f200640 1 -- 172.21.15.133:0/363754067 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8006e710 con 0x7f6f800cda20 2024-03-07T06:51:33.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.984+0000 7f6f7e800640 1 --2- 172.21.15.133:0/363754067 >> v2:172.21.15.161:3300/0 conn(0x7f6f80145670 0x7f6f800cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.985+0000 7f6f7f200640 1 --2- 172.21.15.133:0/363754067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f800cda20 0x7f6f8015fb00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6f80071760 tx=0x7f6f7802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15cb792f6c0b0bc server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.985+0000 7f6f7de00640 1 -- 172.21.15.133:0/363754067 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7802c9e0 con 0x7f6f800cda20 2024-03-07T06:51:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.985+0000 7f6f7de00640 1 -- 172.21.15.133:0/363754067 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7802cb80 con 0x7f6f800cda20 2024-03-07T06:51:33.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.985+0000 7f6f7de00640 1 -- 172.21.15.133:0/363754067 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f780343f0 con 0x7f6f800cda20 2024-03-07T06:51:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.986+0000 7f6f85400640 1 -- 172.21.15.133:0/363754067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f800cda20 msgr2=0x7f6f8015fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.986+0000 7f6f85400640 1 --2- 172.21.15.133:0/363754067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f800cda20 0x7f6f8015fb00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6f80071760 tx=0x7f6f7802be40 comp rx=0 tx=0).stop 2024-03-07T06:51:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.986+0000 7f6f85400640 1 -- 172.21.15.133:0/363754067 shutdown_connections 2024-03-07T06:51:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.986+0000 7f6f85400640 1 --2- 172.21.15.133:0/363754067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f800cda20 0x7f6f8015fb00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.986+0000 7f6f85400640 1 --2- 172.21.15.133:0/363754067 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f8015cf90 0x7f6f800cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.986+0000 7f6f85400640 1 --2- 172.21.15.133:0/363754067 >> v2:172.21.15.161:3300/0 conn(0x7f6f80145670 0x7f6f800cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.986+0000 7f6f85400640 1 -- 172.21.15.133:0/363754067 >> 172.21.15.133:0/363754067 conn(0x7f6f800b1d30 msgr2=0x7f6f800b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.986+0000 7f6f85400640 1 -- 172.21.15.133:0/363754067 shutdown_connections 2024-03-07T06:51:33.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.986+0000 7f6f85400640 1 -- 172.21.15.133:0/363754067 wait complete. 2024-03-07T06:51:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.988+0000 7f6f85400640 1 Processor -- start 2024-03-07T06:51:33.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.988+0000 7f6f85400640 1 -- start start 2024-03-07T06:51:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.989+0000 7f6f85400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f800cda20 0x7f6f800a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.989+0000 7f6f85400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f80145670 0x7f6f801373e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.989+0000 7f6f85400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f8015cf90 0x7f6f80137920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.989+0000 7f6f85400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f80145a50 con 0x7f6f8015cf90 2024-03-07T06:51:33.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.989+0000 7f6f85400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8015d3f0 con 0x7f6f80145670 2024-03-07T06:51:33.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.989+0000 7f6f85400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f80145bd0 con 0x7f6f800cda20 2024-03-07T06:51:33.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.990+0000 7f6f7f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f8015cf90 0x7f6f80137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.990+0000 7f6f7f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f8015cf90 0x7f6f80137920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55932/0 (socket says 172.21.15.133:55932) 2024-03-07T06:51:33.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.990+0000 7f6f7f200640 1 -- 172.21.15.133:0/466566793 learned_addr learned my addr 172.21.15.133:0/466566793 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:33.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.990+0000 7f6f7e800640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f800cda20 0x7f6f800a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.990+0000 7f6f75e00640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f80145670 0x7f6f801373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.990+0000 7f6f7f200640 1 -- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f800cda20 msgr2=0x7f6f800a9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.990+0000 7f6f7f200640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f800cda20 0x7f6f800a9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.990+0000 7f6f7f200640 1 -- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f80145670 msgr2=0x7f6f801373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:33.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.990+0000 7f6f7f200640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f80145670 0x7f6f801373e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:33.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.990+0000 7f6f7f200640 1 -- 172.21.15.133:0/466566793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f800b3000 con 0x7f6f8015cf90 2024-03-07T06:51:33.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.990+0000 7f6f7e800640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f800cda20 0x7f6f800a9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:33.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.990+0000 7f6f75e00640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f80145670 0x7f6f801373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:33.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.990+0000 7f6f7f200640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f8015cf90 0x7f6f80137920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009f40 tx=0x7f6f78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:33.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.991+0000 7f6f77400640 1 -- 172.21.15.133:0/466566793 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78043020 con 0x7f6f8015cf90 2024-03-07T06:51:33.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.991+0000 7f6f77400640 1 -- 172.21.15.133:0/466566793 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7803e450 con 0x7f6f8015cf90 2024-03-07T06:51:33.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.991+0000 7f6f77400640 1 -- 172.21.15.133:0/466566793 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78009510 con 0x7f6f8015cf90 2024-03-07T06:51:33.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.991+0000 7f6f85400640 1 -- 172.21.15.133:0/466566793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f800cba40 con 0x7f6f8015cf90 2024-03-07T06:51:33.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.991+0000 7f6f85400640 1 -- 172.21.15.133:0/466566793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f800b3580 con 0x7f6f8015cf90 2024-03-07T06:51:33.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.992+0000 7f6f6fe00640 1 -- 172.21.15.133:0/466566793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f800ce070 con 0x7f6f8015cf90 2024-03-07T06:51:33.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.993+0000 7f6f77400640 1 -- 172.21.15.133:0/466566793 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6f78046020 con 0x7f6f8015cf90 2024-03-07T06:51:33.994 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.994+0000 7f6f77400640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6f5c06b500 0x7f6f5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:33.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.994+0000 7f6f77400640 1 -- 172.21.15.133:0/466566793 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f780ac8a0 con 0x7f6f8015cf90 2024-03-07T06:51:33.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.994+0000 7f6f7e800640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6f5c06b500 0x7f6f5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:33.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.995+0000 7f6f7e800640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6f5c06b500 0x7f6f5c06d9c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6f70006fd0 tx=0x7f6f70008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:33.998+0000 7f6f77400640 1 -- 172.21.15.133:0/466566793 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6f800ce070 con 0x7f6f8015cf90 2024-03-07T06:51:34.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.192+0000 7f6f6fe00640 1 -- 172.21.15.133:0/466566793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f80062c90 con 0x7f6f8015cf90 2024-03-07T06:51:34.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.193+0000 7f6f77400640 1 -- 172.21.15.133:0/466566793 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f7804d070 con 0x7f6f8015cf90 2024-03-07T06:51:34.193 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:51:34.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.195+0000 7f6f6fe00640 1 -- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6f5c06b500 msgr2=0x7f6f5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:34.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.195+0000 7f6f6fe00640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6f5c06b500 0x7f6f5c06d9c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6f70006fd0 tx=0x7f6f70008040 comp rx=0 tx=0).stop 2024-03-07T06:51:34.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.195+0000 7f6f6fe00640 1 -- 172.21.15.133:0/466566793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f8015cf90 msgr2=0x7f6f80137920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:34.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.195+0000 7f6f6fe00640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f8015cf90 0x7f6f80137920 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009f40 tx=0x7f6f78004a40 comp rx=0 tx=0).stop 2024-03-07T06:51:34.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.196+0000 7f6f6fe00640 1 -- 172.21.15.133:0/466566793 shutdown_connections 2024-03-07T06:51:34.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.196+0000 7f6f6fe00640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f8015cf90 0x7f6f80137920 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.196+0000 7f6f6fe00640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6f5c06b500 0x7f6f5c06d9c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.196+0000 7f6f6fe00640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f80145670 0x7f6f801373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.196+0000 7f6f6fe00640 1 --2- 172.21.15.133:0/466566793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f800cda20 0x7f6f800a9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.196+0000 7f6f6fe00640 1 -- 172.21.15.133:0/466566793 >> 172.21.15.133:0/466566793 conn(0x7f6f800b1d30 msgr2=0x7f6f80144810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:34.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.196+0000 7f6f6fe00640 1 -- 172.21.15.133:0/466566793 shutdown_connections 2024-03-07T06:51:34.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.196+0000 7f6f6fe00640 1 -- 172.21.15.133:0/466566793 wait complete. 2024-03-07T06:51:34.214 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:51:34.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.424+0000 7f031bc00640 1 Processor -- start 2024-03-07T06:51:34.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.424+0000 7f031bc00640 1 -- start start 2024-03-07T06:51:34.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f031bc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0314159ab0 0x7f031415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:34.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f031bc00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f03140b3a20 0x7f03140b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:34.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f031bc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f03140d1d30 0x7f03140d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:34.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f031bc00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03140b43c0 con 0x7f03140b3a20 2024-03-07T06:51:34.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f031bc00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031415d430 con 0x7f03140d1d30 2024-03-07T06:51:34.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f031bc00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0314159e90 con 0x7f0314159ab0 2024-03-07T06:51:34.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f0313e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f03140b3a20 0x7f03140b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:34.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f0313e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f03140b3a20 0x7f03140b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55944/0 (socket says 172.21.15.133:55944) 2024-03-07T06:51:34.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f0313e00640 1 -- 172.21.15.133:0/792611963 learned_addr learned my addr 172.21.15.133:0/792611963 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:34.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f0319c00640 1 --2- 172.21.15.133:0/792611963 >> v2:172.21.15.161:3300/0 conn(0x7f03140d1d30 0x7f03140d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:34.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f0313e00640 1 -- 172.21.15.133:0/792611963 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0314159ab0 msgr2=0x7f031415cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:34.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f0313e00640 1 --2- 172.21.15.133:0/792611963 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0314159ab0 0x7f031415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f0313e00640 1 -- 172.21.15.133:0/792611963 >> v2:172.21.15.161:3300/0 conn(0x7f03140d1d30 msgr2=0x7f03140d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:34.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f0313e00640 1 --2- 172.21.15.133:0/792611963 >> v2:172.21.15.161:3300/0 conn(0x7f03140d1d30 0x7f03140d2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f0313e00640 1 -- 172.21.15.133:0/792611963 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031415fde0 con 0x7f03140b3a20 2024-03-07T06:51:34.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.426+0000 7f0319c00640 1 --2- 172.21.15.133:0/792611963 >> v2:172.21.15.161:3300/0 conn(0x7f03140d1d30 0x7f03140d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:34.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.427+0000 7f0313e00640 1 --2- 172.21.15.133:0/792611963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140b3a20 0x7f03140b3e80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f03140b6a30 tx=0x7f030002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f53557132eb163 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:34.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.428+0000 7f0313400640 1 -- 172.21.15.133:0/792611963 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03000317d0 con 0x7f03140b3a20 2024-03-07T06:51:34.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.428+0000 7f0313400640 1 -- 172.21.15.133:0/792611963 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0300037070 con 0x7f03140b3a20 2024-03-07T06:51:34.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.428+0000 7f0313400640 1 -- 172.21.15.133:0/792611963 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0300031b90 con 0x7f03140b3a20 2024-03-07T06:51:34.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.428+0000 7f031bc00640 1 -- 172.21.15.133:0/792611963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140b3a20 msgr2=0x7f03140b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:34.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.428+0000 7f031bc00640 1 --2- 172.21.15.133:0/792611963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140b3a20 0x7f03140b3e80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f03140b6a30 tx=0x7f030002b870 comp rx=0 tx=0).stop 2024-03-07T06:51:34.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.428+0000 7f031bc00640 1 -- 172.21.15.133:0/792611963 shutdown_connections 2024-03-07T06:51:34.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.429+0000 7f031bc00640 1 --2- 172.21.15.133:0/792611963 >> v2:172.21.15.161:3300/0 conn(0x7f03140d1d30 0x7f03140d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.429+0000 7f031bc00640 1 --2- 172.21.15.133:0/792611963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140b3a20 0x7f03140b3e80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.429+0000 7f031bc00640 1 --2- 172.21.15.133:0/792611963 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0314159ab0 0x7f031415cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.429+0000 7f031bc00640 1 -- 172.21.15.133:0/792611963 >> 172.21.15.133:0/792611963 conn(0x7f0314070400 msgr2=0x7f03140707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:34.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.429+0000 7f031bc00640 1 -- 172.21.15.133:0/792611963 shutdown_connections 2024-03-07T06:51:34.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.429+0000 7f031bc00640 1 -- 172.21.15.133:0/792611963 wait complete. 2024-03-07T06:51:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.430+0000 7f031bc00640 1 Processor -- start 2024-03-07T06:51:34.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.430+0000 7f031bc00640 1 -- start start 2024-03-07T06:51:34.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.431+0000 7f031bc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03140b3a20 0x7f031415bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:34.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.431+0000 7f031bc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140d1d30 0x7f031413f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:34.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.431+0000 7f031bc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0314159ab0 0x7f031415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:34.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.431+0000 7f031bc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0314160680 con 0x7f03140d1d30 2024-03-07T06:51:34.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.431+0000 7f031bc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03141603a0 con 0x7f03140b3a20 2024-03-07T06:51:34.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.431+0000 7f031bc00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03140b6ec0 con 0x7f0314159ab0 2024-03-07T06:51:34.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.432+0000 7f0313e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140d1d30 0x7f031413f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:34.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.432+0000 7f0313e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140d1d30 0x7f031413f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55956/0 (socket says 172.21.15.133:55956) 2024-03-07T06:51:34.432 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.432+0000 7f0313e00640 1 -- 172.21.15.133:0/2361755811 learned_addr learned my addr 172.21.15.133:0/2361755811 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:34.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.432+0000 7f0319200640 1 --2- 172.21.15.133:0/2361755811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03140b3a20 0x7f031415bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:34.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.432+0000 7f0319c00640 1 --2- 172.21.15.133:0/2361755811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0314159ab0 0x7f031415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:34.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.432+0000 7f0313e00640 1 -- 172.21.15.133:0/2361755811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0314159ab0 msgr2=0x7f031415a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:34.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.432+0000 7f0313e00640 1 --2- 172.21.15.133:0/2361755811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0314159ab0 0x7f031415a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.432+0000 7f0313e00640 1 -- 172.21.15.133:0/2361755811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03140b3a20 msgr2=0x7f031415bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:34.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.432+0000 7f0313e00640 1 --2- 172.21.15.133:0/2361755811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03140b3a20 0x7f031415bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.432+0000 7f0313e00640 1 -- 172.21.15.133:0/2361755811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031413fcc0 con 0x7f03140d1d30 2024-03-07T06:51:34.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.433+0000 7f0313e00640 1 --2- 172.21.15.133:0/2361755811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140d1d30 0x7f031413f780 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0300000c00 tx=0x7f03000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:34.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.433+0000 7f0310c00640 1 -- 172.21.15.133:0/2361755811 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0300032870 con 0x7f03140d1d30 2024-03-07T06:51:34.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.433+0000 7f0310c00640 1 -- 172.21.15.133:0/2361755811 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0300032a10 con 0x7f03140d1d30 2024-03-07T06:51:34.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.433+0000 7f0310c00640 1 -- 172.21.15.133:0/2361755811 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0300032e00 con 0x7f03140d1d30 2024-03-07T06:51:34.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.433+0000 7f031bc00640 1 -- 172.21.15.133:0/2361755811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031415aac0 con 0x7f03140d1d30 2024-03-07T06:51:34.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.433+0000 7f031bc00640 1 -- 172.21.15.133:0/2361755811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031415b080 con 0x7f03140d1d30 2024-03-07T06:51:34.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.434+0000 7f031bc00640 1 -- 172.21.15.133:0/2361755811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0314062c90 con 0x7f03140d1d30 2024-03-07T06:51:34.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.435+0000 7f0310c00640 1 -- 172.21.15.133:0/2361755811 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f03000453f0 con 0x7f03140d1d30 2024-03-07T06:51:34.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.436+0000 7f0310c00640 1 --2- 172.21.15.133:0/2361755811 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f02f406b4d0 0x7f02f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:34.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.436+0000 7f0310c00640 1 -- 172.21.15.133:0/2361755811 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03000ac4d0 con 0x7f03140d1d30 2024-03-07T06:51:34.436 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.436+0000 7f0319200640 1 --2- 172.21.15.133:0/2361755811 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f02f406b4d0 0x7f02f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:34.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.438+0000 7f0319200640 1 --2- 172.21.15.133:0/2361755811 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f02f406b4d0 0x7f02f406d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f030800b810 tx=0x7f0308008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:34.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.440+0000 7f0310c00640 1 -- 172.21.15.133:0/2361755811 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0300040070 con 0x7f03140d1d30 2024-03-07T06:51:34.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.624+0000 7f031bc00640 1 -- 172.21.15.133:0/2361755811 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03140680a0 con 0x7f03140d1d30 2024-03-07T06:51:34.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.625+0000 7f0310c00640 1 -- 172.21.15.133:0/2361755811 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03000aae20 con 0x7f03140d1d30 2024-03-07T06:51:34.624 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:51:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.627+0000 7f031bc00640 1 -- 172.21.15.133:0/2361755811 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f02f406b4d0 msgr2=0x7f02f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.627+0000 7f031bc00640 1 --2- 172.21.15.133:0/2361755811 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f02f406b4d0 0x7f02f406d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f030800b810 tx=0x7f0308008040 comp rx=0 tx=0).stop 2024-03-07T06:51:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.627+0000 7f031bc00640 1 -- 172.21.15.133:0/2361755811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140d1d30 msgr2=0x7f031413f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.627+0000 7f031bc00640 1 --2- 172.21.15.133:0/2361755811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140d1d30 0x7f031413f780 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0300000c00 tx=0x7f03000046c0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.627+0000 7f031bc00640 1 -- 172.21.15.133:0/2361755811 shutdown_connections 2024-03-07T06:51:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.627+0000 7f031bc00640 1 --2- 172.21.15.133:0/2361755811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0314159ab0 0x7f031415a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.627+0000 7f031bc00640 1 --2- 172.21.15.133:0/2361755811 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03140d1d30 0x7f031413f780 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.627+0000 7f031bc00640 1 --2- 172.21.15.133:0/2361755811 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f02f406b4d0 0x7f02f406d990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.627+0000 7f031bc00640 1 --2- 172.21.15.133:0/2361755811 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f03140b3a20 0x7f031415bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.627+0000 7f031bc00640 1 -- 172.21.15.133:0/2361755811 >> 172.21.15.133:0/2361755811 conn(0x7f0314070400 msgr2=0x7f03140d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.627+0000 7f031bc00640 1 -- 172.21.15.133:0/2361755811 shutdown_connections 2024-03-07T06:51:34.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.627+0000 7f031bc00640 1 -- 172.21.15.133:0/2361755811 wait complete. 2024-03-07T06:51:34.643 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:51:34.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.842+0000 7f6ca2e00640 1 Processor -- start 2024-03-07T06:51:34.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.842+0000 7f6ca2e00640 1 -- start start 2024-03-07T06:51:34.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.844+0000 7f6ca2e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6c9c15b370 0x7f6c9c0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:34.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.844+0000 7f6ca2e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6c9c0b3ed0 0x7f6c9c0cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:34.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.844+0000 7f6ca2e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c9c06d740 con 0x7f6c9c06d180 2024-03-07T06:51:34.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.844+0000 7f6ca2e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9c06da20 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.844+0000 7f6ca2e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9c06dd00 con 0x7f6c9c15b370 2024-03-07T06:51:34.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.844+0000 7f6ca0e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6c9c0b3ed0 0x7f6c9c0cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:34.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.844+0000 7f6c9b400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6c9c06d180 0x7f6c9c15b750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46928/0 (socket says 172.21.15.133:46928) 2024-03-07T06:51:34.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.844+0000 7f6c9b400640 1 -- 172.21.15.133:0/1971838119 learned_addr learned my addr 172.21.15.133:0/1971838119 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:34.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.844+0000 7f6c9be00640 1 --2- 172.21.15.133:0/1971838119 >> v2:172.21.15.161:3300/0 conn(0x7f6c9c15b370 0x7f6c9c0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:34.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.845+0000 7f6ca0e00640 1 -- 172.21.15.133:0/1971838119 >> v2:172.21.15.161:3300/0 conn(0x7f6c9c15b370 msgr2=0x7f6c9c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:34.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.845+0000 7f6ca0e00640 1 --2- 172.21.15.133:0/1971838119 >> v2:172.21.15.161:3300/0 conn(0x7f6c9c15b370 0x7f6c9c0cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.845+0000 7f6ca0e00640 1 -- 172.21.15.133:0/1971838119 >> v1:172.21.15.133:6789/0 conn(0x7f6c9c06d180 legacy=0x7f6c9c15b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:34.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.845+0000 7f6ca0e00640 1 -- 172.21.15.133:0/1971838119 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c06dfe0 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.845+0000 7f6c9be00640 1 --2- 172.21.15.133:0/1971838119 >> v2:172.21.15.161:3300/0 conn(0x7f6c9c15b370 0x7f6c9c0cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:51:34.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.846+0000 7f6ca0e00640 1 --2- 172.21.15.133:0/1971838119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9c0b3ed0 0x7f6c9c0cf490 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c073540 tx=0x7f6c9402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1ea287ba0a038a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:34.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.846+0000 7f6c9aa00640 1 -- 172.21.15.133:0/1971838119 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94039070 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.846+0000 7f6c9aa00640 1 -- 172.21.15.133:0/1971838119 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9402cc70 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.846+0000 7f6c9aa00640 1 -- 172.21.15.133:0/1971838119 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94034660 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.847+0000 7f6ca2e00640 1 -- 172.21.15.133:0/1971838119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9c0b3ed0 msgr2=0x7f6c9c0cf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:34.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.847+0000 7f6ca2e00640 1 --2- 172.21.15.133:0/1971838119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9c0b3ed0 0x7f6c9c0cf490 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c073540 tx=0x7f6c9402bec0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.847+0000 7f6ca2e00640 1 -- 172.21.15.133:0/1971838119 shutdown_connections 2024-03-07T06:51:34.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.847+0000 7f6ca2e00640 1 --2- 172.21.15.133:0/1971838119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9c0b3ed0 0x7f6c9c0cf490 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.847+0000 7f6ca2e00640 1 --2- 172.21.15.133:0/1971838119 >> v2:172.21.15.161:3300/0 conn(0x7f6c9c15b370 0x7f6c9c0cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.847+0000 7f6ca2e00640 1 -- 172.21.15.133:0/1971838119 >> 172.21.15.133:0/1971838119 conn(0x7f6c9c0b1d30 msgr2=0x7f6c9c0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:34.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.847+0000 7f6ca2e00640 1 -- 172.21.15.133:0/1971838119 shutdown_connections 2024-03-07T06:51:34.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.847+0000 7f6ca2e00640 1 -- 172.21.15.133:0/1971838119 wait complete. 2024-03-07T06:51:34.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.849+0000 7f6ca2e00640 1 Processor -- start 2024-03-07T06:51:34.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.849+0000 7f6ca2e00640 1 -- start start 2024-03-07T06:51:34.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.850+0000 7f6ca2e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c9c06d180 0x7f6c9c149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:34.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.850+0000 7f6ca2e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9c0b3ed0 0x7f6c9c156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:34.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.850+0000 7f6ca2e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c9c15b370 0x7f6c9c139590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:34.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.850+0000 7f6ca2e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c15bfe0 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.850+0000 7f6ca2e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c15be60 con 0x7f6c9c15b370 2024-03-07T06:51:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.850+0000 7f6ca2e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c15c160 con 0x7f6c9c06d180 2024-03-07T06:51:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.850+0000 7f6c9b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9c0b3ed0 0x7f6c9c156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.850+0000 7f6c9b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9c0b3ed0 0x7f6c9c156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55966/0 (socket says 172.21.15.133:55966) 2024-03-07T06:51:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.850+0000 7f6c9b400640 1 -- 172.21.15.133:0/3532543525 learned_addr learned my addr 172.21.15.133:0/3532543525 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.851+0000 7f6c9be00640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c9c06d180 0x7f6c9c149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.851+0000 7f6ca0e00640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c9c15b370 0x7f6c9c139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.851+0000 7f6c9b400640 1 -- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c9c06d180 msgr2=0x7f6c9c149960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.851+0000 7f6c9b400640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c9c06d180 0x7f6c9c149960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.851+0000 7f6c9b400640 1 -- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c9c15b370 msgr2=0x7f6c9c139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.851+0000 7f6c9b400640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c9c15b370 0x7f6c9c139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.851+0000 7f6c9b400640 1 -- 172.21.15.133:0/3532543525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c147930 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.851+0000 7f6ca0e00640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c9c15b370 0x7f6c9c139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.851+0000 7f6c9b400640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9c0b3ed0 0x7f6c9c156040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800c400 tx=0x7f6c8800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:34.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.851+0000 7f6c93e00640 1 -- 172.21.15.133:0/3532543525 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88017070 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.851+0000 7f6c93e00640 1 -- 172.21.15.133:0/3532543525 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c88005ce0 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.851+0000 7f6c93e00640 1 -- 172.21.15.133:0/3532543525 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c880072f0 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.851+0000 7f6ca2e00640 1 -- 172.21.15.133:0/3532543525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9c149f40 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.851+0000 7f6ca2e00640 1 -- 172.21.15.133:0/3532543525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9c14a800 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.852+0000 7f6ca2e00640 1 -- 172.21.15.133:0/3532543525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9c062c90 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.853+0000 7f6c93e00640 1 -- 172.21.15.133:0/3532543525 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6c8801a070 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.854+0000 7f6c93e00640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6c7c06b4d0 0x7f6c7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:34.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.854+0000 7f6c93e00640 1 -- 172.21.15.133:0/3532543525 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c880896e0 con 0x7f6c9c0b3ed0 2024-03-07T06:51:34.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.855+0000 7f6c9be00640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6c7c06b4d0 0x7f6c7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:34.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.856+0000 7f6c9be00640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6c7c06b4d0 0x7f6c7c06d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0096d0 tx=0x7f6c8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:34.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:34.859+0000 7f6c93e00640 1 -- 172.21.15.133:0/3532543525 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6c880584c0 con 0x7f6c9c0b3ed0 2024-03-07T06:51:35.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.053+0000 7f6ca2e00640 1 -- 172.21.15.133:0/3532543525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6c9c1548d0 con 0x7f6c9c0b3ed0 2024-03-07T06:51:35.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.054+0000 7f6c93e00640 1 -- 172.21.15.133:0/3532543525 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6c88057dc0 con 0x7f6c9c0b3ed0 2024-03-07T06:51:35.054 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:51:35.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.057+0000 7f6ca2e00640 1 -- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6c7c06b4d0 msgr2=0x7f6c7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.057+0000 7f6ca2e00640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6c7c06b4d0 0x7f6c7c06d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0096d0 tx=0x7f6c8c008040 comp rx=0 tx=0).stop 2024-03-07T06:51:35.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.057+0000 7f6ca2e00640 1 -- 172.21.15.133:0/3532543525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9c0b3ed0 msgr2=0x7f6c9c156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.057+0000 7f6ca2e00640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9c0b3ed0 0x7f6c9c156040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800c400 tx=0x7f6c8800c8c0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.057+0000 7f6ca2e00640 1 -- 172.21.15.133:0/3532543525 shutdown_connections 2024-03-07T06:51:35.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.057+0000 7f6ca2e00640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6c7c06b4d0 0x7f6c7c06d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.057+0000 7f6ca2e00640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6c9c15b370 0x7f6c9c139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.057+0000 7f6ca2e00640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c9c0b3ed0 0x7f6c9c156040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.057+0000 7f6ca2e00640 1 --2- 172.21.15.133:0/3532543525 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6c9c06d180 0x7f6c9c149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.057+0000 7f6ca2e00640 1 -- 172.21.15.133:0/3532543525 >> 172.21.15.133:0/3532543525 conn(0x7f6c9c0b1d30 msgr2=0x7f6c9c0724a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:35.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.057+0000 7f6ca2e00640 1 -- 172.21.15.133:0/3532543525 shutdown_connections 2024-03-07T06:51:35.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.058+0000 7f6ca2e00640 1 -- 172.21.15.133:0/3532543525 wait complete. 2024-03-07T06:51:35.075 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:51:35.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.287+0000 7f3964e00640 1 Processor -- start 2024-03-07T06:51:35.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.287+0000 7f3964e00640 1 -- start start 2024-03-07T06:51:35.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.288+0000 7f3964e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39600b70d0 0x7f39600b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:35.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.288+0000 7f3964e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3960143810 con 0x7f39600d1af0 2024-03-07T06:51:35.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.288+0000 7f3964e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3960143af0 con 0x7f3960140cb0 2024-03-07T06:51:35.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.288+0000 7f3964e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39600b7a70 con 0x7f39600b70d0 2024-03-07T06:51:35.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.288+0000 7f395fe00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f39600d1af0 0x7f39600d1f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46938/0 (socket says 172.21.15.133:46938) 2024-03-07T06:51:35.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.288+0000 7f395fe00640 1 -- 172.21.15.133:0/838612669 learned_addr learned my addr 172.21.15.133:0/838612669 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:35.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395ea00640 1 --2- 172.21.15.133:0/838612669 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39600b70d0 0x7f39600b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:35.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 -- 172.21.15.133:0/838612669 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651337488 0 0) 0x7f3960143810 con 0x7f39600d1af0 2024-03-07T06:51:35.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 -- 172.21.15.133:0/838612669 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3944003520 con 0x7f39600d1af0 2024-03-07T06:51:35.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 -- 172.21.15.133:0/838612669 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3185948182 0 0) 0x7f3944003520 con 0x7f39600d1af0 2024-03-07T06:51:35.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 -- 172.21.15.133:0/838612669 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39600b70d0 msgr2=0x7f39600b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:35.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 --2- 172.21.15.133:0/838612669 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39600b70d0 0x7f39600b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 -- 172.21.15.133:0/838612669 >> v1:172.21.15.161:6789/0 conn(0x7f3960140cb0 legacy=0x7f3960162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 -- 172.21.15.133:0/838612669 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3960143dd0 con 0x7f39600d1af0 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 -- 172.21.15.133:0/838612669 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f3958001670 con 0x7f39600d1af0 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 -- 172.21.15.133:0/838612669 >> v1:172.21.15.133:6789/0 conn(0x7f39600d1af0 legacy=0x7f39600d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 --2- 172.21.15.133:0/838612669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39440039e0 0x7f3944003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 --2- 172.21.15.133:0/838612669 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39600b70d0 0x7f3944004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 --2- 172.21.15.133:0/838612669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3944004d40 0x7f39440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 -- 172.21.15.133:0/838612669 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394400ba70 con 0x7f3944004d40 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 -- 172.21.15.133:0/838612669 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394400bc20 con 0x7f39440039e0 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.289+0000 7f395e000640 1 -- 172.21.15.133:0/838612669 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394400be20 con 0x7f39600b70d0 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.290+0000 7f395fe00640 1 --2- 172.21.15.133:0/838612669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3944004d40 0x7f39440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.290+0000 7f395ea00640 1 --2- 172.21.15.133:0/838612669 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39600b70d0 0x7f3944004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.290+0000 7f395fe00640 1 -- 172.21.15.133:0/838612669 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39600b70d0 msgr2=0x7f3944004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.290+0000 7f395fe00640 1 --2- 172.21.15.133:0/838612669 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39600b70d0 0x7f3944004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.290+0000 7f395fe00640 1 -- 172.21.15.133:0/838612669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39440039e0 msgr2=0x7f3944003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.290+0000 7f395f400640 1 --2- 172.21.15.133:0/838612669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39440039e0 0x7f3944003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:35.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.290+0000 7f395fe00640 1 --2- 172.21.15.133:0/838612669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39440039e0 0x7f3944003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.290+0000 7f395fe00640 1 -- 172.21.15.133:0/838612669 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3944003520 con 0x7f3944004d40 2024-03-07T06:51:35.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.291+0000 7f395fe00640 1 --2- 172.21.15.133:0/838612669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3944004d40 0x7f39440051c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f395800c210 tx=0x7f3944005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396a3cc875d0e9a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:35.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.291+0000 7f395e000640 1 -- 172.21.15.133:0/838612669 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3958007d10 con 0x7f3944004d40 2024-03-07T06:51:35.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.291+0000 7f395e000640 1 -- 172.21.15.133:0/838612669 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3958030070 con 0x7f3944004d40 2024-03-07T06:51:35.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.291+0000 7f395e000640 1 -- 172.21.15.133:0/838612669 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3958005d30 con 0x7f3944004d40 2024-03-07T06:51:35.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.292+0000 7f3964e00640 1 -- 172.21.15.133:0/838612669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3944004d40 msgr2=0x7f39440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.292+0000 7f3964e00640 1 --2- 172.21.15.133:0/838612669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3944004d40 0x7f39440051c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f395800c210 tx=0x7f3944005830 comp rx=0 tx=0).stop 2024-03-07T06:51:35.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.292+0000 7f395fe00640 1 -- 172.21.15.133:0/838612669 reap_dead start 2024-03-07T06:51:35.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.292+0000 7f3964e00640 1 -- 172.21.15.133:0/838612669 shutdown_connections 2024-03-07T06:51:35.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.292+0000 7f3964e00640 1 -- 172.21.15.133:0/838612669 >> 172.21.15.133:0/838612669 conn(0x7f39600674e0 msgr2=0x7f39600d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:35.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.292+0000 7f3964e00640 1 -- 172.21.15.133:0/838612669 shutdown_connections 2024-03-07T06:51:35.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.292+0000 7f3964e00640 1 -- 172.21.15.133:0/838612669 wait complete. 2024-03-07T06:51:35.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.294+0000 7f3964e00640 1 Processor -- start 2024-03-07T06:51:35.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.294+0000 7f3964e00640 1 -- start start 2024-03-07T06:51:35.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f3964e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f396013d810 0x7f39601f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:35.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f3964e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3960141090 0x7f39600b5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:35.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f3964e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f396013dbf0 0x7f39600b5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:35.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f3964e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3960144420 con 0x7f3960141090 2024-03-07T06:51:35.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f3964e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39601435c0 con 0x7f396013dbf0 2024-03-07T06:51:35.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f3964e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3960161bb0 con 0x7f396013d810 2024-03-07T06:51:35.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f395ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3960141090 0x7f39600b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:35.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f395ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3960141090 0x7f39600b5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55982/0 (socket says 172.21.15.133:55982) 2024-03-07T06:51:35.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f395ea00640 1 -- 172.21.15.133:0/2959462781 learned_addr learned my addr 172.21.15.133:0/2959462781 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:35.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f395f400640 1 --2- 172.21.15.133:0/2959462781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f396013d810 0x7f39601f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:35.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f395fe00640 1 --2- 172.21.15.133:0/2959462781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f396013dbf0 0x7f39600b5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:35.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f395ea00640 1 -- 172.21.15.133:0/2959462781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f396013d810 msgr2=0x7f39601f7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f395ea00640 1 --2- 172.21.15.133:0/2959462781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f396013d810 0x7f39601f7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f395ea00640 1 -- 172.21.15.133:0/2959462781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f396013dbf0 msgr2=0x7f39600b5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.295+0000 7f395ea00640 1 --2- 172.21.15.133:0/2959462781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f396013dbf0 0x7f39600b5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.296+0000 7f395ea00640 1 -- 172.21.15.133:0/2959462781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39601f8370 con 0x7f3960141090 2024-03-07T06:51:35.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.296+0000 7f395ea00640 1 --2- 172.21.15.133:0/2959462781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3960141090 0x7f39600b5800 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f394c011630 tx=0x7f394c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:35.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.296+0000 7f3957400640 1 -- 172.21.15.133:0/2959462781 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c00f920 con 0x7f3960141090 2024-03-07T06:51:35.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.296+0000 7f3957400640 1 -- 172.21.15.133:0/2959462781 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394c00fac0 con 0x7f3960141090 2024-03-07T06:51:35.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.296+0000 7f3964e00640 1 -- 172.21.15.133:0/2959462781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39601429b0 con 0x7f3960141090 2024-03-07T06:51:35.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.296+0000 7f3957400640 1 -- 172.21.15.133:0/2959462781 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c00fe70 con 0x7f3960141090 2024-03-07T06:51:35.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.296+0000 7f3964e00640 1 -- 172.21.15.133:0/2959462781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3960141830 con 0x7f3960141090 2024-03-07T06:51:35.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.297+0000 7f3964e00640 1 -- 172.21.15.133:0/2959462781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39600ca380 con 0x7f3960141090 2024-03-07T06:51:35.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.298+0000 7f3957400640 1 -- 172.21.15.133:0/2959462781 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f394c015460 con 0x7f3960141090 2024-03-07T06:51:35.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.299+0000 7f3957400640 1 --2- 172.21.15.133:0/2959462781 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f394006b4d0 0x7f394006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:35.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.300+0000 7f3957400640 1 -- 172.21.15.133:0/2959462781 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f394c08abb0 con 0x7f3960141090 2024-03-07T06:51:35.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.300+0000 7f395f400640 1 --2- 172.21.15.133:0/2959462781 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f394006b4d0 0x7f394006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:35.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.301+0000 7f395f400640 1 --2- 172.21.15.133:0/2959462781 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f394006b4d0 0x7f394006d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3950007fd0 tx=0x7f3950009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.304+0000 7f3957400640 1 -- 172.21.15.133:0/2959462781 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f39601f8370 con 0x7f3960141090 2024-03-07T06:51:35.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.542+0000 7f3964e00640 1 -- 172.21.15.133:0/2959462781 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3960062c90 con 0x7f3960141090 2024-03-07T06:51:35.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.543+0000 7f3957400640 1 -- 172.21.15.133:0/2959462781 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4983 (secure 0 0 0) 0x7f394c019030 con 0x7f3960141090 2024-03-07T06:51:35.543 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:35.543 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1116649196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1116649196},{"type":"v1","addr":"172.21.15.161:6835","nonce":1116649196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3485625105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3485625105},{"type":"v1","addr":"172.21.15.133:6835","nonce":3485625105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3866978630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3866978630},{"type":"v1","addr":"172.21.15.133:6833","nonce":3866978630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:51:14.594513+0000","modified":"2024-03-07T06:51:17.675044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2885273912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2885273912},{"type":"v1","addr":"172.21.15.161:6837","nonce":2885273912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-03-07T06:51:35.543 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 8 2024-03-07T06:51:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.546+0000 7f3964e00640 1 -- 172.21.15.133:0/2959462781 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f394006b4d0 msgr2=0x7f394006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.546+0000 7f3964e00640 1 --2- 172.21.15.133:0/2959462781 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f394006b4d0 0x7f394006d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3950007fd0 tx=0x7f3950009040 comp rx=0 tx=0).stop 2024-03-07T06:51:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.546+0000 7f3964e00640 1 -- 172.21.15.133:0/2959462781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3960141090 msgr2=0x7f39600b5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.546+0000 7f3964e00640 1 --2- 172.21.15.133:0/2959462781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3960141090 0x7f39600b5800 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f394c011630 tx=0x7f394c011af0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.546+0000 7f3964e00640 1 -- 172.21.15.133:0/2959462781 shutdown_connections 2024-03-07T06:51:35.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.546+0000 7f3964e00640 1 --2- 172.21.15.133:0/2959462781 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f394006b4d0 0x7f394006d990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.546+0000 7f3964e00640 1 --2- 172.21.15.133:0/2959462781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f396013dbf0 0x7f39600b5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.546+0000 7f3964e00640 1 --2- 172.21.15.133:0/2959462781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3960141090 0x7f39600b5800 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.546+0000 7f3964e00640 1 --2- 172.21.15.133:0/2959462781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f396013d810 0x7f39601f7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.546+0000 7f3964e00640 1 -- 172.21.15.133:0/2959462781 >> 172.21.15.133:0/2959462781 conn(0x7f39600674e0 msgr2=0x7f3960160c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:35.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.546+0000 7f3964e00640 1 -- 172.21.15.133:0/2959462781 shutdown_connections 2024-03-07T06:51:35.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.546+0000 7f3964e00640 1 -- 172.21.15.133:0/2959462781 wait complete. 2024-03-07T06:51:35.565 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:35.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.774+0000 7fb54f400640 1 Processor -- start 2024-03-07T06:51:35.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.774+0000 7fb54f400640 1 -- start start 2024-03-07T06:51:35.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.775+0000 7fb54f400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb54815b1b0 0x7fb5480b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:35.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.775+0000 7fb54f400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb5480af950 0x7fb5480cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:35.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.775+0000 7fb54f400640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54806f140 con 0x7fb5480ce420 2024-03-07T06:51:35.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.775+0000 7fb54f400640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb548071610 con 0x7fb5480af950 2024-03-07T06:51:35.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.775+0000 7fb54f400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480ce850 con 0x7fb54815b1b0 2024-03-07T06:51:35.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.776+0000 7fb547e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb5480af950 0x7fb5480cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:35.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.776+0000 7fb547e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb5480af950 0x7fb5480cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55984/0 (socket says 172.21.15.133:55984) 2024-03-07T06:51:35.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.776+0000 7fb54d400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb5480ce420 0x7fb54806de20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46950/0 (socket says 172.21.15.133:46950) 2024-03-07T06:51:35.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.776+0000 7fb547e00640 1 -- 172.21.15.133:0/1666828167 learned_addr learned my addr 172.21.15.133:0/1666828167 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:35.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.776+0000 7fb54ca00640 1 --2- 172.21.15.133:0/1666828167 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb54815b1b0 0x7fb5480b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:35.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.776+0000 7fb547e00640 1 -- 172.21.15.133:0/1666828167 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb54815b1b0 msgr2=0x7fb5480b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:35.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.776+0000 7fb547e00640 1 --2- 172.21.15.133:0/1666828167 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb54815b1b0 0x7fb5480b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.776+0000 7fb547e00640 1 -- 172.21.15.133:0/1666828167 >> v1:172.21.15.133:6789/0 conn(0x7fb5480ce420 legacy=0x7fb54806de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.776+0000 7fb547e00640 1 -- 172.21.15.133:0/1666828167 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54806f420 con 0x7fb5480af950 2024-03-07T06:51:35.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.776+0000 7fb547400640 1 -- 172.21.15.133:0/1666828167 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1642796570 0 0) 0x7fb54806f140 con 0x7fb5480ce420 2024-03-07T06:51:35.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.777+0000 7fb547e00640 1 --2- 172.21.15.133:0/1666828167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480af950 0x7fb5480cdee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb548071790 tx=0x7fb53402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6933b71fe24794 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:35.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.777+0000 7fb547400640 1 -- 172.21.15.133:0/1666828167 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534035020 con 0x7fb5480af950 2024-03-07T06:51:35.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.778+0000 7fb547400640 1 -- 172.21.15.133:0/1666828167 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb534031640 con 0x7fb5480af950 2024-03-07T06:51:35.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.778+0000 7fb547400640 1 -- 172.21.15.133:0/1666828167 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53402d040 con 0x7fb5480af950 2024-03-07T06:51:35.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.778+0000 7fb54f400640 1 -- 172.21.15.133:0/1666828167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480af950 msgr2=0x7fb5480cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.778+0000 7fb54f400640 1 --2- 172.21.15.133:0/1666828167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480af950 0x7fb5480cdee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb548071790 tx=0x7fb53402b8f0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.778+0000 7fb54f400640 1 -- 172.21.15.133:0/1666828167 shutdown_connections 2024-03-07T06:51:35.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.778+0000 7fb54f400640 1 --2- 172.21.15.133:0/1666828167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480af950 0x7fb5480cdee0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.778+0000 7fb54f400640 1 --2- 172.21.15.133:0/1666828167 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb54815b1b0 0x7fb5480b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.778+0000 7fb54f400640 1 -- 172.21.15.133:0/1666828167 >> 172.21.15.133:0/1666828167 conn(0x7fb5480b3120 msgr2=0x7fb5480b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:35.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.778+0000 7fb54f400640 1 -- 172.21.15.133:0/1666828167 shutdown_connections 2024-03-07T06:51:35.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.778+0000 7fb54f400640 1 -- 172.21.15.133:0/1666828167 wait complete. 2024-03-07T06:51:35.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.780+0000 7fb54f400640 1 Processor -- start 2024-03-07T06:51:35.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.780+0000 7fb54f400640 1 -- start start 2024-03-07T06:51:35.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.781+0000 7fb54f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480af950 0x7fb548158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:35.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.781+0000 7fb54f400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5480ce420 0x7fb54815b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:35.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.781+0000 7fb54f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb54815b1b0 0x7fb54814d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:35.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.781+0000 7fb54f400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54815b590 con 0x7fb5480af950 2024-03-07T06:51:35.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.781+0000 7fb54f400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480cc880 con 0x7fb54815b1b0 2024-03-07T06:51:35.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.781+0000 7fb54f400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54815b710 con 0x7fb5480ce420 2024-03-07T06:51:35.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.781+0000 7fb54ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480af950 0x7fb548158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:35.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.781+0000 7fb54ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480af950 0x7fb548158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55998/0 (socket says 172.21.15.133:55998) 2024-03-07T06:51:35.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.781+0000 7fb54ca00640 1 -- 172.21.15.133:0/3386101581 learned_addr learned my addr 172.21.15.133:0/3386101581 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.782+0000 7fb547e00640 1 --2- 172.21.15.133:0/3386101581 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5480ce420 0x7fb54815b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.782+0000 7fb54ca00640 1 -- 172.21.15.133:0/3386101581 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5480ce420 msgr2=0x7fb54815b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.782+0000 7fb54ca00640 1 --2- 172.21.15.133:0/3386101581 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5480ce420 0x7fb54815b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.782+0000 7fb54ca00640 1 -- 172.21.15.133:0/3386101581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb54815b1b0 msgr2=0x7fb54814d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.782+0000 7fb54ca00640 1 --2- 172.21.15.133:0/3386101581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb54815b1b0 0x7fb54814d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.782+0000 7fb54ca00640 1 -- 172.21.15.133:0/3386101581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54815bf00 con 0x7fb5480af950 2024-03-07T06:51:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.782+0000 7fb54ca00640 1 --2- 172.21.15.133:0/3386101581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480af950 0x7fb548158110 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00d620 tx=0x7fb53c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.782+0000 7fb544c00640 1 -- 172.21.15.133:0/3386101581 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c00b620 con 0x7fb5480af950 2024-03-07T06:51:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.782+0000 7fb544c00640 1 -- 172.21.15.133:0/3386101581 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c019070 con 0x7fb5480af950 2024-03-07T06:51:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.782+0000 7fb544c00640 1 -- 172.21.15.133:0/3386101581 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c00ba00 con 0x7fb5480af950 2024-03-07T06:51:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.782+0000 7fb54f400640 1 -- 172.21.15.133:0/3386101581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54815c1e0 con 0x7fb5480af950 2024-03-07T06:51:35.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.782+0000 7fb54f400640 1 -- 172.21.15.133:0/3386101581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb548156230 con 0x7fb5480af950 2024-03-07T06:51:35.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.783+0000 7fb54f400640 1 -- 172.21.15.133:0/3386101581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb548062c90 con 0x7fb5480af950 2024-03-07T06:51:35.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.784+0000 7fb544c00640 1 -- 172.21.15.133:0/3386101581 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb53c00bba0 con 0x7fb5480af950 2024-03-07T06:51:35.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.785+0000 7fb544c00640 1 --2- 172.21.15.133:0/3386101581 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb52806b500 0x7fb52806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:35.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.785+0000 7fb544c00640 1 -- 172.21.15.133:0/3386101581 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb53c08a5f0 con 0x7fb5480af950 2024-03-07T06:51:35.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.786+0000 7fb547e00640 1 --2- 172.21.15.133:0/3386101581 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb52806b500 0x7fb52806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:35.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.787+0000 7fb547e00640 1 --2- 172.21.15.133:0/3386101581 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb52806b500 0x7fb52806d9c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb534036000 tx=0x7fb534008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:35.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.789+0000 7fb544c00640 1 -- 172.21.15.133:0/3386101581 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb53c059460 con 0x7fb5480af950 2024-03-07T06:51:35.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.983+0000 7fb54f400640 1 -- 172.21.15.133:0/3386101581 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5480cc660 con 0x7fb5480af950 2024-03-07T06:51:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.986+0000 7fb544c00640 1 -- 172.21.15.133:0/3386101581 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16653 (secure 0 0 0) 0x7fb53c058d60 con 0x7fb5480af950 2024-03-07T06:51:35.986 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:35.986 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":21,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:16.174650+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.989+0000 7fb54f400640 1 -- 172.21.15.133:0/3386101581 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb52806b500 msgr2=0x7fb52806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.989+0000 7fb54f400640 1 --2- 172.21.15.133:0/3386101581 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb52806b500 0x7fb52806d9c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb534036000 tx=0x7fb534008030 comp rx=0 tx=0).stop 2024-03-07T06:51:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.989+0000 7fb54f400640 1 -- 172.21.15.133:0/3386101581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480af950 msgr2=0x7fb548158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.989+0000 7fb54f400640 1 --2- 172.21.15.133:0/3386101581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480af950 0x7fb548158110 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00d620 tx=0x7fb53c00dae0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.989+0000 7fb54f400640 1 -- 172.21.15.133:0/3386101581 shutdown_connections 2024-03-07T06:51:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.989+0000 7fb54f400640 1 --2- 172.21.15.133:0/3386101581 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb52806b500 0x7fb52806d9c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.989+0000 7fb54f400640 1 --2- 172.21.15.133:0/3386101581 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb54815b1b0 0x7fb54814d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.989+0000 7fb54f400640 1 --2- 172.21.15.133:0/3386101581 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5480ce420 0x7fb54815b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.989+0000 7fb54f400640 1 --2- 172.21.15.133:0/3386101581 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5480af950 0x7fb548158110 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.989+0000 7fb54f400640 1 -- 172.21.15.133:0/3386101581 >> 172.21.15.133:0/3386101581 conn(0x7fb5480b3120 msgr2=0x7fb54806e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.989+0000 7fb54f400640 1 -- 172.21.15.133:0/3386101581 shutdown_connections 2024-03-07T06:51:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:35.989+0000 7fb54f400640 1 -- 172.21.15.133:0/3386101581 wait complete. 2024-03-07T06:51:36.008 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:51:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.202+0000 7f73cc200640 1 Processor -- start 2024-03-07T06:51:36.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.202+0000 7f73cc200640 1 -- start start 2024-03-07T06:51:36.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.203+0000 7f73cc200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f73c415fbb0 0x7f73c40cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.203+0000 7f73cc200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f73c4072950 0x7f73c40cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.203+0000 7f73cc200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c40731b0 con 0x7f73c415ff90 2024-03-07T06:51:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.203+0000 7f73cc200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c4072f60 con 0x7f73c415fbb0 2024-03-07T06:51:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.203+0000 7f73cc200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c4073490 con 0x7f73c4072950 2024-03-07T06:51:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.203+0000 7f73c9800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f73c415fbb0 0x7f73c40cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.204+0000 7f73c9800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f73c415fbb0 0x7f73c40cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56014/0 (socket says 172.21.15.133:56014) 2024-03-07T06:51:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.204+0000 7f73ca200640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f73c415ff90 0x7f73c40acd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46956/0 (socket says 172.21.15.133:46956) 2024-03-07T06:51:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.204+0000 7f73c9800640 1 -- 172.21.15.133:0/1143194322 learned_addr learned my addr 172.21.15.133:0/1143194322 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.204+0000 7f73c8e00640 1 --2- 172.21.15.133:0/1143194322 >> v2:172.21.15.161:3300/0 conn(0x7f73c4072950 0x7f73c40cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.204+0000 7f73c9800640 1 -- 172.21.15.133:0/1143194322 >> v2:172.21.15.161:3300/0 conn(0x7f73c4072950 msgr2=0x7f73c40cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.204+0000 7f73c9800640 1 --2- 172.21.15.133:0/1143194322 >> v2:172.21.15.161:3300/0 conn(0x7f73c4072950 0x7f73c40cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.204+0000 7f73c9800640 1 -- 172.21.15.133:0/1143194322 >> v1:172.21.15.133:6789/0 conn(0x7f73c415ff90 legacy=0x7f73c40acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:36.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.204+0000 7f73c9800640 1 -- 172.21.15.133:0/1143194322 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c4072150 con 0x7f73c415fbb0 2024-03-07T06:51:36.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.204+0000 7f73c3e00640 1 -- 172.21.15.133:0/1143194322 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1603761805 0 0) 0x7f73c40731b0 con 0x7f73c415ff90 2024-03-07T06:51:36.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.205+0000 7f73c9800640 1 --2- 172.21.15.133:0/1143194322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c415fbb0 0x7f73c40cf7d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f73c40cdd10 tx=0x7f73b802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82a65fcae75605b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:36.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.205+0000 7f73c3e00640 1 -- 172.21.15.133:0/1143194322 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8035020 con 0x7f73c415fbb0 2024-03-07T06:51:36.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.205+0000 7f73c3e00640 1 -- 172.21.15.133:0/1143194322 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b8031870 con 0x7f73c415fbb0 2024-03-07T06:51:36.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.205+0000 7f73c3e00640 1 -- 172.21.15.133:0/1143194322 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8031c40 con 0x7f73c415fbb0 2024-03-07T06:51:36.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.206+0000 7f73cc200640 1 -- 172.21.15.133:0/1143194322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c415fbb0 msgr2=0x7f73c40cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:36.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.206+0000 7f73cc200640 1 --2- 172.21.15.133:0/1143194322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c415fbb0 0x7f73c40cf7d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f73c40cdd10 tx=0x7f73b802b910 comp rx=0 tx=0).stop 2024-03-07T06:51:36.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.206+0000 7f73cc200640 1 -- 172.21.15.133:0/1143194322 shutdown_connections 2024-03-07T06:51:36.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.206+0000 7f73cc200640 1 --2- 172.21.15.133:0/1143194322 >> v2:172.21.15.161:3300/0 conn(0x7f73c4072950 0x7f73c40cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:36.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.206+0000 7f73cc200640 1 --2- 172.21.15.133:0/1143194322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c415fbb0 0x7f73c40cf7d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:36.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.206+0000 7f73cc200640 1 -- 172.21.15.133:0/1143194322 >> 172.21.15.133:0/1143194322 conn(0x7f73c40b3120 msgr2=0x7f73c40b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:36.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.206+0000 7f73cc200640 1 -- 172.21.15.133:0/1143194322 shutdown_connections 2024-03-07T06:51:36.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.206+0000 7f73cc200640 1 -- 172.21.15.133:0/1143194322 wait complete. 2024-03-07T06:51:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.208+0000 7f73cc200640 1 Processor -- start 2024-03-07T06:51:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.208+0000 7f73cc200640 1 -- start start 2024-03-07T06:51:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.209+0000 7f73cc200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c415fbb0 0x7f73c409c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.209+0000 7f73cc200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f73c415ff90 0x7f73c41f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.209+0000 7f73cc200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c40c5c10 0x7f73c415e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.209+0000 7f73cc200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c40b1a00 con 0x7f73c415fbb0 2024-03-07T06:51:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.209+0000 7f73cc200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c4160980 con 0x7f73c40c5c10 2024-03-07T06:51:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.209+0000 7f73cc200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c41603b0 con 0x7f73c415ff90 2024-03-07T06:51:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.209+0000 7f73c9800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c415fbb0 0x7f73c409c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.209+0000 7f73c9800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c415fbb0 0x7f73c409c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56024/0 (socket says 172.21.15.133:56024) 2024-03-07T06:51:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.209+0000 7f73c9800640 1 -- 172.21.15.133:0/3591156880 learned_addr learned my addr 172.21.15.133:0/3591156880 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.209+0000 7f73c9800640 1 -- 172.21.15.133:0/3591156880 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f73c415ff90 msgr2=0x7f73c41f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.209+0000 7f73c9800640 1 --2- 172.21.15.133:0/3591156880 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f73c415ff90 0x7f73c41f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.209+0000 7f73c9800640 1 -- 172.21.15.133:0/3591156880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c40c5c10 msgr2=0x7f73c415e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.209+0000 7f73c9800640 1 --2- 172.21.15.133:0/3591156880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c40c5c10 0x7f73c415e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.209+0000 7f73c9800640 1 -- 172.21.15.133:0/3591156880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c409c9d0 con 0x7f73c415fbb0 2024-03-07T06:51:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.210+0000 7f73c9800640 1 --2- 172.21.15.133:0/3591156880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c415fbb0 0x7f73c409c3c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f73b80090b0 tx=0x7f73b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.210+0000 7f73c1600640 1 -- 172.21.15.133:0/3591156880 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8041070 con 0x7f73c415fbb0 2024-03-07T06:51:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.210+0000 7f73c1600640 1 -- 172.21.15.133:0/3591156880 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b803ad00 con 0x7f73c415fbb0 2024-03-07T06:51:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.210+0000 7f73c1600640 1 -- 172.21.15.133:0/3591156880 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b8035040 con 0x7f73c415fbb0 2024-03-07T06:51:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.210+0000 7f73cc200640 1 -- 172.21.15.133:0/3591156880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c409ccb0 con 0x7f73c415fbb0 2024-03-07T06:51:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.210+0000 7f73cc200640 1 -- 172.21.15.133:0/3591156880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c409d270 con 0x7f73c415fbb0 2024-03-07T06:51:36.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.211+0000 7f73cc200640 1 -- 172.21.15.133:0/3591156880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c4062c90 con 0x7f73c415fbb0 2024-03-07T06:51:36.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.212+0000 7f73c1600640 1 -- 172.21.15.133:0/3591156880 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f73b8038020 con 0x7f73c415fbb0 2024-03-07T06:51:36.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.213+0000 7f73c1600640 1 --2- 172.21.15.133:0/3591156880 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f73a406b4d0 0x7f73a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:36.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.213+0000 7f73c1600640 1 -- 172.21.15.133:0/3591156880 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73b80ac1b0 con 0x7f73c415fbb0 2024-03-07T06:51:36.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.214+0000 7f73c8e00640 1 --2- 172.21.15.133:0/3591156880 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f73a406b4d0 0x7f73a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:36.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.215+0000 7f73c8e00640 1 --2- 172.21.15.133:0/3591156880 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f73a406b4d0 0x7f73a406d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f73b0006fd0 tx=0x7f73b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:36.217 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.217+0000 7f73c1600640 1 -- 172.21.15.133:0/3591156880 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f73b807af90 con 0x7f73c415fbb0 2024-03-07T06:51:36.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.450+0000 7f73cc200640 1 -- 172.21.15.133:0/3591156880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73c40c6040 con 0x7f73c415fbb0 2024-03-07T06:51:36.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.451+0000 7f73c1600640 1 -- 172.21.15.133:0/3591156880 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4983 (secure 0 0 0) 0x7f73b807a890 con 0x7f73c415fbb0 2024-03-07T06:51:36.451 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:36.451 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1116649196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1116649196},{"type":"v1","addr":"172.21.15.161:6835","nonce":1116649196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3485625105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3485625105},{"type":"v1","addr":"172.21.15.133:6835","nonce":3485625105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3866978630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3866978630},{"type":"v1","addr":"172.21.15.133:6833","nonce":3866978630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:51:14.594513+0000","modified":"2024-03-07T06:51:17.675044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2885273912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2885273912},{"type":"v1","addr":"172.21.15.161:6837","nonce":2885273912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-03-07T06:51:36.451 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 8 2024-03-07T06:51:36.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.454+0000 7f73cc200640 1 -- 172.21.15.133:0/3591156880 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f73a406b4d0 msgr2=0x7f73a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:36.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.454+0000 7f73cc200640 1 --2- 172.21.15.133:0/3591156880 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f73a406b4d0 0x7f73a406d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f73b0006fd0 tx=0x7f73b0008040 comp rx=0 tx=0).stop 2024-03-07T06:51:36.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.454+0000 7f73cc200640 1 -- 172.21.15.133:0/3591156880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c415fbb0 msgr2=0x7f73c409c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:36.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.454+0000 7f73cc200640 1 --2- 172.21.15.133:0/3591156880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c415fbb0 0x7f73c409c3c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f73b80090b0 tx=0x7f73b80046c0 comp rx=0 tx=0).stop 2024-03-07T06:51:36.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.454+0000 7f73cc200640 1 -- 172.21.15.133:0/3591156880 shutdown_connections 2024-03-07T06:51:36.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.454+0000 7f73cc200640 1 --2- 172.21.15.133:0/3591156880 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f73a406b4d0 0x7f73a406d990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:36.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.454+0000 7f73cc200640 1 --2- 172.21.15.133:0/3591156880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f73c40c5c10 0x7f73c415e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:36.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.454+0000 7f73cc200640 1 --2- 172.21.15.133:0/3591156880 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f73c415ff90 0x7f73c41f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:36.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.454+0000 7f73cc200640 1 --2- 172.21.15.133:0/3591156880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73c415fbb0 0x7f73c409c3c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:36.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.454+0000 7f73cc200640 1 -- 172.21.15.133:0/3591156880 >> 172.21.15.133:0/3591156880 conn(0x7f73c40b3120 msgr2=0x7f73c40b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:36.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.454+0000 7f73cc200640 1 -- 172.21.15.133:0/3591156880 shutdown_connections 2024-03-07T06:51:36.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:36.454+0000 7f73cc200640 1 -- 172.21.15.133:0/3591156880 wait complete. 2024-03-07T06:51:36.874 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-07T06:51:14.594513+0000', 'modified': '2024-03-07T06:51:17.675044+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6837/2885273912', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6836', 'nonce': 2885273912}, {'type': 'v1', 'addr': '172.21.15.161:6837', 'nonce': 2885273912}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4288, 'qdb_cluster': [4288]} 2024-03-07T06:51:36.875 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-07T06:51:36.875 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-07T06:51:37.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.087+0000 7f5816e00640 1 Processor -- start 2024-03-07T06:51:37.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.088+0000 7f5816e00640 1 -- start start 2024-03-07T06:51:37.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.089+0000 7f5816e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f581015fbb0 0x7f58100cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.089+0000 7f5816e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5810072950 0x7f58100cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.089+0000 7f5816e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58100731b0 con 0x7f581015ff90 2024-03-07T06:51:37.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.089+0000 7f5816e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5810072f60 con 0x7f5810072950 2024-03-07T06:51:37.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.089+0000 7f5816e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5810073490 con 0x7f581015fbb0 2024-03-07T06:51:37.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.089+0000 7f5807400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5810072950 0x7f58100cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.089+0000 7f5814e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f581015ff90 0x7f58100acd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46964/0 (socket says 172.21.15.133:46964) 2024-03-07T06:51:37.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.089+0000 7f5814e00640 1 -- 172.21.15.133:0/1897572029 learned_addr learned my addr 172.21.15.133:0/1897572029 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:37.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.090+0000 7f580fe00640 1 --2- 172.21.15.133:0/1897572029 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f581015fbb0 0x7f58100cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.090+0000 7f5807400640 1 -- 172.21.15.133:0/1897572029 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f581015fbb0 msgr2=0x7f58100cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:37.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.090+0000 7f5807400640 1 --2- 172.21.15.133:0/1897572029 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f581015fbb0 0x7f58100cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.090+0000 7f5807400640 1 -- 172.21.15.133:0/1897572029 >> v1:172.21.15.133:6789/0 conn(0x7f581015ff90 legacy=0x7f58100acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:37.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.090+0000 7f5807400640 1 -- 172.21.15.133:0/1897572029 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5810072150 con 0x7f5810072950 2024-03-07T06:51:37.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.091+0000 7f5807400640 1 --2- 172.21.15.133:0/1897572029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5810072950 0x7f58100cdee0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009d90 tx=0x7f57f802bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a4695cdb7276a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:37.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.091+0000 7f580f400640 1 -- 172.21.15.133:0/1897572029 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8039070 con 0x7f5810072950 2024-03-07T06:51:37.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.091+0000 7f580f400640 1 -- 172.21.15.133:0/1897572029 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f802c980 con 0x7f5810072950 2024-03-07T06:51:37.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.091+0000 7f580f400640 1 -- 172.21.15.133:0/1897572029 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f802cd50 con 0x7f5810072950 2024-03-07T06:51:37.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.092+0000 7f5816e00640 1 -- 172.21.15.133:0/1897572029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5810072950 msgr2=0x7f58100cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:37.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.092+0000 7f5816e00640 1 --2- 172.21.15.133:0/1897572029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5810072950 0x7f58100cdee0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009d90 tx=0x7f57f802bc50 comp rx=0 tx=0).stop 2024-03-07T06:51:37.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.092+0000 7f5816e00640 1 -- 172.21.15.133:0/1897572029 shutdown_connections 2024-03-07T06:51:37.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.092+0000 7f5816e00640 1 --2- 172.21.15.133:0/1897572029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5810072950 0x7f58100cdee0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.092+0000 7f5816e00640 1 --2- 172.21.15.133:0/1897572029 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f581015fbb0 0x7f58100cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.092+0000 7f5816e00640 1 -- 172.21.15.133:0/1897572029 >> 172.21.15.133:0/1897572029 conn(0x7f58100b3120 msgr2=0x7f58100b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:37.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.092+0000 7f5816e00640 1 -- 172.21.15.133:0/1897572029 shutdown_connections 2024-03-07T06:51:37.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.092+0000 7f5816e00640 1 -- 172.21.15.133:0/1897572029 wait complete. 2024-03-07T06:51:37.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.094+0000 7f5816e00640 1 Processor -- start 2024-03-07T06:51:37.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.094+0000 7f5816e00640 1 -- start start 2024-03-07T06:51:37.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5816e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f581015fbb0 0x7f58100c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5816e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f581015ff90 0x7f5810155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5816e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58100b5af0 0x7f58100b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5816e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58100b1a00 con 0x7f58100b5af0 2024-03-07T06:51:37.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5816e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5810160980 con 0x7f581015fbb0 2024-03-07T06:51:37.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5816e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58101603a0 con 0x7f581015ff90 2024-03-07T06:51:37.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5814e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58100b5af0 0x7f58100b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5814e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58100b5af0 0x7f58100b5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56040/0 (socket says 172.21.15.133:56040) 2024-03-07T06:51:37.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5814e00640 1 -- 172.21.15.133:0/1824539239 learned_addr learned my addr 172.21.15.133:0/1824539239 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:37.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f580fe00640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f581015fbb0 0x7f58100c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5807400640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f581015ff90 0x7f5810155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5814e00640 1 -- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f581015ff90 msgr2=0x7f5810155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:37.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5814e00640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f581015ff90 0x7f5810155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5814e00640 1 -- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f581015fbb0 msgr2=0x7f58100c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:37.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5814e00640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f581015fbb0 0x7f58100c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5814e00640 1 -- 172.21.15.133:0/1824539239 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58100cb550 con 0x7f58100b5af0 2024-03-07T06:51:37.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.095+0000 7f5807400640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f581015ff90 0x7f5810155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:37.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.096+0000 7f5814e00640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58100b5af0 0x7f58100b5f80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f580800c780 tx=0x7f580800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:37.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.096+0000 7f580cc00640 1 -- 172.21.15.133:0/1824539239 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808018070 con 0x7f58100b5af0 2024-03-07T06:51:37.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.096+0000 7f580cc00640 1 -- 172.21.15.133:0/1824539239 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58080072d0 con 0x7f58100b5af0 2024-03-07T06:51:37.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.096+0000 7f580cc00640 1 -- 172.21.15.133:0/1824539239 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58080047e0 con 0x7f58100b5af0 2024-03-07T06:51:37.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.096+0000 7f5816e00640 1 -- 172.21.15.133:0/1824539239 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58100cb830 con 0x7f58100b5af0 2024-03-07T06:51:37.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.096+0000 7f5816e00640 1 -- 172.21.15.133:0/1824539239 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58100cbdf0 con 0x7f58100b5af0 2024-03-07T06:51:37.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.099+0000 7f580cc00640 1 -- 172.21.15.133:0/1824539239 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f580801b070 con 0x7f58100b5af0 2024-03-07T06:51:37.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.100+0000 7f580cc00640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57ec06b4d0 0x7f57ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.100+0000 7f580cc00640 1 -- 172.21.15.133:0/1824539239 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5808089670 con 0x7f58100b5af0 2024-03-07T06:51:37.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.100+0000 7f580fe00640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57ec06b4d0 0x7f57ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.101+0000 7f5819c40740 1 Processor -- start 2024-03-07T06:51:37.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.101+0000 7f5819c40740 1 -- start start 2024-03-07T06:51:37.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.101+0000 7f580fe00640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57ec06b4d0 0x7f57ec06d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5800006fd0 tx=0x7f5800008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:37.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5819c40740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c589610 0x56256c55c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5819c40740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c5899f0 0x56256c58c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5819c40740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c55caf0 0x56256c58c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5819c40740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256c4c66c0 con 0x56256c55caf0 2024-03-07T06:51:37.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5819c40740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256c46f9b0 con 0x56256c5899f0 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5819c40740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x56256c4c2760 con 0x56256c589610 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5814e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c55caf0 0x56256c58c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5814e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c55caf0 0x56256c58c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56046/0 (socket says 172.21.15.133:56046) 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5814e00640 1 -- 172.21.15.133:0/1749018079 learned_addr learned my addr 172.21.15.133:0/1749018079 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5807400640 1 --2- 172.21.15.133:0/1749018079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c5899f0 0x56256c58c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f580fe00640 1 --2- 172.21.15.133:0/1749018079 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c589610 0x56256c55c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5814e00640 1 -- 172.21.15.133:0/1749018079 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c589610 msgr2=0x56256c55c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5814e00640 1 --2- 172.21.15.133:0/1749018079 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c589610 0x56256c55c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5814e00640 1 -- 172.21.15.133:0/1749018079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c5899f0 msgr2=0x56256c58c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5814e00640 1 --2- 172.21.15.133:0/1749018079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c5899f0 0x56256c58c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5814e00640 1 -- 172.21.15.133:0/1749018079 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56256c4636f0 con 0x56256c55caf0 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f580fe00640 1 --2- 172.21.15.133:0/1749018079 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c589610 0x56256c55c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.102+0000 7f5814e00640 1 --2- 172.21.15.133:0/1749018079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c55caf0 0x56256c58c8e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5808000fb0 tx=0x7f5808057eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1306f5400faed39 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.103+0000 7f5804c00640 1 -- 172.21.15.133:0/1749018079 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580805ce60 con 0x56256c55caf0 2024-03-07T06:51:37.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.103+0000 7f5804c00640 1 -- 172.21.15.133:0/1749018079 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58080012b0 con 0x56256c55caf0 2024-03-07T06:51:37.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.103+0000 7f5804c00640 1 -- 172.21.15.133:0/1749018079 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808054460 con 0x56256c55caf0 2024-03-07T06:51:37.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.103+0000 7f5819c40740 1 -- 172.21.15.133:0/1749018079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c55caf0 msgr2=0x56256c58c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:37.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.103+0000 7f5819c40740 1 --2- 172.21.15.133:0/1749018079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c55caf0 0x56256c58c8e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5808000fb0 tx=0x7f5808057eb0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.103+0000 7f5819c40740 1 -- 172.21.15.133:0/1749018079 shutdown_connections 2024-03-07T06:51:37.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.103+0000 7f5819c40740 1 --2- 172.21.15.133:0/1749018079 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c55caf0 0x56256c58c8e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.103+0000 7f5819c40740 1 --2- 172.21.15.133:0/1749018079 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c5899f0 0x56256c58c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.103+0000 7f5819c40740 1 --2- 172.21.15.133:0/1749018079 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c589610 0x56256c55c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.103+0000 7f5819c40740 1 -- 172.21.15.133:0/1749018079 >> 172.21.15.133:0/1749018079 conn(0x56256c55e820 msgr2=0x56256c5a73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.103+0000 7f5819c40740 1 -- 172.21.15.133:0/1749018079 shutdown_connections 2024-03-07T06:51:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.103+0000 7f5819c40740 1 -- 172.21.15.133:0/1749018079 wait complete. 2024-03-07T06:51:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.104+0000 7f5819c40740 1 -- start start 2024-03-07T06:51:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.104+0000 7f5819c40740 1 Processor -- start 2024-03-07T06:51:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.105+0000 7f5819c40740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c589610 0x56256c58fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.105+0000 7f5819c40740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c5843b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.105+0000 7f5819c40740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c5848f0 0x56256c580fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.105+0000 7f5819c40740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256c560dc0 con 0x56256c5899f0 2024-03-07T06:51:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.105+0000 7f5819c40740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256c482df0 con 0x56256c5848f0 2024-03-07T06:51:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.105+0000 7f5819c40740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x56256c5601b0 con 0x56256c589610 2024-03-07T06:51:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.105+0000 7f5819c40740 10 client.0 _ll_register_callbacks cb 0x56256c4bd8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T06:51:37.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.105+0000 7f5807400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c5843b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.106+0000 7f5807400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c5843b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56054/0 (socket says 172.21.15.133:56054) 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.106+0000 7f5807400640 1 -- 172.21.15.133:0/1805855760 learned_addr learned my addr 172.21.15.133:0/1805855760 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.106+0000 7f5814e00640 1 --2- 172.21.15.133:0/1805855760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c5848f0 0x56256c580fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.106+0000 7f580fe00640 1 --2- 172.21.15.133:0/1805855760 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c589610 0x56256c58fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.106+0000 7f5807400640 1 -- 172.21.15.133:0/1805855760 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c589610 msgr2=0x56256c58fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.106+0000 7f5807400640 1 --2- 172.21.15.133:0/1805855760 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c589610 0x56256c58fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.106+0000 7f5807400640 1 -- 172.21.15.133:0/1805855760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c5848f0 msgr2=0x56256c580fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.106+0000 7f5807400640 1 --2- 172.21.15.133:0/1805855760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c5848f0 0x56256c580fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.106+0000 7f5807400640 1 -- 172.21.15.133:0/1805855760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56256c1db2d0 con 0x56256c5899f0 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.106+0000 7f580fe00640 1 --2- 172.21.15.133:0/1805855760 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c589610 0x56256c58fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.106+0000 7f5807400640 1 --2- 172.21.15.133:0/1805855760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c5843b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009eb0 tx=0x7f57f8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971911cd815db46f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.106+0000 7f57fd600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.106+0000 7f57fd600640 1 -- 172.21.15.133:0/1805855760 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8045070 con 0x56256c5899f0 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.106+0000 7f57fd600640 1 -- 172.21.15.133:0/1805855760 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57f803f650 con 0x56256c5899f0 2024-03-07T06:51:37.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.107+0000 7f5819c40740 1 -- 172.21.15.133:0/1805855760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58080012b0 con 0x56256c5899f0 2024-03-07T06:51:37.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.107+0000 7f57fd600640 1 -- 172.21.15.133:0/1805855760 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57f8050050 con 0x56256c5899f0 2024-03-07T06:51:37.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.107+0000 7f5819c40740 10 client.4689 fetch_fsmap learned FSMap version 8 2024-03-07T06:51:37.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.107+0000 7f5819c40740 1 -- 172.21.15.133:0/1805855760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56256c2a56b0 con 0x56256c5899f0 2024-03-07T06:51:37.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.107+0000 7f57fd600640 1 -- 172.21.15.133:0/1805855760 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58080012b0 con 0x56256c5899f0 2024-03-07T06:51:37.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.107+0000 7f57fd600640 1 -- 172.21.15.133:0/1805855760 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 8) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f57f8053020 con 0x56256c5899f0 2024-03-07T06:51:37.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.107+0000 7f5819c40740 10 client.4689 fetch_fsmap finished waiting for FSMap version 8 2024-03-07T06:51:37.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.107+0000 7f5819c40740 10 client.4689 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-07T06:51:37.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.107+0000 7f5819c40740 20 client.4689 populate_metadata read hostname 'smithi133' 2024-03-07T06:51:37.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.107+0000 7f5819c40740 1 --2- 172.21.15.133:0/1805855760 >> [v2:172.21.15.161:6836/2885273912,v1:172.21.15.161:6837/2885273912] conn(0x56256c58e9d0 0x56256c58edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.107+0000 7f5819c40740 4 client.4689 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T06:51:37.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.107+0000 7f5819c40740 1 -- 172.21.15.133:0/1805855760 --> [v2:172.21.15.161:6836/2885273912,v1:172.21.15.161:6837/2885273912] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56256c4d3c00 con 0x56256c58e9d0 2024-03-07T06:51:37.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.108+0000 7f580fe00640 1 --2- 172.21.15.133:0/1805855760 >> [v2:172.21.15.161:6836/2885273912,v1:172.21.15.161:6837/2885273912] conn(0x56256c58e9d0 0x56256c58edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.109+0000 7f580fe00640 1 --2- 172.21.15.133:0/1805855760 >> [v2:172.21.15.161:6836/2885273912,v1:172.21.15.161:6837/2885273912] conn(0x56256c58e9d0 0x56256c58edb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76bf02bd119f43ae server_cookie=22e669377e7277a4 in_seq=0 out_seq=0 2024-03-07T06:51:37.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.109+0000 7f57fd600640 10 client.4689 ms_handle_connect on v2:172.21.15.161:6836/2885273912 2024-03-07T06:51:37.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.111+0000 7f57fd600640 1 -- 172.21.15.133:0/1805855760 <== mds.0 v2:172.21.15.161:6836/2885273912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56256c4d3c00 con 0x56256c58e9d0 2024-03-07T06:51:37.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.111+0000 7f57fd600640 10 client.4689 handle_command_reply: tid=0 2024-03-07T06:51:37.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.111+0000 7f5819c40740 1 client.4689 shutdown 2024-03-07T06:51:37.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.111+0000 7f5819c40740 1 -- 172.21.15.133:0/1805855760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 msgr2=0x56256c5843b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:37.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.111+0000 7f5819c40740 1 --2- 172.21.15.133:0/1805855760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c5843b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009eb0 tx=0x7f57f8004ab0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.111+0000 7f5819c40740 1 -- 172.21.15.133:0/1805855760 shutdown_connections 2024-03-07T06:51:37.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.111+0000 7f5819c40740 1 --2- 172.21.15.133:0/1805855760 >> [v2:172.21.15.161:6836/2885273912,v1:172.21.15.161:6837/2885273912] conn(0x56256c58e9d0 0x56256c58edb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.111+0000 7f5819c40740 1 --2- 172.21.15.133:0/1805855760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c5848f0 0x56256c580fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.111+0000 7f5819c40740 1 --2- 172.21.15.133:0/1805855760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c5843b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.111+0000 7f5819c40740 1 --2- 172.21.15.133:0/1805855760 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c589610 0x56256c58fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.111+0000 7f57fd600640 0 client.4689 ms_handle_reset on v2:172.21.15.161:6836/2885273912 2024-03-07T06:51:37.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.111+0000 7f5819c40740 1 -- 172.21.15.133:0/1805855760 >> 172.21.15.133:0/1805855760 conn(0x56256c55caf0 msgr2=0x56256c55bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:37.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.111+0000 7f5819c40740 1 -- 172.21.15.133:0/1805855760 shutdown_connections 2024-03-07T06:51:37.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.111+0000 7f5819c40740 1 -- 172.21.15.133:0/1805855760 wait complete. 2024-03-07T06:51:37.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.112+0000 7f5819c40740 20 client.4689 trim_cache size 0 max 16384 2024-03-07T06:51:37.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.129+0000 7f5819c40740 1 Processor -- start 2024-03-07T06:51:37.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.130+0000 7f5819c40740 1 -- start start 2024-03-07T06:51:37.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.130+0000 7f5819c40740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c5848f0 0x56256c5aa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.130+0000 7f5819c40740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c589610 0x56256c5aab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.130+0000 7f5819c40740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c5b9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.130+0000 7f5819c40740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5810200480 con 0x56256c5899f0 2024-03-07T06:51:37.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.130+0000 7f5819c40740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58100b4fd0 con 0x56256c589610 2024-03-07T06:51:37.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.130+0000 7f5819c40740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58100b43d0 con 0x56256c5848f0 2024-03-07T06:51:37.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.131+0000 7f5807400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c589610 0x56256c5aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.131+0000 7f5807400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c589610 0x56256c5aab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:53142/0 (socket says 172.21.15.133:53142) 2024-03-07T06:51:37.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.131+0000 7f5807400640 1 -- 172.21.15.133:0/3852761945 learned_addr learned my addr 172.21.15.133:0/3852761945 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:37.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.131+0000 7f580fe00640 1 --2- 172.21.15.133:0/3852761945 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c5848f0 0x56256c5aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.131+0000 7f5814e00640 1 --2- 172.21.15.133:0/3852761945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c5b9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.131+0000 7f5814e00640 1 -- 172.21.15.133:0/3852761945 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c5848f0 msgr2=0x56256c5aa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:37.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.131+0000 7f5814e00640 1 --2- 172.21.15.133:0/3852761945 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c5848f0 0x56256c5aa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.131+0000 7f5814e00640 1 -- 172.21.15.133:0/3852761945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c589610 msgr2=0x56256c5aab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:37.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.131+0000 7f5814e00640 1 --2- 172.21.15.133:0/3852761945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c589610 0x56256c5aab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.131+0000 7f5814e00640 1 -- 172.21.15.133:0/3852761945 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56256c581d70 con 0x56256c5899f0 2024-03-07T06:51:37.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.132+0000 7f5814e00640 1 --2- 172.21.15.133:0/3852761945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c5b9360 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f580800c750 tx=0x7f580805e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf840235894d6950 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:37.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.132+0000 7f57ffe00640 1 -- 172.21.15.133:0/3852761945 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58080012b0 con 0x56256c5899f0 2024-03-07T06:51:37.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.132+0000 7f57ffe00640 1 -- 172.21.15.133:0/3852761945 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58080560f0 con 0x56256c5899f0 2024-03-07T06:51:37.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.132+0000 7f57ffe00640 1 -- 172.21.15.133:0/3852761945 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580805e070 con 0x56256c5899f0 2024-03-07T06:51:37.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.132+0000 7f5819c40740 1 -- 172.21.15.133:0/3852761945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 msgr2=0x56256c5b9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:37.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.132+0000 7f5819c40740 1 --2- 172.21.15.133:0/3852761945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c5b9360 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f580800c750 tx=0x7f580805e000 comp rx=0 tx=0).stop 2024-03-07T06:51:37.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.132+0000 7f5819c40740 1 -- 172.21.15.133:0/3852761945 shutdown_connections 2024-03-07T06:51:37.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.132+0000 7f5819c40740 1 --2- 172.21.15.133:0/3852761945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c5b9360 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.132+0000 7f5819c40740 1 --2- 172.21.15.133:0/3852761945 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c589610 0x56256c5aab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.132+0000 7f5819c40740 1 --2- 172.21.15.133:0/3852761945 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c5848f0 0x56256c5aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.132+0000 7f5819c40740 1 -- 172.21.15.133:0/3852761945 >> 172.21.15.133:0/3852761945 conn(0x56256c55caf0 msgr2=0x56256c588770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:37.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.132+0000 7f5819c40740 1 -- 172.21.15.133:0/3852761945 shutdown_connections 2024-03-07T06:51:37.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.133+0000 7f5819c40740 1 -- 172.21.15.133:0/3852761945 wait complete. 2024-03-07T06:51:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.133+0000 7f5819c40740 1 -- start start 2024-03-07T06:51:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.133+0000 7f5819c40740 1 Processor -- start 2024-03-07T06:51:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.134+0000 7f5819c40740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c5848f0 0x56256c6160e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.134+0000 7f5819c40740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c589610 0x56256c57fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.134+0000 7f5819c40740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c587640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.134+0000 7f5819c40740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256c585410 con 0x56256c5899f0 2024-03-07T06:51:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.134+0000 7f5819c40740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58100b49d0 con 0x56256c589610 2024-03-07T06:51:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.134+0000 7f5819c40740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x56256c58e040 con 0x56256c5848f0 2024-03-07T06:51:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.134+0000 7f5819c40740 10 client.0 _ll_register_callbacks cb 0x7f57f8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f5814e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c587640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f5814e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c587640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56082/0 (socket says 172.21.15.133:56082) 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f5814e00640 1 -- 172.21.15.133:0/3801189078 learned_addr learned my addr 172.21.15.133:0/3801189078 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f5807400640 1 --2- 172.21.15.133:0/3801189078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c589610 0x56256c57fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f580fe00640 1 --2- 172.21.15.133:0/3801189078 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c5848f0 0x56256c6160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f5814e00640 1 -- 172.21.15.133:0/3801189078 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c5848f0 msgr2=0x56256c6160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f5814e00640 1 --2- 172.21.15.133:0/3801189078 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c5848f0 0x56256c6160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f5814e00640 1 -- 172.21.15.133:0/3801189078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c589610 msgr2=0x56256c57fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f5814e00640 1 --2- 172.21.15.133:0/3801189078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c589610 0x56256c57fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f5814e00640 1 -- 172.21.15.133:0/3801189078 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56256c557cc0 con 0x56256c5899f0 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f5814e00640 1 --2- 172.21.15.133:0/3801189078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c587640 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f580800c750 tx=0x7f5808007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d68c0ce1b1de29 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f57fd600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f57fd600640 1 -- 172.21.15.133:0/3801189078 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58080012b0 con 0x56256c5899f0 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f57fd600640 1 -- 172.21.15.133:0/3801189078 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5808076960 con 0x56256c5899f0 2024-03-07T06:51:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.135+0000 7f5819c40740 1 -- 172.21.15.133:0/3801189078 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58080560f0 con 0x56256c5899f0 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.136+0000 7f57fd600640 1 -- 172.21.15.133:0/3801189078 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f580807b050 con 0x56256c5899f0 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.136+0000 7f5819c40740 10 client.4695 fetch_fsmap learned FSMap version 8 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.136+0000 7f5819c40740 1 -- 172.21.15.133:0/3801189078 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x56256c5c9d40 con 0x56256c5899f0 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.136+0000 7f57fd600640 1 -- 172.21.15.133:0/3801189078 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58080560f0 con 0x56256c5899f0 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.136+0000 7f57fd600640 1 -- 172.21.15.133:0/3801189078 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 8) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f580807e020 con 0x56256c5899f0 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.136+0000 7f5819c40740 10 client.4695 fetch_fsmap finished waiting for FSMap version 8 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.136+0000 7f5819c40740 10 client.4695 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.136+0000 7f5819c40740 20 client.4695 populate_metadata read hostname 'smithi133' 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.136+0000 7f5819c40740 1 --2- 172.21.15.133:0/3801189078 >> [v2:172.21.15.161:6836/2885273912,v1:172.21.15.161:6837/2885273912] conn(0x56256c5aa930 0x56256c5a6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.136+0000 7f5819c40740 4 client.4695 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.136+0000 7f5819c40740 1 -- 172.21.15.133:0/3801189078 --> [v2:172.21.15.161:6836/2885273912,v1:172.21.15.161:6837/2885273912] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56256c4cdf50 con 0x56256c5aa930 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.136+0000 7f580fe00640 1 --2- 172.21.15.133:0/3801189078 >> [v2:172.21.15.161:6836/2885273912,v1:172.21.15.161:6837/2885273912] conn(0x56256c5aa930 0x56256c5a6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.137+0000 7f580fe00640 1 --2- 172.21.15.133:0/3801189078 >> [v2:172.21.15.161:6836/2885273912,v1:172.21.15.161:6837/2885273912] conn(0x56256c5aa930 0x56256c5a6d80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9aa8eda526a9ad5 server_cookie=65c38a56fa5232c7 in_seq=0 out_seq=0 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.137+0000 7f57fd600640 10 client.4695 ms_handle_connect on v2:172.21.15.161:6836/2885273912 2024-03-07T06:51:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f57fd600640 1 -- 172.21.15.133:0/3801189078 <== mds.0 v2:172.21.15.161:6836/2885273912 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x56256c4cdf50 con 0x56256c5aa930 2024-03-07T06:51:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f57fd600640 10 client.4695 handle_command_reply: tid=0 2024-03-07T06:51:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f5819c40740 1 client.4695 shutdown 2024-03-07T06:51:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f5819c40740 1 -- 172.21.15.133:0/3801189078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 msgr2=0x56256c587640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f5819c40740 1 --2- 172.21.15.133:0/3801189078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c587640 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f580800c750 tx=0x7f5808007f90 comp rx=0 tx=0).stop 2024-03-07T06:51:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f5819c40740 1 -- 172.21.15.133:0/3801189078 shutdown_connections 2024-03-07T06:51:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f5819c40740 1 --2- 172.21.15.133:0/3801189078 >> [v2:172.21.15.161:6836/2885273912,v1:172.21.15.161:6837/2885273912] conn(0x56256c5aa930 0x56256c5a6d80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f5819c40740 1 --2- 172.21.15.133:0/3801189078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56256c5899f0 0x56256c587640 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f5819c40740 1 --2- 172.21.15.133:0/3801189078 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x56256c589610 0x56256c57fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f57fd600640 0 client.4695 ms_handle_reset on v2:172.21.15.161:6836/2885273912 2024-03-07T06:51:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f5819c40740 1 --2- 172.21.15.133:0/3801189078 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x56256c5848f0 0x56256c6160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f5819c40740 1 -- 172.21.15.133:0/3801189078 >> 172.21.15.133:0/3801189078 conn(0x56256c55caf0 msgr2=0x56256c5635d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f5819c40740 1 -- 172.21.15.133:0/3801189078 shutdown_connections 2024-03-07T06:51:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f5819c40740 1 -- 172.21.15.133:0/3801189078 wait complete. 2024-03-07T06:51:37.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.138+0000 7f5819c40740 20 client.4695 trim_cache size 0 max 16384 2024-03-07T06:51:37.142 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:51:37.142 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "1655279d-73c2-44d4-ba1b-3e8a15ae005a", 2024-03-07T06:51:37.142 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-03-07T06:51:37.142 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4288, 2024-03-07T06:51:37.142 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-03-07T06:51:37.142 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-03-07T06:51:37.142 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-03-07T06:51:37.142 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 21.951703837, 2024-03-07T06:51:37.142 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 8, 2024-03-07T06:51:37.142 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 21, 2024-03-07T06:51:37.142 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 21, 2024-03-07T06:51:37.142 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 27.216569378999999 2024-03-07T06:51:37.143 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:51:37.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.139+0000 7f5816e00640 1 -- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57ec06b4d0 msgr2=0x7f57ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:37.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.139+0000 7f5816e00640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57ec06b4d0 0x7f57ec06d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5800006fd0 tx=0x7f5800008040 comp rx=0 tx=0).stop 2024-03-07T06:51:37.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.139+0000 7f5816e00640 1 -- 172.21.15.133:0/1824539239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58100b5af0 msgr2=0x7f58100b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:37.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.139+0000 7f5816e00640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58100b5af0 0x7f58100b5f80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f580800c780 tx=0x7f580800cc40 comp rx=0 tx=0).stop 2024-03-07T06:51:37.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.139+0000 7f5816e00640 1 -- 172.21.15.133:0/1824539239 shutdown_connections 2024-03-07T06:51:37.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.139+0000 7f5816e00640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58100b5af0 0x7f58100b5f80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.139+0000 7f5816e00640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f581015ff90 0x7f5810155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.139+0000 7f5816e00640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57ec06b4d0 0x7f57ec06d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.139+0000 7f5816e00640 1 --2- 172.21.15.133:0/1824539239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f581015fbb0 0x7f58100c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:37.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.139+0000 7f5816e00640 1 -- 172.21.15.133:0/1824539239 >> 172.21.15.133:0/1824539239 conn(0x7f58100b3120 msgr2=0x7f581015c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:37.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.139+0000 7f5816e00640 1 -- 172.21.15.133:0/1824539239 shutdown_connections 2024-03-07T06:51:37.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:37.139+0000 7f5816e00640 1 -- 172.21.15.133:0/1824539239 wait complete. 2024-03-07T06:51:37.157 INFO:teuthology.run_tasks:Running task kclient... 2024-03-07T06:51:37.191 INFO:tasks.kclient:Mounting kernel clients... 2024-03-07T06:51:37.192 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-07T06:51:37.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:37.192 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-03-07T06:51:37.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:37.248 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-03-07T06:51:37.343 INFO:teuthology.orchestra.run.smithi133.stderr:Cannot find device "ceph-brx" 2024-03-07T06:51:37.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:51:37.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:37.347 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-03-07T06:51:37.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:37.400 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-03-07T06:51:37.496 INFO:teuthology.orchestra.run.smithi133.stderr:Cannot find device "ceph-brx" 2024-03-07T06:51:37.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:51:37.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:37.502 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-03-07T06:51:37.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:37.558 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2024-03-07T06:51:37.654 INFO:teuthology.orchestra.run.smithi161.stderr:Cannot find device "ceph-brx" 2024-03-07T06:51:37.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:51:37.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:37.658 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-03-07T06:51:37.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:37.712 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2024-03-07T06:51:37.804 INFO:teuthology.orchestra.run.smithi161.stderr:Cannot find device "ceph-brx" 2024-03-07T06:51:37.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:51:37.808 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-03-07T06:51:37.808 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-07T06:51:37.808 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-07T06:51:37.808 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-07T06:51:38.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.025+0000 7f0f76a00640 1 Processor -- start 2024-03-07T06:51:38.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.025+0000 7f0f76a00640 1 -- start start 2024-03-07T06:51:38.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.026+0000 7f0f76a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f70072f40 0x7f0f70166c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:38.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.026+0000 7f0f76a00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f70160760 con 0x7f0f7015bd30 2024-03-07T06:51:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.027+0000 7f0f76a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7015f9a0 con 0x7f0f700cf5d0 2024-03-07T06:51:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.027+0000 7f0f76a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7015fc80 con 0x7f0f70072f40 2024-03-07T06:51:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.027+0000 7f0f75600640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0f7015bd30 0x7f0f7006e8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46978/0 (socket says 172.21.15.133:46978) 2024-03-07T06:51:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.027+0000 7f0f75600640 1 -- 172.21.15.133:0/1846977026 learned_addr learned my addr 172.21.15.133:0/1846977026 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.027+0000 7f0f76000640 1 --2- 172.21.15.133:0/1846977026 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f70072f40 0x7f0f70166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.027+0000 7f0f6fe00640 1 -- 172.21.15.133:0/1846977026 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613006378 0 0) 0x7f0f70160760 con 0x7f0f7015bd30 2024-03-07T06:51:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.027+0000 7f0f6fe00640 1 -- 172.21.15.133:0/1846977026 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f58003520 con 0x7f0f7015bd30 2024-03-07T06:51:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f6fe00640 1 -- 172.21.15.133:0/1846977026 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 775587548 0 0) 0x7f0f58003520 con 0x7f0f7015bd30 2024-03-07T06:51:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f6fe00640 1 -- 172.21.15.133:0/1846977026 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f70072f40 msgr2=0x7f0f70166c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f6fe00640 1 --2- 172.21.15.133:0/1846977026 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f70072f40 0x7f0f70166c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f6fe00640 1 -- 172.21.15.133:0/1846977026 >> v1:172.21.15.161:6789/0 conn(0x7f0f700cf5d0 legacy=0x7f0f70072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f6fe00640 1 -- 172.21.15.133:0/1846977026 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7015ff60 con 0x7f0f7015bd30 2024-03-07T06:51:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f6fe00640 1 -- 172.21.15.133:0/1846977026 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f0f64001670 con 0x7f0f7015bd30 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f6fe00640 1 -- 172.21.15.133:0/1846977026 >> v1:172.21.15.133:6789/0 conn(0x7f0f7015bd30 legacy=0x7f0f7006e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f6fe00640 1 --2- 172.21.15.133:0/1846977026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f580039e0 0x7f0f58003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f6fe00640 1 --2- 172.21.15.133:0/1846977026 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f70072f40 0x7f0f58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f6fe00640 1 --2- 172.21.15.133:0/1846977026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58004d40 0x7f0f580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f6fe00640 1 -- 172.21.15.133:0/1846977026 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7015fc80 con 0x7f0f58004d40 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f6fe00640 1 -- 172.21.15.133:0/1846977026 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5800baa0 con 0x7f0f580039e0 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f6fe00640 1 -- 172.21.15.133:0/1846977026 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5800bca0 con 0x7f0f70072f40 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f76000640 1 --2- 172.21.15.133:0/1846977026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58004d40 0x7f0f580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f75600640 1 --2- 172.21.15.133:0/1846977026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f580039e0 0x7f0f58003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f74c00640 1 --2- 172.21.15.133:0/1846977026 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f70072f40 0x7f0f58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f76000640 1 -- 172.21.15.133:0/1846977026 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f70072f40 msgr2=0x7f0f58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f76000640 1 --2- 172.21.15.133:0/1846977026 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f70072f40 0x7f0f58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f76000640 1 -- 172.21.15.133:0/1846977026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f580039e0 msgr2=0x7f0f58003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f76000640 1 --2- 172.21.15.133:0/1846977026 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f580039e0 0x7f0f58003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f76000640 1 -- 172.21.15.133:0/1846977026 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f58003520 con 0x7f0f58004d40 2024-03-07T06:51:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.028+0000 7f0f74c00640 1 --2- 172.21.15.133:0/1846977026 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f70072f40 0x7f0f58004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:38.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.029+0000 7f0f76000640 1 --2- 172.21.15.133:0/1846977026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58004d40 0x7f0f580051c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0f70068570 tx=0x7f0f58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520f7e31260ee7ef server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:38.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.030+0000 7f0f6fe00640 1 -- 172.21.15.133:0/1846977026 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68039070 con 0x7f0f58004d40 2024-03-07T06:51:38.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.030+0000 7f0f6fe00640 1 -- 172.21.15.133:0/1846977026 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6802cb90 con 0x7f0f58004d40 2024-03-07T06:51:38.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.030+0000 7f0f6fe00640 1 -- 172.21.15.133:0/1846977026 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68009c10 con 0x7f0f58004d40 2024-03-07T06:51:38.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.030+0000 7f0f76a00640 1 -- 172.21.15.133:0/1846977026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58004d40 msgr2=0x7f0f580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:38.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.030+0000 7f0f76a00640 1 --2- 172.21.15.133:0/1846977026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f58004d40 0x7f0f580051c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0f70068570 tx=0x7f0f58005830 comp rx=0 tx=0).stop 2024-03-07T06:51:38.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.030+0000 7f0f76000640 1 -- 172.21.15.133:0/1846977026 reap_dead start 2024-03-07T06:51:38.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.030+0000 7f0f76a00640 1 -- 172.21.15.133:0/1846977026 shutdown_connections 2024-03-07T06:51:38.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.030+0000 7f0f76a00640 1 -- 172.21.15.133:0/1846977026 >> 172.21.15.133:0/1846977026 conn(0x7f0f700b2e90 msgr2=0x7f0f700b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:38.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.030+0000 7f0f76a00640 1 -- 172.21.15.133:0/1846977026 shutdown_connections 2024-03-07T06:51:38.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.030+0000 7f0f76a00640 1 -- 172.21.15.133:0/1846977026 wait complete. 2024-03-07T06:51:38.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.032+0000 7f0f76a00640 1 Processor -- start 2024-03-07T06:51:38.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.032+0000 7f0f76a00640 1 -- start start 2024-03-07T06:51:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.033+0000 7f0f76a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f58004d40 0x7f0f701f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.033+0000 7f0f76a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f7014fed0 0x7f0f701e7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.033+0000 7f0f76a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f701e78c0 0x7f0f7015c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.033+0000 7f0f76a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7015d120 con 0x7f0f7014fed0 2024-03-07T06:51:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.033+0000 7f0f76a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7006f130 con 0x7f0f701e78c0 2024-03-07T06:51:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.033+0000 7f0f76a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7015d2a0 con 0x7f0f58004d40 2024-03-07T06:51:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.033+0000 7f0f74c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f7014fed0 0x7f0f701e7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.034+0000 7f0f74c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f7014fed0 0x7f0f701e7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:56094/0 (socket says 172.21.15.133:56094) 2024-03-07T06:51:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.034+0000 7f0f74c00640 1 -- 172.21.15.133:0/2660903510 learned_addr learned my addr 172.21.15.133:0/2660903510 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.034+0000 7f0f75600640 1 --2- 172.21.15.133:0/2660903510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f58004d40 0x7f0f701f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.034+0000 7f0f76000640 1 --2- 172.21.15.133:0/2660903510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f701e78c0 0x7f0f7015c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.034+0000 7f0f74c00640 1 -- 172.21.15.133:0/2660903510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f58004d40 msgr2=0x7f0f701f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.034+0000 7f0f74c00640 1 --2- 172.21.15.133:0/2660903510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f58004d40 0x7f0f701f5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.034+0000 7f0f74c00640 1 -- 172.21.15.133:0/2660903510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f701e78c0 msgr2=0x7f0f7015c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.034+0000 7f0f74c00640 1 --2- 172.21.15.133:0/2660903510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f701e78c0 0x7f0f7015c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.034+0000 7f0f74c00640 1 -- 172.21.15.133:0/2660903510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f701e7db0 con 0x7f0f7014fed0 2024-03-07T06:51:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.034+0000 7f0f74c00640 1 --2- 172.21.15.133:0/2660903510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f7014fed0 0x7f0f701e7380 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c00d7d0 tx=0x7f0f5c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.034+0000 7f0f6d600640 1 -- 172.21.15.133:0/2660903510 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c005070 con 0x7f0f7014fed0 2024-03-07T06:51:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.034+0000 7f0f6d600640 1 -- 172.21.15.133:0/2660903510 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5c005230 con 0x7f0f7014fed0 2024-03-07T06:51:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.034+0000 7f0f76a00640 1 -- 172.21.15.133:0/2660903510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f701f5420 con 0x7f0f7014fed0 2024-03-07T06:51:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.035+0000 7f0f6d600640 1 -- 172.21.15.133:0/2660903510 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c007030 con 0x7f0f7014fed0 2024-03-07T06:51:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.035+0000 7f0f76a00640 1 -- 172.21.15.133:0/2660903510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f701e8090 con 0x7f0f7014fed0 2024-03-07T06:51:38.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.036+0000 7f0f76a00640 1 -- 172.21.15.133:0/2660903510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f70062c90 con 0x7f0f7014fed0 2024-03-07T06:51:38.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.036+0000 7f0f6d600640 1 -- 172.21.15.133:0/2660903510 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0f5c01a020 con 0x7f0f7014fed0 2024-03-07T06:51:38.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.037+0000 7f0f6d600640 1 --2- 172.21.15.133:0/2660903510 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0f5006b4d0 0x7f0f5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:38.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.037+0000 7f0f6d600640 1 -- 172.21.15.133:0/2660903510 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f5c089ac0 con 0x7f0f7014fed0 2024-03-07T06:51:38.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.038+0000 7f0f75600640 1 --2- 172.21.15.133:0/2660903510 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0f5006b4d0 0x7f0f5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:38.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.039+0000 7f0f75600640 1 --2- 172.21.15.133:0/2660903510 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0f5006b4d0 0x7f0f5006d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0f64001cd0 tx=0x7f0f64009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:38.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.042+0000 7f0f6d600640 1 -- 172.21.15.133:0/2660903510 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0f5c0588a0 con 0x7f0f7014fed0 2024-03-07T06:51:38.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.278+0000 7f0f76a00640 1 -- 172.21.15.133:0/2660903510 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0f700734a0 con 0x7f0f7014fed0 2024-03-07T06:51:38.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.279+0000 7f0f6d600640 1 -- 172.21.15.133:0/2660903510 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0f5c0581a0 con 0x7f0f7014fed0 2024-03-07T06:51:38.279 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-07T06:51:38.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.282+0000 7f0f76a00640 1 -- 172.21.15.133:0/2660903510 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0f5006b4d0 msgr2=0x7f0f5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:38.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.282+0000 7f0f76a00640 1 --2- 172.21.15.133:0/2660903510 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0f5006b4d0 0x7f0f5006d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0f64001cd0 tx=0x7f0f64009040 comp rx=0 tx=0).stop 2024-03-07T06:51:38.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.282+0000 7f0f76a00640 1 -- 172.21.15.133:0/2660903510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f7014fed0 msgr2=0x7f0f701e7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:38.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.282+0000 7f0f76a00640 1 --2- 172.21.15.133:0/2660903510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f7014fed0 0x7f0f701e7380 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c00d7d0 tx=0x7f0f5c00dc90 comp rx=0 tx=0).stop 2024-03-07T06:51:38.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.282+0000 7f0f76a00640 1 -- 172.21.15.133:0/2660903510 shutdown_connections 2024-03-07T06:51:38.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.282+0000 7f0f76a00640 1 --2- 172.21.15.133:0/2660903510 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0f5006b4d0 0x7f0f5006d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:38.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.282+0000 7f0f76a00640 1 --2- 172.21.15.133:0/2660903510 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0f701e78c0 0x7f0f7015c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:38.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.282+0000 7f0f76a00640 1 --2- 172.21.15.133:0/2660903510 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f7014fed0 0x7f0f701e7380 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:38.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.282+0000 7f0f76a00640 1 --2- 172.21.15.133:0/2660903510 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0f58004d40 0x7f0f701f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:38.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.282+0000 7f0f76a00640 1 -- 172.21.15.133:0/2660903510 >> 172.21.15.133:0/2660903510 conn(0x7f0f700b2e90 msgr2=0x7f0f70158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:38.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.282+0000 7f0f76a00640 1 -- 172.21.15.133:0/2660903510 shutdown_connections 2024-03-07T06:51:38.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:38.282+0000 7f0f76a00640 1 -- 172.21.15.133:0/2660903510 wait complete. 2024-03-07T06:51:38.301 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-07T06:51:38.301 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-07T06:51:38.301 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-03-07T06:51:38.301 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-07T06:51:38.301 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-07T06:51:38.301 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-07T06:51:38.301 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-07T06:51:38.301 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-07T06:51:38.301 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-07T06:51:38.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:38.302 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-03-07T06:51:38.351 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T06:51:38.351 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T06:51:38.351 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T06:51:38.351 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:38.351 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-03-07T06:51:38.351 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:38.351 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:51:38.351 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:38.351 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-03-07T06:51:38.352 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:51:38.352 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:38.352 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-03-07T06:51:38.352 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:51:38.352 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:38.352 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-07T06:51:38.352 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:38.352 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-07T06:51:38.352 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42522sec preferred_lft 42522sec 2024-03-07T06:51:38.352 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-03-07T06:51:38.352 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:38.353 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-07T06:51:38.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:38.354 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-07T06:51:38.354 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-07T06:51:38.354 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add name ceph-brx type bridge 2024-03-07T06:51:38.354 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr flush dev ceph-brx 2024-03-07T06:51:38.354 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set ceph-brx up 2024-03-07T06:51:38.354 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-07T06:51:38.354 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-07T06:51:38.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:38.832 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-07T06:51:38.910 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-03-07T06:51:38.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:38.914 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-03-07T06:51:38.977 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-03-07T06:51:38.977 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-07T06:51:38.977 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-07T06:51:38.977 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-07T06:51:38.978 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-07T06:51:38.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:38.978 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-07T06:51:38.978 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-07T06:51:38.978 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-07T06:51:38.978 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-07T06:51:38.978 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-07T06:51:38.978 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-07T06:51:39.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:39.652 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-03-07T06:51:39.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:39.709 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2024-03-07T06:51:39.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:39.764 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-07T06:51:39.764 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-07T06:51:39.764 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-07T06:51:39.764 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-07T06:51:39.765 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-07T06:51:39.913 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-07T06:51:39.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:39.913 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-07T06:51:39.913 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-07T06:51:39.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-07T06:51:39.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-07T06:51:39.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-07T06:51:39.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-07T06:51:39.913 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-07T06:51:39.914 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-07T06:51:40.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:40.190 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-07T06:51:40.191 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-07T06:51:40.191 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-03-07T06:51:40.191 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.0 master ceph-brx 2024-03-07T06:51:40.191 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-07T06:51:40.354 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T06:51:40.354 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-07T06:51:40.399 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-07T06:51:40.399 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T06:51:40.399 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-07T06:51:40.494 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-07T06:51:40.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:40.494 DEBUG:teuthology.orchestra.run.smithi133:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:51:40.641 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:51:40.641 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T06:51:41.306 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:51:41.306 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T06:51:41.306 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: resolved to: "172.21.15.133:3300,172.21.15.161:3300,172.21.15.161:3301" 2024-03-07T06:51:41.306 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: trying mount with new device syntax: 0@1655279d-73c2-44d4-ba1b-3e8a15ae005a.cephfs=/ 2024-03-07T06:51:41.306 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.133:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2024-03-07T06:51:41.311 INFO:teuthology.orchestra.run.smithi133.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-07T06:51:41.311 INFO:teuthology.orchestra.run.smithi133.stdout: You just mounted a file system that supports labels which does not 2024-03-07T06:51:41.311 INFO:teuthology.orchestra.run.smithi133.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-07T06:51:41.311 INFO:teuthology.orchestra.run.smithi133.stdout: applications will generate AVC messages and not be allowed access to 2024-03-07T06:51:41.311 INFO:teuthology.orchestra.run.smithi133.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-07T06:51:41.316 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-07T06:51:41.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:41.316 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-07T06:51:41.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:41.405 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T06:51:41.485 INFO:teuthology.orchestra.run.smithi133.stdout:client4576 2024-03-07T06:51:41.486 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:51:41.486 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:51:41.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:41.541 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T06:51:41.617 INFO:teuthology.orchestra.run.smithi133.stdout:client4576 2024-03-07T06:51:41.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:41.617 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T06:51:41.693 INFO:teuthology.orchestra.run.smithi133.stdout:client4576 2024-03-07T06:51:41.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:41.694 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/1655279d-73c2-44d4-ba1b-3e8a15ae005a.client4576/status') 2024-03-07T06:51:41.798 INFO:teuthology.orchestra.run.smithi133.stdout:instance: client.4576 (3)172.21.15.133:0/3199615356 2024-03-07T06:51:41.798 INFO:teuthology.orchestra.run.smithi133.stdout:blocklisted: false 2024-03-07T06:51:41.798 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records in 2024-03-07T06:51:41.799 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records out 2024-03-07T06:51:41.799 INFO:teuthology.orchestra.run.smithi133.stderr:71 bytes copied, 8.7505e-05 s, 811 kB/s 2024-03-07T06:51:41.803 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-03-07T06:51:41.803 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-07T06:51:41.803 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-07T06:51:41.803 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-07T06:51:42.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.019+0000 7fbeeb000640 1 Processor -- start 2024-03-07T06:51:42.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.019+0000 7fbeeb000640 1 -- start start 2024-03-07T06:51:42.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.020+0000 7fbeeb000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbee40cf5d0 0x7fbee4072830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:42.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.020+0000 7fbeeb000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee4160760 con 0x7fbee415bd30 2024-03-07T06:51:42.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.020+0000 7fbeeb000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee415f9a0 con 0x7fbee4072d70 2024-03-07T06:51:42.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.020+0000 7fbeeb000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee40731e0 con 0x7fbee40cf5d0 2024-03-07T06:51:42.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.021+0000 7fbee9c00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fbee415bd30 0x7fbee406e8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42444/0 (socket says 172.21.15.133:42444) 2024-03-07T06:51:42.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.021+0000 7fbee9c00640 1 -- 172.21.15.133:0/2357619046 learned_addr learned my addr 172.21.15.133:0/2357619046 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:42.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.021+0000 7fbee9200640 1 --2- 172.21.15.133:0/2357619046 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbee40cf5d0 0x7fbee4072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:42.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.021+0000 7fbee3e00640 1 -- 172.21.15.133:0/2357619046 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857268578 0 0) 0x7fbee4160760 con 0x7fbee415bd30 2024-03-07T06:51:42.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.021+0000 7fbee3e00640 1 -- 172.21.15.133:0/2357619046 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbecc003520 con 0x7fbee415bd30 2024-03-07T06:51:42.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee3e00640 1 -- 172.21.15.133:0/2357619046 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1099704489 0 0) 0x7fbecc003520 con 0x7fbee415bd30 2024-03-07T06:51:42.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee3e00640 1 -- 172.21.15.133:0/2357619046 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbee40cf5d0 msgr2=0x7fbee4072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:42.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee3e00640 1 --2- 172.21.15.133:0/2357619046 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbee40cf5d0 0x7fbee4072830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:42.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee3e00640 1 -- 172.21.15.133:0/2357619046 >> v1:172.21.15.161:6789/0 conn(0x7fbee4072d70 legacy=0x7fbee415c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:42.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee3e00640 1 -- 172.21.15.133:0/2357619046 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee415fc80 con 0x7fbee415bd30 2024-03-07T06:51:42.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee3e00640 1 -- 172.21.15.133:0/2357619046 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fbed8001670 con 0x7fbee415bd30 2024-03-07T06:51:42.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee3e00640 1 -- 172.21.15.133:0/2357619046 >> v1:172.21.15.133:6789/0 conn(0x7fbee415bd30 legacy=0x7fbee406e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:42.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee3e00640 1 --2- 172.21.15.133:0/2357619046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbecc0039e0 0x7fbecc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:42.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee3e00640 1 --2- 172.21.15.133:0/2357619046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbecc004800 0x7fbecc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:42.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee3e00640 1 --2- 172.21.15.133:0/2357619046 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbee40cf5d0 0x7fbecc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:42.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee3e00640 1 -- 172.21.15.133:0/2357619046 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee40731e0 con 0x7fbecc0039e0 2024-03-07T06:51:42.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee3e00640 1 -- 172.21.15.133:0/2357619046 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc00bad0 con 0x7fbecc004800 2024-03-07T06:51:42.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee3e00640 1 -- 172.21.15.133:0/2357619046 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc00bcd0 con 0x7fbee40cf5d0 2024-03-07T06:51:42.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee9c00640 1 --2- 172.21.15.133:0/2357619046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbecc0039e0 0x7fbecc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:42.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee9200640 1 --2- 172.21.15.133:0/2357619046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbecc004800 0x7fbecc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:42.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee9c00640 1 -- 172.21.15.133:0/2357619046 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbee40cf5d0 msgr2=0x7fbecc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:42.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbeea600640 1 --2- 172.21.15.133:0/2357619046 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbee40cf5d0 0x7fbecc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:42.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee9c00640 1 --2- 172.21.15.133:0/2357619046 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbee40cf5d0 0x7fbecc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:42.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee9c00640 1 -- 172.21.15.133:0/2357619046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbecc004800 msgr2=0x7fbecc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:42.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee9c00640 1 --2- 172.21.15.133:0/2357619046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbecc004800 0x7fbecc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:42.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee9c00640 1 -- 172.21.15.133:0/2357619046 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc003520 con 0x7fbecc0039e0 2024-03-07T06:51:42.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.022+0000 7fbee9200640 1 --2- 172.21.15.133:0/2357619046 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbecc004800 0x7fbecc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:42.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.023+0000 7fbee9c00640 1 --2- 172.21.15.133:0/2357619046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbecc0039e0 0x7fbecc003e00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbed8001eb0 tx=0x7fbecc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d650e23f5744b46 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:42.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.024+0000 7fbee3e00640 1 -- 172.21.15.133:0/2357619046 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8007c80 con 0x7fbecc0039e0 2024-03-07T06:51:42.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.024+0000 7fbee3e00640 1 -- 172.21.15.133:0/2357619046 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed8030070 con 0x7fbecc0039e0 2024-03-07T06:51:42.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.024+0000 7fbee3e00640 1 -- 172.21.15.133:0/2357619046 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8005d10 con 0x7fbecc0039e0 2024-03-07T06:51:42.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.024+0000 7fbeeb000640 1 -- 172.21.15.133:0/2357619046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbecc0039e0 msgr2=0x7fbecc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:42.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.024+0000 7fbeeb000640 1 --2- 172.21.15.133:0/2357619046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbecc0039e0 0x7fbecc003e00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbed8001eb0 tx=0x7fbecc005830 comp rx=0 tx=0).stop 2024-03-07T06:51:42.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.024+0000 7fbeea600640 1 -- 172.21.15.133:0/2357619046 reap_dead start 2024-03-07T06:51:42.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.024+0000 7fbeeb000640 1 -- 172.21.15.133:0/2357619046 shutdown_connections 2024-03-07T06:51:42.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.024+0000 7fbeeb000640 1 -- 172.21.15.133:0/2357619046 >> 172.21.15.133:0/2357619046 conn(0x7fbee40b2e90 msgr2=0x7fbee40b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:42.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.024+0000 7fbeeb000640 1 -- 172.21.15.133:0/2357619046 shutdown_connections 2024-03-07T06:51:42.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.024+0000 7fbeeb000640 1 -- 172.21.15.133:0/2357619046 wait complete. 2024-03-07T06:51:42.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.026+0000 7fbeeb000640 1 Processor -- start 2024-03-07T06:51:42.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.026+0000 7fbeeb000640 1 -- start start 2024-03-07T06:51:42.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.027+0000 7fbeeb000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbecc0039e0 0x7fbee41f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:42.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.027+0000 7fbeeb000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee415c110 0x7fbee415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:42.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.027+0000 7fbeeb000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbee415e720 0x7fbee415eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:42.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.027+0000 7fbeeb000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee406f130 con 0x7fbee415e720 2024-03-07T06:51:42.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.027+0000 7fbeeb000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee4160240 con 0x7fbee415c110 2024-03-07T06:51:42.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.027+0000 7fbeeb000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee4072830 con 0x7fbecc0039e0 2024-03-07T06:51:42.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.028+0000 7fbeea600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbee415e720 0x7fbee415eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:42.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.028+0000 7fbeea600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbee415e720 0x7fbee415eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45944/0 (socket says 172.21.15.133:45944) 2024-03-07T06:51:42.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.028+0000 7fbeea600640 1 -- 172.21.15.133:0/597780530 learned_addr learned my addr 172.21.15.133:0/597780530 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:42.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.028+0000 7fbee9c00640 1 --2- 172.21.15.133:0/597780530 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbecc0039e0 0x7fbee41f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:42.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.028+0000 7fbee9200640 1 --2- 172.21.15.133:0/597780530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee415c110 0x7fbee415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:42.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.028+0000 7fbeea600640 1 -- 172.21.15.133:0/597780530 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbecc0039e0 msgr2=0x7fbee41f4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:42.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.028+0000 7fbeea600640 1 --2- 172.21.15.133:0/597780530 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbecc0039e0 0x7fbee41f4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:42.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.028+0000 7fbeea600640 1 -- 172.21.15.133:0/597780530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee415c110 msgr2=0x7fbee415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:42.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.028+0000 7fbeea600640 1 --2- 172.21.15.133:0/597780530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee415c110 0x7fbee415d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:42.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.028+0000 7fbeea600640 1 -- 172.21.15.133:0/597780530 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee4151b80 con 0x7fbee415e720 2024-03-07T06:51:42.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.028+0000 7fbeea600640 1 --2- 172.21.15.133:0/597780530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbee415e720 0x7fbee415eba0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbee4073c90 tx=0x7fbedc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:42.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.029+0000 7fbee1600640 1 -- 172.21.15.133:0/597780530 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc018070 con 0x7fbee415e720 2024-03-07T06:51:42.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.029+0000 7fbee1600640 1 -- 172.21.15.133:0/597780530 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc003b70 con 0x7fbee415e720 2024-03-07T06:51:42.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.029+0000 7fbee1600640 1 -- 172.21.15.133:0/597780530 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc005070 con 0x7fbee415e720 2024-03-07T06:51:42.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.029+0000 7fbeeb000640 1 -- 172.21.15.133:0/597780530 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee415c640 con 0x7fbee415e720 2024-03-07T06:51:42.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.029+0000 7fbeeb000640 1 -- 172.21.15.133:0/597780530 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee415cc00 con 0x7fbee415e720 2024-03-07T06:51:42.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.030+0000 7fbeeb000640 1 -- 172.21.15.133:0/597780530 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee4062c90 con 0x7fbee415e720 2024-03-07T06:51:42.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.031+0000 7fbee1600640 1 -- 172.21.15.133:0/597780530 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbedc01b070 con 0x7fbee415e720 2024-03-07T06:51:42.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.032+0000 7fbee1600640 1 --2- 172.21.15.133:0/597780530 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbec406b500 0x7fbec406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:42.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.032+0000 7fbee1600640 1 -- 172.21.15.133:0/597780530 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbedc089960 con 0x7fbee415e720 2024-03-07T06:51:42.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.033+0000 7fbee9c00640 1 --2- 172.21.15.133:0/597780530 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbec406b500 0x7fbec406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:42.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.033+0000 7fbee9c00640 1 --2- 172.21.15.133:0/597780530 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbec406b500 0x7fbec406d9c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbed8034000 tx=0x7fbed8009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:42.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.037+0000 7fbee1600640 1 -- 172.21.15.133:0/597780530 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbee4151b80 con 0x7fbee415e720 2024-03-07T06:51:42.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.273+0000 7fbeeb000640 1 -- 172.21.15.133:0/597780530 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbee40734a0 con 0x7fbee415e720 2024-03-07T06:51:42.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.274+0000 7fbee1600640 1 -- 172.21.15.133:0/597780530 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbedc058740 con 0x7fbee415e720 2024-03-07T06:51:42.274 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-07T06:51:42.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.277+0000 7fbeeb000640 1 -- 172.21.15.133:0/597780530 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbec406b500 msgr2=0x7fbec406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:42.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.277+0000 7fbeeb000640 1 --2- 172.21.15.133:0/597780530 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbec406b500 0x7fbec406d9c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbed8034000 tx=0x7fbed8009030 comp rx=0 tx=0).stop 2024-03-07T06:51:42.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.277+0000 7fbeeb000640 1 -- 172.21.15.133:0/597780530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbee415e720 msgr2=0x7fbee415eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:42.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.277+0000 7fbeeb000640 1 --2- 172.21.15.133:0/597780530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbee415e720 0x7fbee415eba0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbee4073c90 tx=0x7fbedc00dc00 comp rx=0 tx=0).stop 2024-03-07T06:51:42.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.277+0000 7fbeeb000640 1 -- 172.21.15.133:0/597780530 shutdown_connections 2024-03-07T06:51:42.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.277+0000 7fbeeb000640 1 --2- 172.21.15.133:0/597780530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbee415e720 0x7fbee415eba0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:42.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.277+0000 7fbeeb000640 1 --2- 172.21.15.133:0/597780530 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbec406b500 0x7fbec406d9c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:42.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.277+0000 7fbeeb000640 1 --2- 172.21.15.133:0/597780530 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbee415c110 0x7fbee415d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:42.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.277+0000 7fbeeb000640 1 --2- 172.21.15.133:0/597780530 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbecc0039e0 0x7fbee41f4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:42.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.277+0000 7fbeeb000640 1 -- 172.21.15.133:0/597780530 >> 172.21.15.133:0/597780530 conn(0x7fbee40b2e90 msgr2=0x7fbee40b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:42.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.277+0000 7fbeeb000640 1 -- 172.21.15.133:0/597780530 shutdown_connections 2024-03-07T06:51:42.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:42.277+0000 7fbeeb000640 1 -- 172.21.15.133:0/597780530 wait complete. 2024-03-07T06:51:42.297 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-07T06:51:42.297 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-07T06:51:42.297 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-03-07T06:51:42.297 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-07T06:51:42.297 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-07T06:51:42.297 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-07T06:51:42.297 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-07T06:51:42.298 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-07T06:51:42.298 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-07T06:51:42.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:42.298 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-03-07T06:51:42.353 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T06:51:42.353 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T06:51:42.353 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T06:51:42.353 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:42.353 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-03-07T06:51:42.353 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:42.353 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:51:42.353 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:42.353 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-03-07T06:51:42.353 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:51:42.354 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:42.354 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-03-07T06:51:42.354 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:51:42.354 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:42.354 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-07T06:51:42.354 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:42.354 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-07T06:51:42.354 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42518sec preferred_lft 42518sec 2024-03-07T06:51:42.354 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-03-07T06:51:42.354 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:42.354 INFO:teuthology.orchestra.run.smithi133.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T06:51:42.354 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 6e:50:ef:52:0c:c1 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:42.354 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-07T06:51:42.354 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:42.355 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::4c8:8aff:feed:a7f5/64 scope link 2024-03-07T06:51:42.355 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:42.355 INFO:teuthology.orchestra.run.smithi133.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-07T06:51:42.355 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 6e:50:ef:52:0c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-07T06:51:42.355 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::6c50:efff:fe52:cc1/64 scope link 2024-03-07T06:51:42.355 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:42.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:42.355 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-07T06:51:42.442 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-03-07T06:51:42.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:42.446 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-03-07T06:51:42.500 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-03-07T06:51:42.500 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-07T06:51:42.500 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-07T06:51:42.500 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-07T06:51:42.500 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-07T06:51:42.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:42.501 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-07T06:51:42.501 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-07T06:51:42.501 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-07T06:51:42.501 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-07T06:51:42.501 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-07T06:51:42.501 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-07T06:51:42.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:42.683 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-03-07T06:51:42.733 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-07T06:51:42.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:42.734 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2024-03-07T06:51:42.782 INFO:teuthology.orchestra.run.smithi133.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-07T06:51:42.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:42.784 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-07T06:51:42.784 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-07T06:51:42.784 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-07T06:51:42.784 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-07T06:51:42.784 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-07T06:51:42.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:42.919 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-07T06:51:43.009 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T06:51:43.009 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T06:51:43.009 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T06:51:43.010 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:43.010 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-03-07T06:51:43.010 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:43.010 INFO:teuthology.orchestra.run.smithi133.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T06:51:43.010 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 32:09:d4:3e:aa:df brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-07T06:51:43.010 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-07T06:51:43.010 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:43.010 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::3009:d4ff:fe3e:aadf/64 scope link 2024-03-07T06:51:43.010 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:43.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:43.014 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-07T06:51:43.107 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T06:51:43.107 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T06:51:43.107 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T06:51:43.107 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:43.107 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-03-07T06:51:43.107 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:43.107 INFO:teuthology.orchestra.run.smithi133.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T06:51:43.107 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 32:09:d4:3e:aa:df brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-07T06:51:43.107 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-07T06:51:43.107 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:43.107 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::3009:d4ff:fe3e:aadf/64 scope link 2024-03-07T06:51:43.107 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:43.110 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-07T06:51:43.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:43.111 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-07T06:51:43.111 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-07T06:51:43.111 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-07T06:51:43.111 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-07T06:51:43.111 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-07T06:51:43.111 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-07T06:51:43.111 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-07T06:51:43.111 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-07T06:51:43.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:43.381 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-07T06:51:43.381 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-07T06:51:43.381 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.1 up 2024-03-07T06:51:43.382 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.1 master ceph-brx 2024-03-07T06:51:43.382 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-07T06:51:43.550 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T06:51:43.550 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-07T06:51:43.595 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-07T06:51:43.595 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T06:51:43.595 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-07T06:51:43.691 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-03-07T06:51:43.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:43.692 DEBUG:teuthology.orchestra.run.smithi133:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:51:43.824 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:51:43.824 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T06:51:43.860 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:51:43.860 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T06:51:43.860 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: resolved to: "172.21.15.133:3300,172.21.15.161:3300,172.21.15.161:3301" 2024-03-07T06:51:43.860 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: trying mount with new device syntax: 2@1655279d-73c2-44d4-ba1b-3e8a15ae005a.cephfs=/ 2024-03-07T06:51:43.860 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.133:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2024-03-07T06:51:43.862 INFO:teuthology.orchestra.run.smithi133.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-03-07T06:51:43.862 INFO:teuthology.orchestra.run.smithi133.stdout: You just mounted a file system that supports labels which does not 2024-03-07T06:51:43.862 INFO:teuthology.orchestra.run.smithi133.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-07T06:51:43.862 INFO:teuthology.orchestra.run.smithi133.stdout: applications will generate AVC messages and not be allowed access to 2024-03-07T06:51:43.862 INFO:teuthology.orchestra.run.smithi133.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-07T06:51:43.867 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-07T06:51:43.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:43.867 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-07T06:51:43.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:43.952 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T06:51:44.007 INFO:teuthology.orchestra.run.smithi133.stdout:client4454 2024-03-07T06:51:44.007 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:51:44.007 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:51:44.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:44.059 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T06:51:44.161 INFO:teuthology.orchestra.run.smithi133.stdout:client4454 2024-03-07T06:51:44.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:44.162 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T06:51:44.216 INFO:teuthology.orchestra.run.smithi133.stdout:client4454 2024-03-07T06:51:44.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:44.216 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/1655279d-73c2-44d4-ba1b-3e8a15ae005a.client4454/status') 2024-03-07T06:51:44.306 INFO:teuthology.orchestra.run.smithi133.stdout:instance: client.4454 (3)172.21.15.133:0/475439090 2024-03-07T06:51:44.307 INFO:teuthology.orchestra.run.smithi133.stdout:blocklisted: false 2024-03-07T06:51:44.307 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records in 2024-03-07T06:51:44.307 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records out 2024-03-07T06:51:44.307 INFO:teuthology.orchestra.run.smithi133.stderr:70 bytes copied, 6.9789e-05 s, 1.0 MB/s 2024-03-07T06:51:44.311 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-03-07T06:51:44.311 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-07T06:51:44.311 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-07T06:51:44.311 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-07T06:51:44.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.521+0000 7fdb1f200640 1 Processor -- start 2024-03-07T06:51:44.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.522+0000 7fdb1f200640 1 -- start start 2024-03-07T06:51:44.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.523+0000 7fdb1f200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb2015eac0 0x7fdb2015eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:44.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.523+0000 7fdb1f200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdb20072f40 0x7fdb200ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:44.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.523+0000 7fdb1f200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb2006e3f0 con 0x7fdb2006dac0 2024-03-07T06:51:44.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.523+0000 7fdb1f200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb2006e6d0 con 0x7fdb20072f40 2024-03-07T06:51:44.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.523+0000 7fdb1f200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2015f3e0 con 0x7fdb2015eac0 2024-03-07T06:51:44.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.524+0000 7fdb1e800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdb20072f40 0x7fdb200ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:44.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.524+0000 7fdb1e800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fdb20072f40 0x7fdb200ae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45950/0 (socket says 172.21.15.133:45950) 2024-03-07T06:51:44.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.524+0000 7fdb1e800640 1 -- 172.21.15.133:0/761994286 learned_addr learned my addr 172.21.15.133:0/761994286 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:44.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.524+0000 7fdb1de00640 1 --2- 172.21.15.133:0/761994286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb2015eac0 0x7fdb2015eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:44.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.524+0000 7fdb1e800640 1 -- 172.21.15.133:0/761994286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb2015eac0 msgr2=0x7fdb2015eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:44.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.524+0000 7fdb1e800640 1 --2- 172.21.15.133:0/761994286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb2015eac0 0x7fdb2015eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:44.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.524+0000 7fdb1e800640 1 -- 172.21.15.133:0/761994286 >> v1:172.21.15.133:6789/0 conn(0x7fdb2006dac0 legacy=0x7fdb20072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:44.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.524+0000 7fdb1e800640 1 -- 172.21.15.133:0/761994286 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb200ad9c0 con 0x7fdb20072f40 2024-03-07T06:51:44.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.524+0000 7fdb1ca00640 1 -- 172.21.15.133:0/761994286 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 240695323 0 0) 0x7fdb2006e3f0 con 0x7fdb2006dac0 2024-03-07T06:51:44.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.525+0000 7fdb1e800640 1 --2- 172.21.15.133:0/761994286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb20072f40 0x7fdb200ae4a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009e80 tx=0x7fdb1802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fecf1893a17f31e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:44.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.525+0000 7fdb1ca00640 1 -- 172.21.15.133:0/761994286 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1802caf0 con 0x7fdb20072f40 2024-03-07T06:51:44.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.525+0000 7fdb1ca00640 1 -- 172.21.15.133:0/761994286 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1802cc90 con 0x7fdb20072f40 2024-03-07T06:51:44.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.525+0000 7fdb1ca00640 1 -- 172.21.15.133:0/761994286 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb18034610 con 0x7fdb20072f40 2024-03-07T06:51:44.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.526+0000 7fdb1f200640 1 -- 172.21.15.133:0/761994286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb20072f40 msgr2=0x7fdb200ae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:44.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.526+0000 7fdb1f200640 1 --2- 172.21.15.133:0/761994286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb20072f40 0x7fdb200ae4a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdb18009e80 tx=0x7fdb1802bf50 comp rx=0 tx=0).stop 2024-03-07T06:51:44.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.526+0000 7fdb1f200640 1 -- 172.21.15.133:0/761994286 shutdown_connections 2024-03-07T06:51:44.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.526+0000 7fdb1f200640 1 --2- 172.21.15.133:0/761994286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb20072f40 0x7fdb200ae4a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:44.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.526+0000 7fdb1f200640 1 --2- 172.21.15.133:0/761994286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb2015eac0 0x7fdb2015eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:44.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.526+0000 7fdb1f200640 1 -- 172.21.15.133:0/761994286 >> 172.21.15.133:0/761994286 conn(0x7fdb200b2e90 msgr2=0x7fdb200b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:44.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.526+0000 7fdb1f200640 1 -- 172.21.15.133:0/761994286 shutdown_connections 2024-03-07T06:51:44.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.526+0000 7fdb1f200640 1 -- 172.21.15.133:0/761994286 wait complete. 2024-03-07T06:51:44.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.528+0000 7fdb1f200640 1 Processor -- start 2024-03-07T06:51:44.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.528+0000 7fdb1f200640 1 -- start start 2024-03-07T06:51:44.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.529+0000 7fdb1f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb2006dac0 0x7fdb20157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:44.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.529+0000 7fdb1f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb20072f40 0x7fdb20155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:44.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.529+0000 7fdb1f200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb2015eac0 0x7fdb201483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:44.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.529+0000 7fdb1f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2006df20 con 0x7fdb2006dac0 2024-03-07T06:51:44.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.529+0000 7fdb1f200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb200adf80 con 0x7fdb20072f40 2024-03-07T06:51:44.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.529+0000 7fdb1f200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb2006e0a0 con 0x7fdb2015eac0 2024-03-07T06:51:44.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.529+0000 7fdb1de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb2006dac0 0x7fdb20157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:44.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.529+0000 7fdb1de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb2006dac0 0x7fdb20157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45954/0 (socket says 172.21.15.133:45954) 2024-03-07T06:51:44.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.529+0000 7fdb1de00640 1 -- 172.21.15.133:0/3177148265 learned_addr learned my addr 172.21.15.133:0/3177148265 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:44.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.529+0000 7fdb1de00640 1 -- 172.21.15.133:0/3177148265 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb2015eac0 msgr2=0x7fdb201483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:44.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.529+0000 7fdb1de00640 1 --2- 172.21.15.133:0/3177148265 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb2015eac0 0x7fdb201483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:44.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.529+0000 7fdb1de00640 1 -- 172.21.15.133:0/3177148265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb20072f40 msgr2=0x7fdb20155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:44.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.529+0000 7fdb1de00640 1 --2- 172.21.15.133:0/3177148265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb20072f40 0x7fdb20155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:44.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.529+0000 7fdb1de00640 1 -- 172.21.15.133:0/3177148265 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb20155ff0 con 0x7fdb2006dac0 2024-03-07T06:51:44.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.530+0000 7fdb1de00640 1 --2- 172.21.15.133:0/3177148265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb2006dac0 0x7fdb20157540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000d800 tx=0x7fdb1000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:44.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.530+0000 7fdb16000640 1 -- 172.21.15.133:0/3177148265 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1000b6c0 con 0x7fdb2006dac0 2024-03-07T06:51:44.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.530+0000 7fdb16000640 1 -- 172.21.15.133:0/3177148265 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb10019070 con 0x7fdb2006dac0 2024-03-07T06:51:44.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.530+0000 7fdb1f200640 1 -- 172.21.15.133:0/3177148265 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb201562d0 con 0x7fdb2006dac0 2024-03-07T06:51:44.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.530+0000 7fdb16000640 1 -- 172.21.15.133:0/3177148265 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1000ba80 con 0x7fdb2006dac0 2024-03-07T06:51:44.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.530+0000 7fdb1f200640 1 -- 172.21.15.133:0/3177148265 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb20156b70 con 0x7fdb2006dac0 2024-03-07T06:51:44.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.531+0000 7fdb1f200640 1 -- 172.21.15.133:0/3177148265 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb201564a0 con 0x7fdb2006dac0 2024-03-07T06:51:44.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.532+0000 7fdb16000640 1 -- 172.21.15.133:0/3177148265 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdb1000bc20 con 0x7fdb2006dac0 2024-03-07T06:51:44.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.533+0000 7fdb16000640 1 --2- 172.21.15.133:0/3177148265 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdb0006b4d0 0x7fdb0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:44.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.533+0000 7fdb16000640 1 -- 172.21.15.133:0/3177148265 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb1008a750 con 0x7fdb2006dac0 2024-03-07T06:51:44.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.533+0000 7fdb1d400640 1 --2- 172.21.15.133:0/3177148265 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdb0006b4d0 0x7fdb0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:44.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.534+0000 7fdb1d400640 1 --2- 172.21.15.133:0/3177148265 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdb0006b4d0 0x7fdb0006d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdb200b3380 tx=0x7fdb0c007930 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:44.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.537+0000 7fdb16000640 1 -- 172.21.15.133:0/3177148265 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdb10059530 con 0x7fdb2006dac0 2024-03-07T06:51:44.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.771+0000 7fdb1f200640 1 -- 172.21.15.133:0/3177148265 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdb20062c90 con 0x7fdb2006dac0 2024-03-07T06:51:44.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.772+0000 7fdb16000640 1 -- 172.21.15.133:0/3177148265 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdb10058e30 con 0x7fdb2006dac0 2024-03-07T06:51:44.772 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-07T06:51:44.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.775+0000 7fdb1f200640 1 -- 172.21.15.133:0/3177148265 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdb0006b4d0 msgr2=0x7fdb0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:44.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.775+0000 7fdb1f200640 1 --2- 172.21.15.133:0/3177148265 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdb0006b4d0 0x7fdb0006d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdb200b3380 tx=0x7fdb0c007930 comp rx=0 tx=0).stop 2024-03-07T06:51:44.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.775+0000 7fdb1f200640 1 -- 172.21.15.133:0/3177148265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb2006dac0 msgr2=0x7fdb20157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:44.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.775+0000 7fdb1f200640 1 --2- 172.21.15.133:0/3177148265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb2006dac0 0x7fdb20157540 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdb1000d800 tx=0x7fdb1000dcc0 comp rx=0 tx=0).stop 2024-03-07T06:51:44.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.775+0000 7fdb1f200640 1 -- 172.21.15.133:0/3177148265 shutdown_connections 2024-03-07T06:51:44.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.775+0000 7fdb1f200640 1 --2- 172.21.15.133:0/3177148265 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb2015eac0 0x7fdb201483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:44.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.775+0000 7fdb1f200640 1 --2- 172.21.15.133:0/3177148265 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdb0006b4d0 0x7fdb0006d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:44.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.775+0000 7fdb1f200640 1 --2- 172.21.15.133:0/3177148265 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb20072f40 0x7fdb20155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:44.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.775+0000 7fdb1f200640 1 --2- 172.21.15.133:0/3177148265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb2006dac0 0x7fdb20157540 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:44.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.775+0000 7fdb1f200640 1 -- 172.21.15.133:0/3177148265 >> 172.21.15.133:0/3177148265 conn(0x7fdb200b2e90 msgr2=0x7fdb20160480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:44.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.775+0000 7fdb1f200640 1 -- 172.21.15.133:0/3177148265 shutdown_connections 2024-03-07T06:51:44.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:44.775+0000 7fdb1f200640 1 -- 172.21.15.133:0/3177148265 wait complete. 2024-03-07T06:51:44.793 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-07T06:51:44.794 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-07T06:51:44.794 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-03-07T06:51:44.794 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-07T06:51:44.794 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-07T06:51:44.794 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-07T06:51:44.794 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-07T06:51:44.794 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-07T06:51:44.794 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-07T06:51:44.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:44.794 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-03-07T06:51:44.848 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T06:51:44.848 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T06:51:44.848 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T06:51:44.848 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42550sec preferred_lft 42550sec 2024-03-07T06:51:44.849 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-03-07T06:51:44.850 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:44.850 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-07T06:51:44.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:44.850 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-03-07T06:51:44.850 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-03-07T06:51:44.850 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add name ceph-brx type bridge 2024-03-07T06:51:44.850 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip addr flush dev ceph-brx 2024-03-07T06:51:44.850 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set ceph-brx up 2024-03-07T06:51:44.850 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-07T06:51:44.850 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-03-07T06:51:45.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:45.271 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-07T06:51:45.344 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-03-07T06:51:45.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:45.349 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-03-07T06:51:45.399 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-03-07T06:51:45.399 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-07T06:51:45.399 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-07T06:51:45.399 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-07T06:51:45.399 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-07T06:51:45.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:45.400 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-03-07T06:51:45.400 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-03-07T06:51:45.400 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-07T06:51:45.400 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-07T06:51:45.400 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-07T06:51:45.400 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-03-07T06:51:46.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:46.126 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-03-07T06:51:46.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:46.181 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list-id 2024-03-07T06:51:46.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:46.227 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-03-07T06:51:46.227 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-03-07T06:51:46.228 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-07T06:51:46.228 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-07T06:51:46.228 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-03-07T06:51:46.414 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-07T06:51:46.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:46.415 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-03-07T06:51:46.415 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-03-07T06:51:46.415 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-07T06:51:46.415 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-07T06:51:46.415 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-07T06:51:46.415 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-07T06:51:46.415 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-07T06:51:46.415 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-03-07T06:51:46.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:46.683 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-03-07T06:51:46.683 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-03-07T06:51:46.683 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.0 up 2024-03-07T06:51:46.683 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set dev brx.0 master ceph-brx 2024-03-07T06:51:46.683 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-03-07T06:51:46.836 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T06:51:46.836 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-07T06:51:46.882 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-07T06:51:46.882 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T06:51:46.882 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-07T06:51:46.978 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-03-07T06:51:46.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:46.978 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:51:47.124 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:51:47.124 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T06:51:47.802 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:51:47.802 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T06:51:47.802 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.133:3300,172.21.15.161:3300,172.21.15.161:3301" 2024-03-07T06:51:47.802 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 1@1655279d-73c2-44d4-ba1b-3e8a15ae005a.cephfs=/ 2024-03-07T06:51:47.803 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.133:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2024-03-07T06:51:47.805 INFO:teuthology.orchestra.run.smithi161.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-07T06:51:47.805 INFO:teuthology.orchestra.run.smithi161.stdout: You just mounted a file system that supports labels which does not 2024-03-07T06:51:47.805 INFO:teuthology.orchestra.run.smithi161.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-07T06:51:47.805 INFO:teuthology.orchestra.run.smithi161.stdout: applications will generate AVC messages and not be allowed access to 2024-03-07T06:51:47.805 INFO:teuthology.orchestra.run.smithi161.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-07T06:51:47.809 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-07T06:51:47.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:47.810 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-07T06:51:47.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:47.888 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T06:51:47.941 INFO:teuthology.orchestra.run.smithi161.stdout:client4585 2024-03-07T06:51:47.941 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:51:47.941 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:51:47.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:47.992 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T06:51:48.043 INFO:teuthology.orchestra.run.smithi161.stdout:client4585 2024-03-07T06:51:48.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:48.043 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T06:51:48.095 INFO:teuthology.orchestra.run.smithi161.stdout:client4585 2024-03-07T06:51:48.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:48.095 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/1655279d-73c2-44d4-ba1b-3e8a15ae005a.client4585/status') 2024-03-07T06:51:48.175 INFO:teuthology.orchestra.run.smithi161.stdout:instance: client.4585 (3)192.168.0.1:0/2103472987 2024-03-07T06:51:48.176 INFO:teuthology.orchestra.run.smithi161.stdout:blocklisted: false 2024-03-07T06:51:48.176 INFO:teuthology.orchestra.run.smithi161.stderr:0+1 records in 2024-03-07T06:51:48.176 INFO:teuthology.orchestra.run.smithi161.stderr:0+1 records out 2024-03-07T06:51:48.176 INFO:teuthology.orchestra.run.smithi161.stderr:69 bytes copied, 6.9516e-05 s, 993 kB/s 2024-03-07T06:51:48.179 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-03-07T06:51:48.180 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-07T06:51:48.180 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-07T06:51:48.180 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-07T06:51:48.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.402+0000 7fa6fac00640 1 Processor -- start 2024-03-07T06:51:48.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.402+0000 7fa6fac00640 1 -- start start 2024-03-07T06:51:48.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.404+0000 7fa6fac00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa6f415b1b0 0x7fa6f40b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:48.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.404+0000 7fa6fac00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6f40ce5f0 0x7fa6f4071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:48.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.404+0000 7fa6fac00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f406f140 con 0x7fa6f40af950 2024-03-07T06:51:48.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.404+0000 7fa6fac00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f4071610 con 0x7fa6f415b1b0 2024-03-07T06:51:48.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.404+0000 7fa6fac00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f406f420 con 0x7fa6f40ce5f0 2024-03-07T06:51:48.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.404+0000 7fa6ebe00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa6f40af950 0x7fa6f40cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42464/0 (socket says 172.21.15.133:42464) 2024-03-07T06:51:48.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.404+0000 7fa6ebe00640 1 -- 172.21.15.133:0/1028711316 learned_addr learned my addr 172.21.15.133:0/1028711316 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:48.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.404+0000 7fa6f3e00640 1 --2- 172.21.15.133:0/1028711316 >> v2:172.21.15.161:3300/0 conn(0x7fa6f415b1b0 0x7fa6f40b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:48.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.405+0000 7fa6f8c00640 1 --2- 172.21.15.133:0/1028711316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6f40ce5f0 0x7fa6f4071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:48.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.405+0000 7fa6f3400640 1 -- 172.21.15.133:0/1028711316 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944722513 0 0) 0x7fa6f406f140 con 0x7fa6f40af950 2024-03-07T06:51:48.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.405+0000 7fa6f3400640 1 -- 172.21.15.133:0/1028711316 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6d8003520 con 0x7fa6f40af950 2024-03-07T06:51:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.405+0000 7fa6f3e00640 1 -- 172.21.15.133:0/1028711316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6f40ce5f0 msgr2=0x7fa6f4071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.405+0000 7fa6f3e00640 1 --2- 172.21.15.133:0/1028711316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6f40ce5f0 0x7fa6f4071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.405+0000 7fa6f3e00640 1 -- 172.21.15.133:0/1028711316 >> v1:172.21.15.133:6789/0 conn(0x7fa6f40af950 legacy=0x7fa6f40cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.405+0000 7fa6f3e00640 1 -- 172.21.15.133:0/1028711316 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f4071860 con 0x7fa6f415b1b0 2024-03-07T06:51:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.405+0000 7fa6f8c00640 1 --2- 172.21.15.133:0/1028711316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6f40ce5f0 0x7fa6f4071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:51:48.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.406+0000 7fa6f3e00640 1 --2- 172.21.15.133:0/1028711316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f415b1b0 0x7fa6f40b3c10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40097a0 tx=0x7fa6e402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1193e9dc456dc2ec server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:48.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.406+0000 7fa6f3400640 1 -- 172.21.15.133:0/1028711316 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e40315a0 con 0x7fa6f415b1b0 2024-03-07T06:51:48.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.406+0000 7fa6f3400640 1 -- 172.21.15.133:0/1028711316 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e4037070 con 0x7fa6f415b1b0 2024-03-07T06:51:48.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.406+0000 7fa6f3400640 1 -- 172.21.15.133:0/1028711316 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4031960 con 0x7fa6f415b1b0 2024-03-07T06:51:48.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.407+0000 7fa6fac00640 1 -- 172.21.15.133:0/1028711316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f415b1b0 msgr2=0x7fa6f40b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:48.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.407+0000 7fa6fac00640 1 --2- 172.21.15.133:0/1028711316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f415b1b0 0x7fa6f40b3c10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa6e40097a0 tx=0x7fa6e402b840 comp rx=0 tx=0).stop 2024-03-07T06:51:48.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.407+0000 7fa6fac00640 1 -- 172.21.15.133:0/1028711316 shutdown_connections 2024-03-07T06:51:48.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.407+0000 7fa6fac00640 1 --2- 172.21.15.133:0/1028711316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6f40ce5f0 0x7fa6f4071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:48.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.407+0000 7fa6fac00640 1 --2- 172.21.15.133:0/1028711316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f415b1b0 0x7fa6f40b3c10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:48.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.407+0000 7fa6fac00640 1 -- 172.21.15.133:0/1028711316 >> 172.21.15.133:0/1028711316 conn(0x7fa6f40b3120 msgr2=0x7fa6f40b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:48.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.407+0000 7fa6fac00640 1 -- 172.21.15.133:0/1028711316 shutdown_connections 2024-03-07T06:51:48.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.407+0000 7fa6fac00640 1 -- 172.21.15.133:0/1028711316 wait complete. 2024-03-07T06:51:48.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.409+0000 7fa6fac00640 1 Processor -- start 2024-03-07T06:51:48.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.409+0000 7fa6fac00640 1 -- start start 2024-03-07T06:51:48.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.410+0000 7fa6fac00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f40af950 0x7fa6f415bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:48.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.410+0000 7fa6fac00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f40ce5f0 0x7fa6f409d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:48.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.410+0000 7fa6fac00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6f415b1b0 0x7fa6f4156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:48.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.410+0000 7fa6fac00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f415b590 con 0x7fa6f40ce5f0 2024-03-07T06:51:48.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.410+0000 7fa6fac00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f406ee30 con 0x7fa6f40af950 2024-03-07T06:51:48.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.410+0000 7fa6fac00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f415b710 con 0x7fa6f415b1b0 2024-03-07T06:51:48.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.411+0000 7fa6ebe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f40ce5f0 0x7fa6f409d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:48.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.411+0000 7fa6ebe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f40ce5f0 0x7fa6f409d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45958/0 (socket says 172.21.15.133:45958) 2024-03-07T06:51:48.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.411+0000 7fa6ebe00640 1 -- 172.21.15.133:0/2373056778 learned_addr learned my addr 172.21.15.133:0/2373056778 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:48.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.411+0000 7fa6f3e00640 1 --2- 172.21.15.133:0/2373056778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f40af950 0x7fa6f415bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:48.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.411+0000 7fa6f8c00640 1 --2- 172.21.15.133:0/2373056778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6f415b1b0 0x7fa6f4156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:48.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.411+0000 7fa6ebe00640 1 -- 172.21.15.133:0/2373056778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6f415b1b0 msgr2=0x7fa6f4156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:48.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.411+0000 7fa6ebe00640 1 --2- 172.21.15.133:0/2373056778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6f415b1b0 0x7fa6f4156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:48.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.411+0000 7fa6ebe00640 1 -- 172.21.15.133:0/2373056778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f40af950 msgr2=0x7fa6f415bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:48.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.411+0000 7fa6ebe00640 1 --2- 172.21.15.133:0/2373056778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f40af950 0x7fa6f415bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:48.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.411+0000 7fa6ebe00640 1 -- 172.21.15.133:0/2373056778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f409d830 con 0x7fa6f40ce5f0 2024-03-07T06:51:48.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.411+0000 7fa6ebe00640 1 --2- 172.21.15.133:0/2373056778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f40ce5f0 0x7fa6f409d2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00c430 tx=0x7fa6dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:48.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.412+0000 7fa6f0c00640 1 -- 172.21.15.133:0/2373056778 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc017070 con 0x7fa6f40ce5f0 2024-03-07T06:51:48.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.412+0000 7fa6f0c00640 1 -- 172.21.15.133:0/2373056778 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6dc005ce0 con 0x7fa6f40ce5f0 2024-03-07T06:51:48.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.412+0000 7fa6f0c00640 1 -- 172.21.15.133:0/2373056778 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6dc0072f0 con 0x7fa6f40ce5f0 2024-03-07T06:51:48.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.412+0000 7fa6fac00640 1 -- 172.21.15.133:0/2373056778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f41570e0 con 0x7fa6f40ce5f0 2024-03-07T06:51:48.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.412+0000 7fa6fac00640 1 -- 172.21.15.133:0/2373056778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f415c6d0 con 0x7fa6f40ce5f0 2024-03-07T06:51:48.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.413+0000 7fa6fac00640 1 -- 172.21.15.133:0/2373056778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f4062c90 con 0x7fa6f40ce5f0 2024-03-07T06:51:48.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.414+0000 7fa6f0c00640 1 -- 172.21.15.133:0/2373056778 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa6dc01a070 con 0x7fa6f40ce5f0 2024-03-07T06:51:48.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.415+0000 7fa6f0c00640 1 --2- 172.21.15.133:0/2373056778 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa6d006b4d0 0x7fa6d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:48.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.415+0000 7fa6f0c00640 1 -- 172.21.15.133:0/2373056778 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6dc089600 con 0x7fa6f40ce5f0 2024-03-07T06:51:48.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.415+0000 7fa6f3e00640 1 --2- 172.21.15.133:0/2373056778 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa6d006b4d0 0x7fa6d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:48.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.416+0000 7fa6f3e00640 1 --2- 172.21.15.133:0/2373056778 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa6d006b4d0 0x7fa6d006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4036000 tx=0x7fa6e403d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:48.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.419+0000 7fa6f0c00640 1 -- 172.21.15.133:0/2373056778 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa6dc057e30 con 0x7fa6f40ce5f0 2024-03-07T06:51:48.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.655+0000 7fa6fac00640 1 -- 172.21.15.133:0/2373056778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa6f415a960 con 0x7fa6f40ce5f0 2024-03-07T06:51:48.656 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.656+0000 7fa6f0c00640 1 -- 172.21.15.133:0/2373056778 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa6dc05bed0 con 0x7fa6f40ce5f0 2024-03-07T06:51:48.656 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-07T06:51:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.659+0000 7fa6fac00640 1 -- 172.21.15.133:0/2373056778 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa6d006b4d0 msgr2=0x7fa6d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.659+0000 7fa6fac00640 1 --2- 172.21.15.133:0/2373056778 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa6d006b4d0 0x7fa6d006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4036000 tx=0x7fa6e403d040 comp rx=0 tx=0).stop 2024-03-07T06:51:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.659+0000 7fa6fac00640 1 -- 172.21.15.133:0/2373056778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f40ce5f0 msgr2=0x7fa6f409d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.659+0000 7fa6fac00640 1 --2- 172.21.15.133:0/2373056778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f40ce5f0 0x7fa6f409d2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc00c430 tx=0x7fa6dc00c8f0 comp rx=0 tx=0).stop 2024-03-07T06:51:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.659+0000 7fa6fac00640 1 -- 172.21.15.133:0/2373056778 shutdown_connections 2024-03-07T06:51:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.659+0000 7fa6fac00640 1 --2- 172.21.15.133:0/2373056778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6f415b1b0 0x7fa6f4156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.659+0000 7fa6fac00640 1 --2- 172.21.15.133:0/2373056778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f40ce5f0 0x7fa6f409d2f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.659+0000 7fa6fac00640 1 --2- 172.21.15.133:0/2373056778 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa6d006b4d0 0x7fa6d006d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.659+0000 7fa6fac00640 1 --2- 172.21.15.133:0/2373056778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6f40af950 0x7fa6f415bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.659+0000 7fa6fac00640 1 -- 172.21.15.133:0/2373056778 >> 172.21.15.133:0/2373056778 conn(0x7fa6f40b3120 msgr2=0x7fa6f40b4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:48.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.659+0000 7fa6fac00640 1 -- 172.21.15.133:0/2373056778 shutdown_connections 2024-03-07T06:51:48.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:48.659+0000 7fa6fac00640 1 -- 172.21.15.133:0/2373056778 wait complete. 2024-03-07T06:51:48.676 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-07T06:51:48.677 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-07T06:51:48.677 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi161.front.sepia.ceph.com 2024-03-07T06:51:48.677 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-07T06:51:48.677 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-07T06:51:48.677 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-07T06:51:48.677 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-07T06:51:48.677 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-07T06:51:48.677 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-07T06:51:48.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:48.678 DEBUG:teuthology.orchestra.run.smithi161:> ip addr 2024-03-07T06:51:48.732 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T06:51:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T06:51:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T06:51:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-03-07T06:51:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:48.733 INFO:teuthology.orchestra.run.smithi161.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:51:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7c brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f0 2024-03-07T06:51:48.733 INFO:teuthology.orchestra.run.smithi161.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:51:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:d9:36:7d brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:48.733 INFO:teuthology.orchestra.run.smithi161.stdout: altname enp7s0f1 2024-03-07T06:51:48.733 INFO:teuthology.orchestra.run.smithi161.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:50 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 0c:c4:7a:88:72:51 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: inet 172.21.15.161/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft 42547sec preferred_lft 42547sec 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::ec4:7aff:fe88:7251/64 scope link 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 32:37:10:43:04:45 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::7c9d:c3ff:febd:61dc/64 scope link 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-07T06:51:48.734 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 32:37:10:43:04:45 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-07T06:51:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::3037:10ff:fe43:445/64 scope link 2024-03-07T06:51:48.735 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:48.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:48.736 DEBUG:teuthology.orchestra.run.smithi161:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-07T06:51:48.819 INFO:teuthology.orchestra.run.smithi161.stdout:1 2024-03-07T06:51:48.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:48.822 DEBUG:teuthology.orchestra.run.smithi161:> route 2024-03-07T06:51:48.874 INFO:teuthology.orchestra.run.smithi161.stdout:Kernel IP routing table 2024-03-07T06:51:48.874 INFO:teuthology.orchestra.run.smithi161.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-07T06:51:48.874 INFO:teuthology.orchestra.run.smithi161.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-07T06:51:48.874 INFO:teuthology.orchestra.run.smithi161.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-07T06:51:48.874 INFO:teuthology.orchestra.run.smithi161.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-07T06:51:48.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:48.875 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-03-07T06:51:48.875 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-03-07T06:51:48.875 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-07T06:51:48.875 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-07T06:51:48.875 DEBUG:teuthology.orchestra.run.smithi161:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-07T06:51:48.875 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-03-07T06:51:49.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:49.063 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-03-07T06:51:49.116 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-07T06:51:49.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:49.118 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list-id 2024-03-07T06:51:49.168 INFO:teuthology.orchestra.run.smithi161.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-07T06:51:49.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:49.170 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-03-07T06:51:49.170 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-03-07T06:51:49.170 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-07T06:51:49.170 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-07T06:51:49.170 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-03-07T06:51:49.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:49.314 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-07T06:51:49.394 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T06:51:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T06:51:49.394 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T06:51:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-03-07T06:51:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:49.395 INFO:teuthology.orchestra.run.smithi161.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T06:51:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 92:41:3d:71:90:37 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-07T06:51:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-07T06:51:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::9041:3dff:fe71:9037/64 scope link 2024-03-07T06:51:49.395 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:49.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:49.399 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-07T06:51:49.490 INFO:teuthology.orchestra.run.smithi161.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T06:51:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T06:51:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T06:51:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 ::1/128 scope host 2024-03-07T06:51:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:49.491 INFO:teuthology.orchestra.run.smithi161.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T06:51:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: link/ether 92:41:3d:71:90:37 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-07T06:51:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-07T06:51:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: inet6 fe80::9041:3dff:fe71:9037/64 scope link 2024-03-07T06:51:49.491 INFO:teuthology.orchestra.run.smithi161.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:51:49.495 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-07T06:51:49.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:49.495 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-03-07T06:51:49.495 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-03-07T06:51:49.495 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-07T06:51:49.496 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-07T06:51:49.496 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-07T06:51:49.496 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-07T06:51:49.496 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-07T06:51:49.496 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-03-07T06:51:49.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:49.766 DEBUG:teuthology.orchestra.run.smithi161:> (cd / && exec bash -c ' 2024-03-07T06:51:49.766 DEBUG:teuthology.orchestra.run.smithi161:> set -e 2024-03-07T06:51:49.766 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set brx.1 up 2024-03-07T06:51:49.766 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link set dev brx.1 master ceph-brx 2024-03-07T06:51:49.766 DEBUG:teuthology.orchestra.run.smithi161:> ') 2024-03-07T06:51:49.940 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T06:51:49.940 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-07T06:51:49.985 INFO:teuthology.orchestra.run.smithi161.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-07T06:51:49.986 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T06:51:49.986 DEBUG:teuthology.orchestra.run.smithi161:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-07T06:51:50.073 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-03-07T06:51:50.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:50.073 DEBUG:teuthology.orchestra.run.smithi161:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:51:50.255 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:51:50.255 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T06:51:50.289 INFO:teuthology.orchestra.run.smithi161.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:51:50.290 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T06:51:50.290 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: resolved to: "172.21.15.133:3300,172.21.15.161:3300,172.21.15.161:3301" 2024-03-07T06:51:50.290 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: trying mount with new device syntax: 3@1655279d-73c2-44d4-ba1b-3e8a15ae005a.cephfs=/ 2024-03-07T06:51:50.290 INFO:teuthology.orchestra.run.smithi161.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.133:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2024-03-07T06:51:50.291 INFO:teuthology.orchestra.run.smithi161.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-03-07T06:51:50.291 INFO:teuthology.orchestra.run.smithi161.stdout: You just mounted a file system that supports labels which does not 2024-03-07T06:51:50.291 INFO:teuthology.orchestra.run.smithi161.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-07T06:51:50.292 INFO:teuthology.orchestra.run.smithi161.stdout: applications will generate AVC messages and not be allowed access to 2024-03-07T06:51:50.292 INFO:teuthology.orchestra.run.smithi161.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-07T06:51:50.297 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-07T06:51:50.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:50.297 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-07T06:51:50.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:50.366 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T06:51:50.413 INFO:teuthology.orchestra.run.smithi161.stdout:client4725 2024-03-07T06:51:50.414 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:51:50.414 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:51:50.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:50.458 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T06:51:50.556 INFO:teuthology.orchestra.run.smithi161.stdout:client4725 2024-03-07T06:51:50.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:50.556 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T06:51:50.606 INFO:teuthology.orchestra.run.smithi161.stdout:client4725 2024-03-07T06:51:50.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:51:50.607 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/1655279d-73c2-44d4-ba1b-3e8a15ae005a.client4725/status') 2024-03-07T06:51:50.691 INFO:teuthology.orchestra.run.smithi161.stdout:instance: client.4725 (3)172.21.15.161:0/199126240 2024-03-07T06:51:50.691 INFO:teuthology.orchestra.run.smithi161.stdout:blocklisted: false 2024-03-07T06:51:50.691 INFO:teuthology.orchestra.run.smithi161.stderr:0+1 records in 2024-03-07T06:51:50.691 INFO:teuthology.orchestra.run.smithi161.stderr:0+1 records out 2024-03-07T06:51:50.691 INFO:teuthology.orchestra.run.smithi161.stderr:70 bytes copied, 6.8246e-05 s, 1.0 MB/s 2024-03-07T06:51:50.695 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-07T06:51:50.713 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:50.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.925+0000 7f875d000640 1 Processor -- start 2024-03-07T06:51:50.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.925+0000 7f875d000640 1 -- start start 2024-03-07T06:51:50.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.927+0000 7f875d000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f87580b3c10 0x7f8758071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:50.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.927+0000 7f875d000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8758071b50 0x7f8758162f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:50.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.927+0000 7f875d000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8758071fe0 con 0x7f8758165ed0 2024-03-07T06:51:50.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.927+0000 7f875d000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87580734a0 con 0x7f87580b3c10 2024-03-07T06:51:50.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.927+0000 7f875d000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87580ce540 con 0x7f8758071b50 2024-03-07T06:51:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.927+0000 7f874fe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f87580b3c10 0x7f8758071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.927+0000 7f874fe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f87580b3c10 0x7f8758071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37012/0 (socket says 172.21.15.133:37012) 2024-03-07T06:51:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.927+0000 7f874fe00640 1 -- 172.21.15.133:0/491550617 learned_addr learned my addr 172.21.15.133:0/491550617 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.927+0000 7f8756800640 1 --2- 172.21.15.133:0/491550617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8758071b50 0x7f8758162f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.928+0000 7f874fe00640 1 -- 172.21.15.133:0/491550617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8758071b50 msgr2=0x7f8758162f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.928+0000 7f874fe00640 1 --2- 172.21.15.133:0/491550617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8758071b50 0x7f8758162f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.928+0000 7f874fe00640 1 -- 172.21.15.133:0/491550617 >> v1:172.21.15.161:6789/0 conn(0x7f8758165ed0 legacy=0x7f875806ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.928+0000 7f874fe00640 1 -- 172.21.15.133:0/491550617 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87580cdf80 con 0x7f87580b3c10 2024-03-07T06:51:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.929+0000 7f874fe00640 1 --2- 172.21.15.133:0/491550617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87580b3c10 0x7f8758071610 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8740009870 tx=0x7f874002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342463aeb188165f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.929+0000 7f8755400640 1 -- 172.21.15.133:0/491550617 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740034070 con 0x7f87580b3c10 2024-03-07T06:51:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.929+0000 7f8755400640 1 -- 172.21.15.133:0/491550617 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8740030840 con 0x7f87580b3c10 2024-03-07T06:51:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.929+0000 7f8755400640 1 -- 172.21.15.133:0/491550617 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740030c10 con 0x7f87580b3c10 2024-03-07T06:51:50.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.930+0000 7f875d000640 1 -- 172.21.15.133:0/491550617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87580b3c10 msgr2=0x7f8758071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:50.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.930+0000 7f875d000640 1 --2- 172.21.15.133:0/491550617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87580b3c10 0x7f8758071610 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8740009870 tx=0x7f874002b980 comp rx=0 tx=0).stop 2024-03-07T06:51:50.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.930+0000 7f875d000640 1 -- 172.21.15.133:0/491550617 shutdown_connections 2024-03-07T06:51:50.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.930+0000 7f875d000640 1 --2- 172.21.15.133:0/491550617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8758071b50 0x7f8758162f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:50.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.930+0000 7f875d000640 1 --2- 172.21.15.133:0/491550617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87580b3c10 0x7f8758071610 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:50.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.930+0000 7f875d000640 1 -- 172.21.15.133:0/491550617 >> 172.21.15.133:0/491550617 conn(0x7f87580b3120 msgr2=0x7f87580b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:50.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.930+0000 7f875d000640 1 -- 172.21.15.133:0/491550617 shutdown_connections 2024-03-07T06:51:50.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.930+0000 7f875d000640 1 -- 172.21.15.133:0/491550617 wait complete. 2024-03-07T06:51:50.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.932+0000 7f875d000640 1 Processor -- start 2024-03-07T06:51:50.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.932+0000 7f875d000640 1 -- start start 2024-03-07T06:51:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.933+0000 7f875d000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8758071b50 0x7f87581f7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.933+0000 7f875d000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87580b3c10 0x7f875815f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.933+0000 7f875d000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8758165ed0 0x7f87580c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.933+0000 7f875d000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8758072550 con 0x7f8758165ed0 2024-03-07T06:51:50.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.933+0000 7f875d000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8758072270 con 0x7f87580b3c10 2024-03-07T06:51:50.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.933+0000 7f875d000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87581634d0 con 0x7f8758071b50 2024-03-07T06:51:50.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.933+0000 7f8756800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8758165ed0 0x7f87580c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:50.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.933+0000 7f8756800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8758165ed0 0x7f87580c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37020/0 (socket says 172.21.15.133:37020) 2024-03-07T06:51:50.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.933+0000 7f8756800640 1 -- 172.21.15.133:0/2217641151 learned_addr learned my addr 172.21.15.133:0/2217641151 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:50.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.933+0000 7f874fe00640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87580b3c10 0x7f875815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:50.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.933+0000 7f8755e00640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8758071b50 0x7f87581f7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:50.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.934+0000 7f8756800640 1 -- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8758071b50 msgr2=0x7f87581f7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:50.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.934+0000 7f8756800640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8758071b50 0x7f87581f7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:50.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.934+0000 7f8756800640 1 -- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87580b3c10 msgr2=0x7f875815f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:50.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.934+0000 7f8756800640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87580b3c10 0x7f875815f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:50.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.934+0000 7f8756800640 1 -- 172.21.15.133:0/2217641151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875815f990 con 0x7f8758165ed0 2024-03-07T06:51:50.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.934+0000 7f8755e00640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8758071b50 0x7f87581f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:50.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.934+0000 7f8756800640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8758165ed0 0x7f87580c5090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8750007e70 tx=0x7f875000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:50.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.934+0000 7f874e000640 1 -- 172.21.15.133:0/2217641151 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8750014620 con 0x7f8758165ed0 2024-03-07T06:51:50.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.934+0000 7f874e000640 1 -- 172.21.15.133:0/2217641151 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8750019070 con 0x7f8758165ed0 2024-03-07T06:51:50.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.934+0000 7f874e000640 1 -- 172.21.15.133:0/2217641151 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8750014a20 con 0x7f8758165ed0 2024-03-07T06:51:50.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.934+0000 7f875d000640 1 -- 172.21.15.133:0/2217641151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87581f8210 con 0x7f8758165ed0 2024-03-07T06:51:50.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.934+0000 7f875d000640 1 -- 172.21.15.133:0/2217641151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87581f87d0 con 0x7f8758165ed0 2024-03-07T06:51:50.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.935+0000 7f875d000640 1 -- 172.21.15.133:0/2217641151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8758062c90 con 0x7f8758165ed0 2024-03-07T06:51:50.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.936+0000 7f874e000640 1 -- 172.21.15.133:0/2217641151 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8750014bc0 con 0x7f8758165ed0 2024-03-07T06:51:50.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.937+0000 7f874e000640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f873406b4d0 0x7f873406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:50.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.937+0000 7f874e000640 1 -- 172.21.15.133:0/2217641151 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f875008ab10 con 0x7f8758165ed0 2024-03-07T06:51:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.938+0000 7f8755e00640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f873406b4d0 0x7f873406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:50.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.939+0000 7f8755e00640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f873406b4d0 0x7f873406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f87480015e0 tx=0x7f8748009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:50.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:50.942+0000 7f874e000640 1 -- 172.21.15.133:0/2217641151 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8750059980 con 0x7f8758165ed0 2024-03-07T06:51:51.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.140+0000 7f875d000640 1 -- 172.21.15.133:0/2217641151 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87580c55d0 con 0x7f8758165ed0 2024-03-07T06:51:51.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.144+0000 7f874e000640 1 -- 172.21.15.133:0/2217641151 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f8750059280 con 0x7f8758165ed0 2024-03-07T06:51:51.144 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:51.144 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":21,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:16.174650+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:51.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.147+0000 7f875d000640 1 -- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f873406b4d0 msgr2=0x7f873406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:51.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.147+0000 7f875d000640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f873406b4d0 0x7f873406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f87480015e0 tx=0x7f8748009040 comp rx=0 tx=0).stop 2024-03-07T06:51:51.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.147+0000 7f875d000640 1 -- 172.21.15.133:0/2217641151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8758165ed0 msgr2=0x7f87580c5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:51.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.147+0000 7f875d000640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8758165ed0 0x7f87580c5090 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8750007e70 tx=0x7f875000e4b0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.148+0000 7f875d000640 1 -- 172.21.15.133:0/2217641151 shutdown_connections 2024-03-07T06:51:51.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.148+0000 7f875d000640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8758165ed0 0x7f87580c5090 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.148+0000 7f875d000640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f873406b4d0 0x7f873406d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.148+0000 7f875d000640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87580b3c10 0x7f875815f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.148+0000 7f875d000640 1 --2- 172.21.15.133:0/2217641151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8758071b50 0x7f87581f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.148+0000 7f875d000640 1 -- 172.21.15.133:0/2217641151 >> 172.21.15.133:0/2217641151 conn(0x7f87580b3120 msgr2=0x7f8758164720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:51.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.148+0000 7f875d000640 1 -- 172.21.15.133:0/2217641151 shutdown_connections 2024-03-07T06:51:51.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.148+0000 7f875d000640 1 -- 172.21.15.133:0/2217641151 wait complete. 2024-03-07T06:51:51.166 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:51:12.858333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:51:13.615348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:51:51.167 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:51:51.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.383+0000 7f31f4800640 1 Processor -- start 2024-03-07T06:51:51.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.384+0000 7f31f4800640 1 -- start start 2024-03-07T06:51:51.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.385+0000 7f31f4800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f31ec0b7200 0x7f31ec071990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:51.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.385+0000 7f31f4800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31ec071ff0 con 0x7f31ec15e350 2024-03-07T06:51:51.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.385+0000 7f31f4800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31ec0d23b0 con 0x7f31ec1625c0 2024-03-07T06:51:51.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.385+0000 7f31f4800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31ec0d2160 con 0x7f31ec0b7200 2024-03-07T06:51:51.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.385+0000 7f31f1e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f31ec15e350 0x7f31ec161eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57006/0 (socket says 172.21.15.133:57006) 2024-03-07T06:51:51.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.385+0000 7f31f1e00640 1 -- 172.21.15.133:0/2958761320 learned_addr learned my addr 172.21.15.133:0/2958761320 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:51.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 -- 172.21.15.133:0/2958761320 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158024838 0 0) 0x7f31ec071ff0 con 0x7f31ec15e350 2024-03-07T06:51:51.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 -- 172.21.15.133:0/2958761320 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d4003520 con 0x7f31ec15e350 2024-03-07T06:51:51.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f2800640 1 --2- 172.21.15.133:0/2958761320 >> v2:172.21.15.161:3300/0 conn(0x7f31ec0b7200 0x7f31ec071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:51.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 -- 172.21.15.133:0/2958761320 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 569568647 0 0) 0x7f31d4003520 con 0x7f31ec15e350 2024-03-07T06:51:51.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 -- 172.21.15.133:0/2958761320 >> v2:172.21.15.161:3300/0 conn(0x7f31ec0b7200 msgr2=0x7f31ec071990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:51.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 --2- 172.21.15.133:0/2958761320 >> v2:172.21.15.161:3300/0 conn(0x7f31ec0b7200 0x7f31ec071990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 -- 172.21.15.133:0/2958761320 >> v1:172.21.15.161:6789/0 conn(0x7f31ec1625c0 legacy=0x7f31ec071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:51.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 -- 172.21.15.133:0/2958761320 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec0d2690 con 0x7f31ec15e350 2024-03-07T06:51:51.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 -- 172.21.15.133:0/2958761320 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f31e0001670 con 0x7f31ec15e350 2024-03-07T06:51:51.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 -- 172.21.15.133:0/2958761320 >> v1:172.21.15.133:6789/0 conn(0x7f31ec15e350 legacy=0x7f31ec161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:51.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 --2- 172.21.15.133:0/2958761320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31d4003a00 0x7f31d4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:51.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 --2- 172.21.15.133:0/2958761320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31d4004800 0x7f31d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:51.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 --2- 172.21.15.133:0/2958761320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d40052e0 0x7f31d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:51.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 -- 172.21.15.133:0/2958761320 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec0d2160 con 0x7f31d4003a00 2024-03-07T06:51:51.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 -- 172.21.15.133:0/2958761320 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d400c200 con 0x7f31d40052e0 2024-03-07T06:51:51.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f0a00640 1 -- 172.21.15.133:0/2958761320 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d400c400 con 0x7f31d4004800 2024-03-07T06:51:51.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.386+0000 7f31f1e00640 1 --2- 172.21.15.133:0/2958761320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31d4003a00 0x7f31d4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:51.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.387+0000 7f31f2800640 1 --2- 172.21.15.133:0/2958761320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d40052e0 0x7f31d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:51.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.387+0000 7f31f1400640 1 --2- 172.21.15.133:0/2958761320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31d4004800 0x7f31d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:51.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.387+0000 7f31f1e00640 1 -- 172.21.15.133:0/2958761320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31d4004800 msgr2=0x7f31d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:51.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.387+0000 7f31f1e00640 1 --2- 172.21.15.133:0/2958761320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31d4004800 0x7f31d4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.387+0000 7f31f1e00640 1 -- 172.21.15.133:0/2958761320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d40052e0 msgr2=0x7f31d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:51.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.387+0000 7f31f1e00640 1 --2- 172.21.15.133:0/2958761320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d40052e0 0x7f31d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.387+0000 7f31f1e00640 1 -- 172.21.15.133:0/2958761320 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d4003520 con 0x7f31d4003a00 2024-03-07T06:51:51.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.387+0000 7f31f2800640 1 --2- 172.21.15.133:0/2958761320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31d40052e0 0x7f31d400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:51.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.387+0000 7f31f2800640 1 -- 172.21.15.133:0/2958761320 reap_dead start 2024-03-07T06:51:51.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.388+0000 7f31f1e00640 1 --2- 172.21.15.133:0/2958761320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31d4003a00 0x7f31d4003e20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f31e000c190 tx=0x7f31d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c43a82aaac77d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:51.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.390+0000 7f31f0a00640 1 -- 172.21.15.133:0/2958761320 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0007c60 con 0x7f31d4003a00 2024-03-07T06:51:51.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.390+0000 7f31f0a00640 1 -- 172.21.15.133:0/2958761320 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e0030070 con 0x7f31d4003a00 2024-03-07T06:51:51.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.390+0000 7f31f0a00640 1 -- 172.21.15.133:0/2958761320 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0005d90 con 0x7f31d4003a00 2024-03-07T06:51:51.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.390+0000 7f31f4800640 1 -- 172.21.15.133:0/2958761320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31d4003a00 msgr2=0x7f31d4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:51.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.390+0000 7f31f4800640 1 --2- 172.21.15.133:0/2958761320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31d4003a00 0x7f31d4003e20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f31e000c190 tx=0x7f31d4005830 comp rx=0 tx=0).stop 2024-03-07T06:51:51.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.391+0000 7f31f4800640 1 -- 172.21.15.133:0/2958761320 shutdown_connections 2024-03-07T06:51:51.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.391+0000 7f31f4800640 1 --2- 172.21.15.133:0/2958761320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31d4003a00 0x7f31d4003e20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.391+0000 7f31f4800640 1 -- 172.21.15.133:0/2958761320 >> 172.21.15.133:0/2958761320 conn(0x7f31ec0d0e70 msgr2=0x7f31ec0d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:51.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.391+0000 7f31f4800640 1 -- 172.21.15.133:0/2958761320 shutdown_connections 2024-03-07T06:51:51.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.391+0000 7f31f4800640 1 -- 172.21.15.133:0/2958761320 wait complete. 2024-03-07T06:51:51.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.392+0000 7f31f4800640 1 Processor -- start 2024-03-07T06:51:51.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.393+0000 7f31f4800640 1 -- start start 2024-03-07T06:51:51.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.393+0000 7f31f4800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31d4003a00 0x7f31ec0ca520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:51.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.393+0000 7f31f4800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0cf250 0x7f31ec0c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:51.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.393+0000 7f31f4800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31ec160d40 0x7f31ec1611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:51.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.393+0000 7f31f4800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec161eb0 con 0x7f31d4003a00 2024-03-07T06:51:51.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.393+0000 7f31f4800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec15e730 con 0x7f31ec160d40 2024-03-07T06:51:51.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.393+0000 7f31f4800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ec162030 con 0x7f31ec0cf250 2024-03-07T06:51:51.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.394+0000 7f31f1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31d4003a00 0x7f31ec0ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:51.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.394+0000 7f31f1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31d4003a00 0x7f31ec0ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37032/0 (socket says 172.21.15.133:37032) 2024-03-07T06:51:51.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.394+0000 7f31f1e00640 1 -- 172.21.15.133:0/445741497 learned_addr learned my addr 172.21.15.133:0/445741497 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:51.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.394+0000 7f31f1400640 1 --2- 172.21.15.133:0/445741497 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0cf250 0x7f31ec0c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:51.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.394+0000 7f31f2800640 1 --2- 172.21.15.133:0/445741497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31ec160d40 0x7f31ec1611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:51.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.394+0000 7f31f1e00640 1 -- 172.21.15.133:0/445741497 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0cf250 msgr2=0x7f31ec0c9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:51.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.394+0000 7f31f1e00640 1 --2- 172.21.15.133:0/445741497 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0cf250 0x7f31ec0c9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.394+0000 7f31f1e00640 1 -- 172.21.15.133:0/445741497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31ec160d40 msgr2=0x7f31ec1611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:51.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.394+0000 7f31f1e00640 1 --2- 172.21.15.133:0/445741497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31ec160d40 0x7f31ec1611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.394+0000 7f31f1e00640 1 -- 172.21.15.133:0/445741497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec157d90 con 0x7f31d4003a00 2024-03-07T06:51:51.395 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.394+0000 7f31f1e00640 1 --2- 172.21.15.133:0/445741497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31d4003a00 0x7f31ec0ca520 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f31e000c2b0 tx=0x7f31e0001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:51.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.395+0000 7f31e6000640 1 -- 172.21.15.133:0/445741497 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0045070 con 0x7f31d4003a00 2024-03-07T06:51:51.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.395+0000 7f31e6000640 1 -- 172.21.15.133:0/445741497 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e00047e0 con 0x7f31d4003a00 2024-03-07T06:51:51.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.395+0000 7f31e6000640 1 -- 172.21.15.133:0/445741497 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e0004bd0 con 0x7f31d4003a00 2024-03-07T06:51:51.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.395+0000 7f31f4800640 1 -- 172.21.15.133:0/445741497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31ec158070 con 0x7f31d4003a00 2024-03-07T06:51:51.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.395+0000 7f31f4800640 1 -- 172.21.15.133:0/445741497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ec158630 con 0x7f31d4003a00 2024-03-07T06:51:51.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.396+0000 7f31f4800640 1 -- 172.21.15.133:0/445741497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31ec062c90 con 0x7f31d4003a00 2024-03-07T06:51:51.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.397+0000 7f31e6000640 1 -- 172.21.15.133:0/445741497 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f31e0007070 con 0x7f31d4003a00 2024-03-07T06:51:51.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.398+0000 7f31e6000640 1 --2- 172.21.15.133:0/445741497 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f31d006b4d0 0x7f31d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:51.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.398+0000 7f31e6000640 1 -- 172.21.15.133:0/445741497 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31e00aaf20 con 0x7f31d4003a00 2024-03-07T06:51:51.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.399+0000 7f31f1400640 1 --2- 172.21.15.133:0/445741497 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f31d006b4d0 0x7f31d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:51.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.400+0000 7f31f1400640 1 --2- 172.21.15.133:0/445741497 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f31d006b4d0 0x7f31d006d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f31ec15d540 tx=0x7f31dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:51.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.403+0000 7f31e6000640 1 -- 172.21.15.133:0/445741497 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f31e0079d90 con 0x7f31d4003a00 2024-03-07T06:51:51.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.598+0000 7f31f4800640 1 -- 172.21.15.133:0/445741497 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31ec0cfa70 con 0x7f31d4003a00 2024-03-07T06:51:51.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.599+0000 7f31e6000640 1 -- 172.21.15.133:0/445741497 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31e0079690 con 0x7f31d4003a00 2024-03-07T06:51:51.599 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:51:51.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.601+0000 7f31f4800640 1 -- 172.21.15.133:0/445741497 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f31d006b4d0 msgr2=0x7f31d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:51.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.601+0000 7f31f4800640 1 --2- 172.21.15.133:0/445741497 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f31d006b4d0 0x7f31d006d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f31ec15d540 tx=0x7f31dc009040 comp rx=0 tx=0).stop 2024-03-07T06:51:51.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.601+0000 7f31f4800640 1 -- 172.21.15.133:0/445741497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31d4003a00 msgr2=0x7f31ec0ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:51.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.601+0000 7f31f4800640 1 --2- 172.21.15.133:0/445741497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31d4003a00 0x7f31ec0ca520 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f31e000c2b0 tx=0x7f31e0001970 comp rx=0 tx=0).stop 2024-03-07T06:51:51.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.601+0000 7f31f4800640 1 -- 172.21.15.133:0/445741497 shutdown_connections 2024-03-07T06:51:51.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.601+0000 7f31f4800640 1 --2- 172.21.15.133:0/445741497 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f31d006b4d0 0x7f31d006d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.601+0000 7f31f4800640 1 --2- 172.21.15.133:0/445741497 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f31ec160d40 0x7f31ec1611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.601+0000 7f31f4800640 1 --2- 172.21.15.133:0/445741497 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31ec0cf250 0x7f31ec0c9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.601+0000 7f31f4800640 1 --2- 172.21.15.133:0/445741497 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31d4003a00 0x7f31ec0ca520 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.601+0000 7f31f4800640 1 -- 172.21.15.133:0/445741497 >> 172.21.15.133:0/445741497 conn(0x7f31ec0d0e70 msgr2=0x7f31ec0b8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:51.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.602+0000 7f31f4800640 1 -- 172.21.15.133:0/445741497 shutdown_connections 2024-03-07T06:51:51.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.602+0000 7f31f4800640 1 -- 172.21.15.133:0/445741497 wait complete. 2024-03-07T06:51:51.621 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:51:51.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.833+0000 7f40a6c00640 1 Processor -- start 2024-03-07T06:51:51.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.833+0000 7f40a6c00640 1 -- start start 2024-03-07T06:51:51.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.834+0000 7f40a6c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f40a006eb00 0x7f40a00b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:51.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.834+0000 7f40a6c00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a006f450 con 0x7f40a00cf620 2024-03-07T06:51:51.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.835+0000 7f40a6c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a006f170 con 0x7f40a00aec80 2024-03-07T06:51:51.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.835+0000 7f40a6c00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a006ef10 con 0x7f40a006eb00 2024-03-07T06:51:51.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.835+0000 7f409fe00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f40a00cf620 0x7f40a00ae570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57012/0 (socket says 172.21.15.133:57012) 2024-03-07T06:51:51.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.835+0000 7f409fe00640 1 -- 172.21.15.133:0/4178953898 learned_addr learned my addr 172.21.15.133:0/4178953898 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:51.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.835+0000 7f40a4c00640 1 --2- 172.21.15.133:0/4178953898 >> v2:172.21.15.133:3300/0 conn(0x7f40a006eb00 0x7f40a00b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:51.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.835+0000 7f409f400640 1 -- 172.21.15.133:0/4178953898 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737874140 0 0) 0x7f40a006f450 con 0x7f40a00cf620 2024-03-07T06:51:51.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.835+0000 7f40a4c00640 1 -- 172.21.15.133:0/4178953898 >> v1:172.21.15.161:6789/0 conn(0x7f40a00aec80 legacy=0x7f40a006e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:51.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.835+0000 7f40a4c00640 1 -- 172.21.15.133:0/4178953898 >> v1:172.21.15.133:6789/0 conn(0x7f40a00cf620 legacy=0x7f40a00ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:51.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.835+0000 7f40a4c00640 1 -- 172.21.15.133:0/4178953898 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a0071ad0 con 0x7f40a006eb00 2024-03-07T06:51:51.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.837+0000 7f40a4c00640 1 --2- 172.21.15.133:0/4178953898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40a006eb00 0x7f40a00b2e50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4098009d90 tx=0x7f409802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250fb3c33f8768ae server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:51.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.837+0000 7f409f400640 1 -- 172.21.15.133:0/4178953898 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098039070 con 0x7f40a006eb00 2024-03-07T06:51:51.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.837+0000 7f409f400640 1 -- 172.21.15.133:0/4178953898 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409802cc60 con 0x7f40a006eb00 2024-03-07T06:51:51.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.837+0000 7f409f400640 1 -- 172.21.15.133:0/4178953898 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098034660 con 0x7f40a006eb00 2024-03-07T06:51:51.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.837+0000 7f40a6c00640 1 -- 172.21.15.133:0/4178953898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40a006eb00 msgr2=0x7f40a00b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:51.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.837+0000 7f40a6c00640 1 --2- 172.21.15.133:0/4178953898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40a006eb00 0x7f40a00b2e50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4098009d90 tx=0x7f409802beb0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.838+0000 7f40a6c00640 1 -- 172.21.15.133:0/4178953898 shutdown_connections 2024-03-07T06:51:51.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.838+0000 7f40a6c00640 1 --2- 172.21.15.133:0/4178953898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40a006eb00 0x7f40a00b2e50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.838+0000 7f40a6c00640 1 -- 172.21.15.133:0/4178953898 >> 172.21.15.133:0/4178953898 conn(0x7f40a00b1d30 msgr2=0x7f40a00b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:51.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.838+0000 7f40a6c00640 1 -- 172.21.15.133:0/4178953898 shutdown_connections 2024-03-07T06:51:51.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.838+0000 7f40a6c00640 1 -- 172.21.15.133:0/4178953898 wait complete. 2024-03-07T06:51:51.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.839+0000 7f40a6c00640 1 Processor -- start 2024-03-07T06:51:51.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.839+0000 7f40a6c00640 1 -- start start 2024-03-07T06:51:51.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.840+0000 7f40a6c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40a006eb00 0x7f40a01477e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:51.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.840+0000 7f40a6c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40a00aec80 0x7f40a014f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:51.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.840+0000 7f40a6c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40a00cf620 0x7f40a01560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:51.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.840+0000 7f40a6c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00725b0 con 0x7f40a00cf620 2024-03-07T06:51:51.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.840+0000 7f40a6c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a0072090 con 0x7f40a00aec80 2024-03-07T06:51:51.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.840+0000 7f40a6c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a0072730 con 0x7f40a006eb00 2024-03-07T06:51:51.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.841+0000 7f40a4c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40a00cf620 0x7f40a01560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:51.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.841+0000 7f40a4c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40a00cf620 0x7f40a01560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37044/0 (socket says 172.21.15.133:37044) 2024-03-07T06:51:51.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.841+0000 7f40a4c00640 1 -- 172.21.15.133:0/532872521 learned_addr learned my addr 172.21.15.133:0/532872521 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:51.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.841+0000 7f4097e00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40a00aec80 0x7f40a014f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:51.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.841+0000 7f409fe00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40a006eb00 0x7f40a01477e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:51.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.841+0000 7f40a4c00640 1 -- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40a006eb00 msgr2=0x7f40a01477e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:51.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.841+0000 7f40a4c00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40a006eb00 0x7f40a01477e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.841+0000 7f40a4c00640 1 -- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40a00aec80 msgr2=0x7f40a014f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:51.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.841+0000 7f40a4c00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40a00aec80 0x7f40a014f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:51.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.841+0000 7f40a4c00640 1 -- 172.21.15.133:0/532872521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a0152180 con 0x7f40a00cf620 2024-03-07T06:51:51.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.841+0000 7f409fe00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40a006eb00 0x7f40a01477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:51.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.841+0000 7f40a4c00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40a00cf620 0x7f40a01560c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4098037040 tx=0x7f4098004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:51.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.842+0000 7f409cc00640 1 -- 172.21.15.133:0/532872521 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098039050 con 0x7f40a00cf620 2024-03-07T06:51:51.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.842+0000 7f409cc00640 1 -- 172.21.15.133:0/532872521 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409803a040 con 0x7f40a00cf620 2024-03-07T06:51:51.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.842+0000 7f40a6c00640 1 -- 172.21.15.133:0/532872521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a0152460 con 0x7f40a00cf620 2024-03-07T06:51:51.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.842+0000 7f409cc00640 1 -- 172.21.15.133:0/532872521 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409803e610 con 0x7f40a00cf620 2024-03-07T06:51:51.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.842+0000 7f40a6c00640 1 -- 172.21.15.133:0/532872521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a0152a20 con 0x7f40a00cf620 2024-03-07T06:51:51.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.843+0000 7f40a6c00640 1 -- 172.21.15.133:0/532872521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a0062c90 con 0x7f40a00cf620 2024-03-07T06:51:51.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.844+0000 7f409cc00640 1 -- 172.21.15.133:0/532872521 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4098047020 con 0x7f40a00cf620 2024-03-07T06:51:51.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.845+0000 7f409cc00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f407c06b500 0x7f407c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:51.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.845+0000 7f409cc00640 1 -- 172.21.15.133:0/532872521 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40980ac530 con 0x7f40a00cf620 2024-03-07T06:51:51.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.845+0000 7f409fe00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f407c06b500 0x7f407c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:51.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.846+0000 7f409fe00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f407c06b500 0x7f407c06d9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4090000f60 tx=0x7f4090009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:51.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:51.849+0000 7f409cc00640 1 -- 172.21.15.133:0/532872521 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f409804e020 con 0x7f40a00cf620 2024-03-07T06:51:52.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.043+0000 7f40a6c00640 1 -- 172.21.15.133:0/532872521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f40a00b2600 con 0x7f40a00cf620 2024-03-07T06:51:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.044+0000 7f409cc00640 1 -- 172.21.15.133:0/532872521 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f409807b460 con 0x7f40a00cf620 2024-03-07T06:51:52.044 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:51:52.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.046+0000 7f40a6c00640 1 -- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f407c06b500 msgr2=0x7f407c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.046+0000 7f40a6c00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f407c06b500 0x7f407c06d9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4090000f60 tx=0x7f4090009040 comp rx=0 tx=0).stop 2024-03-07T06:51:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.046+0000 7f40a6c00640 1 -- 172.21.15.133:0/532872521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40a00cf620 msgr2=0x7f40a01560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.046+0000 7f40a6c00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40a00cf620 0x7f40a01560c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4098037040 tx=0x7f4098004a40 comp rx=0 tx=0).stop 2024-03-07T06:51:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.047+0000 7f40a6c00640 1 -- 172.21.15.133:0/532872521 shutdown_connections 2024-03-07T06:51:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.047+0000 7f40a6c00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40a00cf620 0x7f40a01560c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.047+0000 7f40a6c00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f407c06b500 0x7f407c06d9c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.047+0000 7f40a6c00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f40a00aec80 0x7f40a014f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.047+0000 7f40a6c00640 1 --2- 172.21.15.133:0/532872521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f40a006eb00 0x7f40a01477e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.047+0000 7f40a6c00640 1 -- 172.21.15.133:0/532872521 >> 172.21.15.133:0/532872521 conn(0x7f40a00b1d30 msgr2=0x7f40a01538c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.047+0000 7f40a6c00640 1 -- 172.21.15.133:0/532872521 shutdown_connections 2024-03-07T06:51:52.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.047+0000 7f40a6c00640 1 -- 172.21.15.133:0/532872521 wait complete. 2024-03-07T06:51:52.066 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:51:52.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.283+0000 7f0853c00640 1 Processor -- start 2024-03-07T06:51:52.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.283+0000 7f0853c00640 1 -- start start 2024-03-07T06:51:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.284+0000 7f0853c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f084c145670 0x7f084c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.285+0000 7f0853c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f084c0cdbf0 0x7f084c0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.285+0000 7f0853c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f084c06e150 con 0x7f084c15cf90 2024-03-07T06:51:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.285+0000 7f0853c00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f084c06ed60 con 0x7f084c145670 2024-03-07T06:51:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.285+0000 7f0853c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f084c06e430 con 0x7f084c0cdbf0 2024-03-07T06:51:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.285+0000 7f0851200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f084c145670 0x7f084c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.285+0000 7f0851200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f084c145670 0x7f084c0cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37052/0 (socket says 172.21.15.133:37052) 2024-03-07T06:51:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.285+0000 7f0851200640 1 -- 172.21.15.133:0/1611126023 learned_addr learned my addr 172.21.15.133:0/1611126023 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.285+0000 7f0851c00640 1 --2- 172.21.15.133:0/1611126023 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f084c0cdbf0 0x7f084c0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:52.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.285+0000 7f0851200640 1 -- 172.21.15.133:0/1611126023 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f084c0cdbf0 msgr2=0x7f084c0719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:52.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.285+0000 7f0851200640 1 --2- 172.21.15.133:0/1611126023 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f084c0cdbf0 0x7f084c0719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.286+0000 7f0851200640 1 -- 172.21.15.133:0/1611126023 >> v1:172.21.15.161:6789/0 conn(0x7f084c15cf90 legacy=0x7f084c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:52.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.286+0000 7f0851200640 1 -- 172.21.15.133:0/1611126023 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c06e710 con 0x7f084c145670 2024-03-07T06:51:52.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.287+0000 7f0851200640 1 --2- 172.21.15.133:0/1611126023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084c145670 0x7f084c0cf610 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f08400098a0 tx=0x7f084002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ba2fede25c750a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:52.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.288+0000 7f084b400640 1 -- 172.21.15.133:0/1611126023 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840035020 con 0x7f084c145670 2024-03-07T06:51:52.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.288+0000 7f084b400640 1 -- 172.21.15.133:0/1611126023 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0840031640 con 0x7f084c145670 2024-03-07T06:51:52.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.288+0000 7f084b400640 1 -- 172.21.15.133:0/1611126023 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084002d040 con 0x7f084c145670 2024-03-07T06:51:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.288+0000 7f0853c00640 1 -- 172.21.15.133:0/1611126023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084c145670 msgr2=0x7f084c0cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.288+0000 7f0853c00640 1 --2- 172.21.15.133:0/1611126023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084c145670 0x7f084c0cf610 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f08400098a0 tx=0x7f084002b8f0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.288+0000 7f0853c00640 1 -- 172.21.15.133:0/1611126023 shutdown_connections 2024-03-07T06:51:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.288+0000 7f0853c00640 1 --2- 172.21.15.133:0/1611126023 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f084c0cdbf0 0x7f084c0719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.288+0000 7f0853c00640 1 --2- 172.21.15.133:0/1611126023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084c145670 0x7f084c0cf610 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.288+0000 7f0853c00640 1 -- 172.21.15.133:0/1611126023 >> 172.21.15.133:0/1611126023 conn(0x7f084c0b1d30 msgr2=0x7f084c0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:52.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.288+0000 7f0853c00640 1 -- 172.21.15.133:0/1611126023 shutdown_connections 2024-03-07T06:51:52.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.288+0000 7f0853c00640 1 -- 172.21.15.133:0/1611126023 wait complete. 2024-03-07T06:51:52.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.290+0000 7f0853c00640 1 Processor -- start 2024-03-07T06:51:52.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.290+0000 7f0853c00640 1 -- start start 2024-03-07T06:51:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.291+0000 7f0853c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f084c0cdbf0 0x7f084c15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.291+0000 7f0853c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084c145670 0x7f084c13fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:52.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.291+0000 7f0853c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f084c15cf90 0x7f084c0b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.291+0000 7f0853c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c145a50 con 0x7f084c145670 2024-03-07T06:51:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.291+0000 7f0853c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c15d420 con 0x7f084c15cf90 2024-03-07T06:51:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.291+0000 7f0853c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f084c145bd0 con 0x7f084c0cdbf0 2024-03-07T06:51:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.292+0000 7f084be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084c145670 0x7f084c13fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.292+0000 7f084be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084c145670 0x7f084c13fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37064/0 (socket says 172.21.15.133:37064) 2024-03-07T06:51:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.292+0000 7f084be00640 1 -- 172.21.15.133:0/2420735919 learned_addr learned my addr 172.21.15.133:0/2420735919 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.292+0000 7f0851200640 1 --2- 172.21.15.133:0/2420735919 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f084c0cdbf0 0x7f084c15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.292+0000 7f0851c00640 1 --2- 172.21.15.133:0/2420735919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f084c15cf90 0x7f084c0b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.292+0000 7f084be00640 1 -- 172.21.15.133:0/2420735919 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f084c0cdbf0 msgr2=0x7f084c15b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.292+0000 7f084be00640 1 --2- 172.21.15.133:0/2420735919 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f084c0cdbf0 0x7f084c15b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.292+0000 7f084be00640 1 -- 172.21.15.133:0/2420735919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f084c15cf90 msgr2=0x7f084c0b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.292+0000 7f084be00640 1 --2- 172.21.15.133:0/2420735919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f084c15cf90 0x7f084c0b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.292+0000 7f084be00640 1 -- 172.21.15.133:0/2420735919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c13e820 con 0x7f084c145670 2024-03-07T06:51:52.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.292+0000 7f084be00640 1 --2- 172.21.15.133:0/2420735919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084c145670 0x7f084c13fba0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f083800c400 tx=0x7f083800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.293+0000 7f0848c00640 1 -- 172.21.15.133:0/2420735919 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08380052d0 con 0x7f084c145670 2024-03-07T06:51:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.293+0000 7f0848c00640 1 -- 172.21.15.133:0/2420735919 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0838005490 con 0x7f084c145670 2024-03-07T06:51:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.293+0000 7f0853c00640 1 -- 172.21.15.133:0/2420735919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084c13eb00 con 0x7f084c145670 2024-03-07T06:51:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.293+0000 7f0848c00640 1 -- 172.21.15.133:0/2420735919 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0838007260 con 0x7f084c145670 2024-03-07T06:51:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.293+0000 7f0853c00640 1 -- 172.21.15.133:0/2420735919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084c13f0c0 con 0x7f084c145670 2024-03-07T06:51:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.294+0000 7f0853c00640 1 -- 172.21.15.133:0/2420735919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084c062c90 con 0x7f084c145670 2024-03-07T06:51:52.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.295+0000 7f0848c00640 1 -- 172.21.15.133:0/2420735919 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0838007400 con 0x7f084c145670 2024-03-07T06:51:52.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.296+0000 7f0848c00640 1 --2- 172.21.15.133:0/2420735919 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f082c06b4d0 0x7f082c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:52.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.296+0000 7f0848c00640 1 -- 172.21.15.133:0/2420735919 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08380897b0 con 0x7f084c145670 2024-03-07T06:51:52.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.296+0000 7f0851200640 1 --2- 172.21.15.133:0/2420735919 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f082c06b4d0 0x7f082c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:52.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.297+0000 7f0851200640 1 --2- 172.21.15.133:0/2420735919 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f082c06b4d0 0x7f082c06d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0840036000 tx=0x7f084003c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:52.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.300+0000 7f0848c00640 1 -- 172.21.15.133:0/2420735919 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0838058590 con 0x7f084c145670 2024-03-07T06:51:52.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.496+0000 7f0853c00640 1 -- 172.21.15.133:0/2420735919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f084c0b21b0 con 0x7f084c145670 2024-03-07T06:51:52.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.497+0000 7f0848c00640 1 -- 172.21.15.133:0/2420735919 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0838057e90 con 0x7f084c145670 2024-03-07T06:51:52.497 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:51:52.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.499+0000 7f0853c00640 1 -- 172.21.15.133:0/2420735919 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f082c06b4d0 msgr2=0x7f082c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:52.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.499+0000 7f0853c00640 1 --2- 172.21.15.133:0/2420735919 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f082c06b4d0 0x7f082c06d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0840036000 tx=0x7f084003c040 comp rx=0 tx=0).stop 2024-03-07T06:51:52.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.499+0000 7f0853c00640 1 -- 172.21.15.133:0/2420735919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084c145670 msgr2=0x7f084c13fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:52.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.499+0000 7f0853c00640 1 --2- 172.21.15.133:0/2420735919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084c145670 0x7f084c13fba0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f083800c400 tx=0x7f083800c8c0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.500+0000 7f0853c00640 1 -- 172.21.15.133:0/2420735919 shutdown_connections 2024-03-07T06:51:52.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.500+0000 7f0853c00640 1 --2- 172.21.15.133:0/2420735919 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f082c06b4d0 0x7f082c06d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.500+0000 7f0853c00640 1 --2- 172.21.15.133:0/2420735919 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f084c15cf90 0x7f084c0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.500+0000 7f0853c00640 1 --2- 172.21.15.133:0/2420735919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f084c145670 0x7f084c13fba0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.500+0000 7f0853c00640 1 --2- 172.21.15.133:0/2420735919 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f084c0cdbf0 0x7f084c15b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.500+0000 7f0853c00640 1 -- 172.21.15.133:0/2420735919 >> 172.21.15.133:0/2420735919 conn(0x7f084c0b1d30 msgr2=0x7f084c071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:52.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.500+0000 7f0853c00640 1 -- 172.21.15.133:0/2420735919 shutdown_connections 2024-03-07T06:51:52.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.500+0000 7f0853c00640 1 -- 172.21.15.133:0/2420735919 wait complete. 2024-03-07T06:51:52.519 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:52.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.723+0000 7f9d83e00640 1 Processor -- start 2024-03-07T06:51:52.723 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.723+0000 7f9d83e00640 1 -- start start 2024-03-07T06:51:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.725+0000 7f9d83e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9d7c165ed0 0x7f9d7c06ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.725+0000 7f9d83e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9d7c0b3c10 0x7f9d7c071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.725+0000 7f9d83e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d7c071b50 0x7f9d7c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.725+0000 7f9d83e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d7c0734a0 con 0x7f9d7c165ed0 2024-03-07T06:51:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.725+0000 7f9d83e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d7c071fe0 con 0x7f9d7c0b3c10 2024-03-07T06:51:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.725+0000 7f9d83e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c0ce540 con 0x7f9d7c071b50 2024-03-07T06:51:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.725+0000 7f9d81400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9d7c165ed0 0x7f9d7c06ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.726+0000 7f9d81400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9d7c165ed0 0x7f9d7c06ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37078/0 (socket says 172.21.15.133:37078) 2024-03-07T06:51:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.726+0000 7f9d81400640 1 -- 172.21.15.133:0/929350270 learned_addr learned my addr 172.21.15.133:0/929350270 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.726+0000 7f9d81e00640 1 --2- 172.21.15.133:0/929350270 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d7c071b50 0x7f9d7c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.726+0000 7f9d80a00640 1 --2- 172.21.15.133:0/929350270 >> v2:172.21.15.161:3300/0 conn(0x7f9d7c0b3c10 0x7f9d7c071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.726+0000 7f9d81400640 1 -- 172.21.15.133:0/929350270 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d7c071b50 msgr2=0x7f9d7c06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:52.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.726+0000 7f9d81400640 1 --2- 172.21.15.133:0/929350270 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d7c071b50 0x7f9d7c06d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.726+0000 7f9d81400640 1 -- 172.21.15.133:0/929350270 >> v2:172.21.15.161:3300/0 conn(0x7f9d7c0b3c10 msgr2=0x7f9d7c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:52.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.726+0000 7f9d81400640 1 --2- 172.21.15.133:0/929350270 >> v2:172.21.15.161:3300/0 conn(0x7f9d7c0b3c10 0x7f9d7c071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.726+0000 7f9d81400640 1 -- 172.21.15.133:0/929350270 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c06f200 con 0x7f9d7c165ed0 2024-03-07T06:51:52.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.726+0000 7f9d80a00640 1 --2- 172.21.15.133:0/929350270 >> v2:172.21.15.161:3300/0 conn(0x7f9d7c0b3c10 0x7f9d7c071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:52.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.728+0000 7f9d81400640 1 --2- 172.21.15.133:0/929350270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d7c165ed0 0x7f9d7c06ecc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c073c90 tx=0x7f9d7002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e6bd20b17ae7af server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:52.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.728+0000 7f9d7be00640 1 -- 172.21.15.133:0/929350270 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70035020 con 0x7f9d7c165ed0 2024-03-07T06:51:52.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.728+0000 7f9d7be00640 1 -- 172.21.15.133:0/929350270 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d70031870 con 0x7f9d7c165ed0 2024-03-07T06:51:52.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.728+0000 7f9d7be00640 1 -- 172.21.15.133:0/929350270 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70031c40 con 0x7f9d7c165ed0 2024-03-07T06:51:52.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.729+0000 7f9d83e00640 1 -- 172.21.15.133:0/929350270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d7c165ed0 msgr2=0x7f9d7c06ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:52.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.729+0000 7f9d83e00640 1 --2- 172.21.15.133:0/929350270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d7c165ed0 0x7f9d7c06ecc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c073c90 tx=0x7f9d7002b910 comp rx=0 tx=0).stop 2024-03-07T06:51:52.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.729+0000 7f9d83e00640 1 -- 172.21.15.133:0/929350270 shutdown_connections 2024-03-07T06:51:52.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.729+0000 7f9d83e00640 1 --2- 172.21.15.133:0/929350270 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d7c071b50 0x7f9d7c06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.729+0000 7f9d83e00640 1 --2- 172.21.15.133:0/929350270 >> v2:172.21.15.161:3300/0 conn(0x7f9d7c0b3c10 0x7f9d7c071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.729+0000 7f9d83e00640 1 --2- 172.21.15.133:0/929350270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d7c165ed0 0x7f9d7c06ecc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.729+0000 7f9d83e00640 1 -- 172.21.15.133:0/929350270 >> 172.21.15.133:0/929350270 conn(0x7f9d7c0b3120 msgr2=0x7f9d7c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:52.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.729+0000 7f9d83e00640 1 -- 172.21.15.133:0/929350270 shutdown_connections 2024-03-07T06:51:52.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.729+0000 7f9d83e00640 1 -- 172.21.15.133:0/929350270 wait complete. 2024-03-07T06:51:52.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.731+0000 7f9d83e00640 1 Processor -- start 2024-03-07T06:51:52.730 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.731+0000 7f9d83e00640 1 -- start start 2024-03-07T06:51:52.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.732+0000 7f9d83e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d7c071b50 0x7f9d7c14f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:52.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.732+0000 7f9d83e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d7c0b3c10 0x7f9d7c160860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:52.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.732+0000 7f9d83e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d7c165ed0 0x7f9d7c160da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:52.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.732+0000 7f9d83e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c072270 con 0x7f9d7c0b3c10 2024-03-07T06:51:52.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.732+0000 7f9d83e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c0ce260 con 0x7f9d7c071b50 2024-03-07T06:51:52.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.732+0000 7f9d83e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7c072550 con 0x7f9d7c165ed0 2024-03-07T06:51:52.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.733+0000 7f9d80a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d7c0b3c10 0x7f9d7c160860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:52.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.733+0000 7f9d80a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d7c0b3c10 0x7f9d7c160860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37092/0 (socket says 172.21.15.133:37092) 2024-03-07T06:51:52.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.733+0000 7f9d80a00640 1 -- 172.21.15.133:0/1869640269 learned_addr learned my addr 172.21.15.133:0/1869640269 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:52.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.733+0000 7f9d81e00640 1 --2- 172.21.15.133:0/1869640269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d7c165ed0 0x7f9d7c160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:52.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.733+0000 7f9d81400640 1 --2- 172.21.15.133:0/1869640269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d7c071b50 0x7f9d7c14f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:52.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.733+0000 7f9d80a00640 1 -- 172.21.15.133:0/1869640269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d7c165ed0 msgr2=0x7f9d7c160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:52.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.733+0000 7f9d80a00640 1 --2- 172.21.15.133:0/1869640269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d7c165ed0 0x7f9d7c160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.733+0000 7f9d80a00640 1 -- 172.21.15.133:0/1869640269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d7c071b50 msgr2=0x7f9d7c14f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:52.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.733+0000 7f9d80a00640 1 --2- 172.21.15.133:0/1869640269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d7c071b50 0x7f9d7c14f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.733+0000 7f9d80a00640 1 -- 172.21.15.133:0/1869640269 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c15df80 con 0x7f9d7c0b3c10 2024-03-07T06:51:52.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.734+0000 7f9d80a00640 1 --2- 172.21.15.133:0/1869640269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d7c0b3c10 0x7f9d7c160860 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800c7f0 tx=0x7f9d6800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:52.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.734+0000 7f9d79600640 1 -- 172.21.15.133:0/1869640269 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68015070 con 0x7f9d7c0b3c10 2024-03-07T06:51:52.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.734+0000 7f9d79600640 1 -- 172.21.15.133:0/1869640269 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d68007a30 con 0x7f9d7c0b3c10 2024-03-07T06:51:52.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.734+0000 7f9d79600640 1 -- 172.21.15.133:0/1869640269 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68007e00 con 0x7f9d7c0b3c10 2024-03-07T06:51:52.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.734+0000 7f9d83e00640 1 -- 172.21.15.133:0/1869640269 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7c15e260 con 0x7f9d7c0b3c10 2024-03-07T06:51:52.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.734+0000 7f9d83e00640 1 -- 172.21.15.133:0/1869640269 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7c15e820 con 0x7f9d7c0b3c10 2024-03-07T06:51:52.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.735+0000 7f9d83e00640 1 -- 172.21.15.133:0/1869640269 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7c062c90 con 0x7f9d7c0b3c10 2024-03-07T06:51:52.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.736+0000 7f9d79600640 1 -- 172.21.15.133:0/1869640269 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9d68018070 con 0x7f9d7c0b3c10 2024-03-07T06:51:52.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.737+0000 7f9d79600640 1 --2- 172.21.15.133:0/1869640269 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9d5c06b4d0 0x7f9d5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:52.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.738+0000 7f9d79600640 1 -- 172.21.15.133:0/1869640269 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d6808a180 con 0x7f9d7c0b3c10 2024-03-07T06:51:52.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.738+0000 7f9d81400640 1 --2- 172.21.15.133:0/1869640269 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9d5c06b4d0 0x7f9d5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:52.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.739+0000 7f9d81400640 1 --2- 172.21.15.133:0/1869640269 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9d5c06b4d0 0x7f9d5c06d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9d70036000 tx=0x7f9d7003c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:52.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.742+0000 7f9d79600640 1 -- 172.21.15.133:0/1869640269 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9d68058ff0 con 0x7f9d7c0b3c10 2024-03-07T06:51:52.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.937+0000 7f9d83e00640 1 -- 172.21.15.133:0/1869640269 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d7c1630e0 con 0x7f9d7c0b3c10 2024-03-07T06:51:52.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.940+0000 7f9d79600640 1 -- 172.21.15.133:0/1869640269 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16653 (secure 0 0 0) 0x7f9d680588f0 con 0x7f9d7c0b3c10 2024-03-07T06:51:52.940 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:52.941 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":21,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:16.174650+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:52.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.942+0000 7f9d83e00640 1 -- 172.21.15.133:0/1869640269 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9d5c06b4d0 msgr2=0x7f9d5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:52.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.942+0000 7f9d83e00640 1 --2- 172.21.15.133:0/1869640269 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9d5c06b4d0 0x7f9d5c06d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9d70036000 tx=0x7f9d7003c040 comp rx=0 tx=0).stop 2024-03-07T06:51:52.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.942+0000 7f9d83e00640 1 -- 172.21.15.133:0/1869640269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d7c0b3c10 msgr2=0x7f9d7c160860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:52.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.942+0000 7f9d83e00640 1 --2- 172.21.15.133:0/1869640269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d7c0b3c10 0x7f9d7c160860 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800c7f0 tx=0x7f9d6800cbf0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.943+0000 7f9d83e00640 1 -- 172.21.15.133:0/1869640269 shutdown_connections 2024-03-07T06:51:52.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.943+0000 7f9d83e00640 1 --2- 172.21.15.133:0/1869640269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d7c165ed0 0x7f9d7c160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.943+0000 7f9d83e00640 1 --2- 172.21.15.133:0/1869640269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d7c0b3c10 0x7f9d7c160860 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.943+0000 7f9d83e00640 1 --2- 172.21.15.133:0/1869640269 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9d5c06b4d0 0x7f9d5c06d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.943+0000 7f9d83e00640 1 --2- 172.21.15.133:0/1869640269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d7c071b50 0x7f9d7c14f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:52.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.943+0000 7f9d83e00640 1 -- 172.21.15.133:0/1869640269 >> 172.21.15.133:0/1869640269 conn(0x7f9d7c0b3120 msgr2=0x7f9d7c06e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:52.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.943+0000 7f9d83e00640 1 -- 172.21.15.133:0/1869640269 shutdown_connections 2024-03-07T06:51:52.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:52.943+0000 7f9d83e00640 1 -- 172.21.15.133:0/1869640269 wait complete. 2024-03-07T06:51:52.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:51:12.858333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:51:13.615348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:51:52.962 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:51:53.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.170+0000 7f82b7200640 1 Processor -- start 2024-03-07T06:51:53.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.170+0000 7f82b7200640 1 -- start start 2024-03-07T06:51:53.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.172+0000 7f82b7200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f82b0145670 0x7f82b00cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.172+0000 7f82b7200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82b00cdbf0 0x7f82b00719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.172+0000 7f82b7200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b006e150 con 0x7f82b015cf90 2024-03-07T06:51:53.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.172+0000 7f82b7200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b006ed60 con 0x7f82b0145670 2024-03-07T06:51:53.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.172+0000 7f82b7200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b006e430 con 0x7f82b00cdbf0 2024-03-07T06:51:53.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.172+0000 7f82af400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f82b015cf90 0x7f82b00cd4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57024/0 (socket says 172.21.15.133:57024) 2024-03-07T06:51:53.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.172+0000 7f82af400640 1 -- 172.21.15.133:0/4079342450 learned_addr learned my addr 172.21.15.133:0/4079342450 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:53.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.172+0000 7f82afe00640 1 --2- 172.21.15.133:0/4079342450 >> v2:172.21.15.161:3300/0 conn(0x7f82b0145670 0x7f82b00cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82b5200640 1 --2- 172.21.15.133:0/4079342450 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82b00cdbf0 0x7f82b00719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 -- 172.21.15.133:0/4079342450 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980457955 0 0) 0x7f82b006e150 con 0x7f82b015cf90 2024-03-07T06:51:53.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 -- 172.21.15.133:0/4079342450 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8294003520 con 0x7f82b015cf90 2024-03-07T06:51:53.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 -- 172.21.15.133:0/4079342450 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2372148651 0 0) 0x7f8294003520 con 0x7f82b015cf90 2024-03-07T06:51:53.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 -- 172.21.15.133:0/4079342450 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82b00cdbf0 msgr2=0x7f82b00719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:53.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 --2- 172.21.15.133:0/4079342450 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82b00cdbf0 0x7f82b00719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 -- 172.21.15.133:0/4079342450 >> v2:172.21.15.161:3300/0 conn(0x7f82b0145670 msgr2=0x7f82b00cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:53.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 --2- 172.21.15.133:0/4079342450 >> v2:172.21.15.161:3300/0 conn(0x7f82b0145670 0x7f82b00cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 -- 172.21.15.133:0/4079342450 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b006e710 con 0x7f82b015cf90 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 -- 172.21.15.133:0/4079342450 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f829c001670 con 0x7f82b015cf90 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 -- 172.21.15.133:0/4079342450 >> v1:172.21.15.133:6789/0 conn(0x7f82b015cf90 legacy=0x7f82b00cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 --2- 172.21.15.133:0/4079342450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82940039b0 0x7f8294003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 --2- 172.21.15.133:0/4079342450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294004800 0x7f8294004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 --2- 172.21.15.133:0/4079342450 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82940051c0 0x7f829400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 -- 172.21.15.133:0/4079342450 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b006ed60 con 0x7f82940039b0 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 -- 172.21.15.133:0/4079342450 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b006e430 con 0x7f8294004800 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.173+0000 7f82aea00640 1 -- 172.21.15.133:0/4079342450 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829400bf80 con 0x7f82940051c0 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.174+0000 7f82b5200640 1 --2- 172.21.15.133:0/4079342450 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82b00cdbf0 0x7f82b00719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.174+0000 7f82afe00640 1 --2- 172.21.15.133:0/4079342450 >> v2:172.21.15.161:3300/0 conn(0x7f82b0145670 0x7f82b00cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.174+0000 7f82afe00640 1 --2- 172.21.15.133:0/4079342450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82940039b0 0x7f8294003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.174+0000 7f82b5200640 1 --2- 172.21.15.133:0/4079342450 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82940051c0 0x7f829400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.174+0000 7f82afe00640 1 -- 172.21.15.133:0/4079342450 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82940051c0 msgr2=0x7f829400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:53.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.174+0000 7f82afe00640 1 --2- 172.21.15.133:0/4079342450 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82940051c0 0x7f829400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.174+0000 7f82afe00640 1 -- 172.21.15.133:0/4079342450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294004800 msgr2=0x7f8294004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:53.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.174+0000 7f82af400640 1 --2- 172.21.15.133:0/4079342450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294004800 0x7f8294004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.174+0000 7f82afe00640 1 --2- 172.21.15.133:0/4079342450 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8294004800 0x7f8294004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.174+0000 7f82afe00640 1 -- 172.21.15.133:0/4079342450 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8294003520 con 0x7f82940039b0 2024-03-07T06:51:53.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.176+0000 7f82afe00640 1 --2- 172.21.15.133:0/4079342450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82940039b0 0x7f8294003dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f82a0009c70 tx=0x7f8294005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e443f180c03c4cec server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:53.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.176+0000 7f82aea00640 1 -- 172.21.15.133:0/4079342450 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a0009840 con 0x7f82940039b0 2024-03-07T06:51:53.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.176+0000 7f82aea00640 1 -- 172.21.15.133:0/4079342450 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a0004020 con 0x7f82940039b0 2024-03-07T06:51:53.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.176+0000 7f82aea00640 1 -- 172.21.15.133:0/4079342450 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a0004410 con 0x7f82940039b0 2024-03-07T06:51:53.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.177+0000 7f82b7200640 1 -- 172.21.15.133:0/4079342450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82940039b0 msgr2=0x7f8294003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:53.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.177+0000 7f82b7200640 1 --2- 172.21.15.133:0/4079342450 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82940039b0 0x7f8294003dd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f82a0009c70 tx=0x7f8294005830 comp rx=0 tx=0).stop 2024-03-07T06:51:53.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.177+0000 7f82b5200640 1 -- 172.21.15.133:0/4079342450 reap_dead start 2024-03-07T06:51:53.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.177+0000 7f82b7200640 1 -- 172.21.15.133:0/4079342450 shutdown_connections 2024-03-07T06:51:53.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.177+0000 7f82b7200640 1 -- 172.21.15.133:0/4079342450 >> 172.21.15.133:0/4079342450 conn(0x7f82b00b1d30 msgr2=0x7f82b00b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:53.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.177+0000 7f82b7200640 1 -- 172.21.15.133:0/4079342450 shutdown_connections 2024-03-07T06:51:53.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.177+0000 7f82b7200640 1 -- 172.21.15.133:0/4079342450 wait complete. 2024-03-07T06:51:53.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.179+0000 7f82b7200640 1 Processor -- start 2024-03-07T06:51:53.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.179+0000 7f82b7200640 1 -- start start 2024-03-07T06:51:53.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.180+0000 7f82b7200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82940039b0 0x7f82b01365f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.180+0000 7f82b7200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b00cbc90 0x7f82b00ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.180+0000 7f82b7200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b013e4c0 0x7f82b015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.180+0000 7f82b7200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b0145a50 con 0x7f82b00cbc90 2024-03-07T06:51:53.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.180+0000 7f82b7200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b015d420 con 0x7f82b013e4c0 2024-03-07T06:51:53.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.180+0000 7f82b7200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b0145bd0 con 0x7f82940039b0 2024-03-07T06:51:53.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.181+0000 7f82af400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b00cbc90 0x7f82b00ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.181+0000 7f82af400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b00cbc90 0x7f82b00ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37118/0 (socket says 172.21.15.133:37118) 2024-03-07T06:51:53.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.181+0000 7f82af400640 1 -- 172.21.15.133:0/2119327933 learned_addr learned my addr 172.21.15.133:0/2119327933 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:53.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.181+0000 7f82afe00640 1 --2- 172.21.15.133:0/2119327933 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82940039b0 0x7f82b01365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.181+0000 7f82af400640 1 -- 172.21.15.133:0/2119327933 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82940039b0 msgr2=0x7f82b01365f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:53.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.181+0000 7f82af400640 1 --2- 172.21.15.133:0/2119327933 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82940039b0 0x7f82b01365f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.181+0000 7f82af400640 1 -- 172.21.15.133:0/2119327933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b013e4c0 msgr2=0x7f82b015d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:53.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.181+0000 7f82b5200640 1 --2- 172.21.15.133:0/2119327933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b013e4c0 0x7f82b015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.181+0000 7f82af400640 1 --2- 172.21.15.133:0/2119327933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b013e4c0 0x7f82b015d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.181+0000 7f82af400640 1 -- 172.21.15.133:0/2119327933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b00caed0 con 0x7f82b00cbc90 2024-03-07T06:51:53.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.181+0000 7f82af400640 1 --2- 172.21.15.133:0/2119327933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b00cbc90 0x7f82b00ccae0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f829c00d7a0 tx=0x7f829c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:53.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.182+0000 7f82a7e00640 1 -- 172.21.15.133:0/2119327933 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c019070 con 0x7f82b00cbc90 2024-03-07T06:51:53.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.182+0000 7f82a7e00640 1 -- 172.21.15.133:0/2119327933 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829c0072d0 con 0x7f82b00cbc90 2024-03-07T06:51:53.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.182+0000 7f82a7e00640 1 -- 172.21.15.133:0/2119327933 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c0047e0 con 0x7f82b00cbc90 2024-03-07T06:51:53.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.182+0000 7f82b7200640 1 -- 172.21.15.133:0/2119327933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b00cb1b0 con 0x7f82b00cbc90 2024-03-07T06:51:53.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.182+0000 7f82b7200640 1 -- 172.21.15.133:0/2119327933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b00cabc0 con 0x7f82b00cbc90 2024-03-07T06:51:53.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.183+0000 7f82b7200640 1 -- 172.21.15.133:0/2119327933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b0074b00 con 0x7f82b00cbc90 2024-03-07T06:51:53.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.184+0000 7f82a7e00640 1 -- 172.21.15.133:0/2119327933 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f829c01c070 con 0x7f82b00cbc90 2024-03-07T06:51:53.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.185+0000 7f82a7e00640 1 --2- 172.21.15.133:0/2119327933 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f829006b4d0 0x7f829006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.185+0000 7f82a7e00640 1 -- 172.21.15.133:0/2119327933 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f829c089b60 con 0x7f82b00cbc90 2024-03-07T06:51:53.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.186+0000 7f82afe00640 1 --2- 172.21.15.133:0/2119327933 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f829006b4d0 0x7f829006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.186+0000 7f82afe00640 1 --2- 172.21.15.133:0/2119327933 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f829006b4d0 0x7f829006d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f82a0035000 tx=0x7f82a0030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:53.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.189+0000 7f82a7e00640 1 -- 172.21.15.133:0/2119327933 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f82b0074b00 con 0x7f82b00cbc90 2024-03-07T06:51:53.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.371+0000 7f82b7200640 1 -- 172.21.15.133:0/2119327933 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82b0073300 con 0x7f82b00cbc90 2024-03-07T06:51:53.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.372+0000 7f82a7e00640 1 -- 172.21.15.133:0/2119327933 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82b0073300 con 0x7f82b00cbc90 2024-03-07T06:51:53.372 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:51:53.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.375+0000 7f82b7200640 1 -- 172.21.15.133:0/2119327933 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f829006b4d0 msgr2=0x7f829006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:53.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.375+0000 7f82b7200640 1 --2- 172.21.15.133:0/2119327933 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f829006b4d0 0x7f829006d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f82a0035000 tx=0x7f82a0030000 comp rx=0 tx=0).stop 2024-03-07T06:51:53.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.375+0000 7f82b7200640 1 -- 172.21.15.133:0/2119327933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b00cbc90 msgr2=0x7f82b00ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:53.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.375+0000 7f82b7200640 1 --2- 172.21.15.133:0/2119327933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b00cbc90 0x7f82b00ccae0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f829c00d7a0 tx=0x7f829c00dc60 comp rx=0 tx=0).stop 2024-03-07T06:51:53.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.375+0000 7f82b7200640 1 -- 172.21.15.133:0/2119327933 shutdown_connections 2024-03-07T06:51:53.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.375+0000 7f82b7200640 1 --2- 172.21.15.133:0/2119327933 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f829006b4d0 0x7f829006d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.375+0000 7f82b7200640 1 --2- 172.21.15.133:0/2119327933 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b013e4c0 0x7f82b015d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.375+0000 7f82b7200640 1 --2- 172.21.15.133:0/2119327933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82b00cbc90 0x7f82b00ccae0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.375+0000 7f82b7200640 1 --2- 172.21.15.133:0/2119327933 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82940039b0 0x7f82b01365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.375+0000 7f82b7200640 1 -- 172.21.15.133:0/2119327933 >> 172.21.15.133:0/2119327933 conn(0x7f82b00b1d30 msgr2=0x7f82b00cd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:53.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.375+0000 7f82b7200640 1 -- 172.21.15.133:0/2119327933 shutdown_connections 2024-03-07T06:51:53.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.375+0000 7f82b7200640 1 -- 172.21.15.133:0/2119327933 wait complete. 2024-03-07T06:51:53.392 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:51:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.600+0000 7fdfcc600640 1 Processor -- start 2024-03-07T06:51:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.600+0000 7fdfcc600640 1 -- start start 2024-03-07T06:51:53.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.601+0000 7fdfcc600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdfc40cf610 0x7fdfc40cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.602+0000 7fdfcc600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc406e150 con 0x7fdfc4145670 2024-03-07T06:51:53.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.602+0000 7fdfcc600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc406e430 con 0x7fdfc40cda20 2024-03-07T06:51:53.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.602+0000 7fdfcc600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40cfa70 con 0x7fdfc40cf610 2024-03-07T06:51:53.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.602+0000 7fdfc9c00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fdfc4145670 0x7fdfc415cf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57032/0 (socket says 172.21.15.133:57032) 2024-03-07T06:51:53.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.602+0000 7fdfc9c00640 1 -- 172.21.15.133:0/94470544 learned_addr learned my addr 172.21.15.133:0/94470544 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:53.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.602+0000 7fdfc9200640 1 --2- 172.21.15.133:0/94470544 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdfc40cf610 0x7fdfc40cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.602+0000 7fdfc3e00640 1 -- 172.21.15.133:0/94470544 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934757929 0 0) 0x7fdfc406e150 con 0x7fdfc4145670 2024-03-07T06:51:53.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.602+0000 7fdfc3e00640 1 -- 172.21.15.133:0/94470544 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfac003520 con 0x7fdfc4145670 2024-03-07T06:51:53.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc3e00640 1 -- 172.21.15.133:0/94470544 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 727080808 0 0) 0x7fdfac003520 con 0x7fdfc4145670 2024-03-07T06:51:53.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc3e00640 1 -- 172.21.15.133:0/94470544 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdfc40cf610 msgr2=0x7fdfc40cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:53.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc3e00640 1 --2- 172.21.15.133:0/94470544 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdfc40cf610 0x7fdfc40cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc3e00640 1 -- 172.21.15.133:0/94470544 >> v1:172.21.15.161:6789/0 conn(0x7fdfc40cda20 legacy=0x7fdfc40719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:53.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc3e00640 1 -- 172.21.15.133:0/94470544 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc406e710 con 0x7fdfc4145670 2024-03-07T06:51:53.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc3e00640 1 -- 172.21.15.133:0/94470544 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fdfb8001670 con 0x7fdfc4145670 2024-03-07T06:51:53.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc3e00640 1 -- 172.21.15.133:0/94470544 >> v1:172.21.15.133:6789/0 conn(0x7fdfc4145670 legacy=0x7fdfc415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc3e00640 1 --2- 172.21.15.133:0/94470544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfac0039e0 0x7fdfac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc3e00640 1 --2- 172.21.15.133:0/94470544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfac004800 0x7fdfac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc3e00640 1 --2- 172.21.15.133:0/94470544 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdfc40cf610 0x7fdfac005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc3e00640 1 -- 172.21.15.133:0/94470544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40cfa70 con 0x7fdfac004800 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc3e00640 1 -- 172.21.15.133:0/94470544 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac00bb80 con 0x7fdfac0039e0 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc3e00640 1 -- 172.21.15.133:0/94470544 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac00bd80 con 0x7fdfc40cf610 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc9200640 1 --2- 172.21.15.133:0/94470544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfac004800 0x7fdfac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfca600640 1 --2- 172.21.15.133:0/94470544 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdfc40cf610 0x7fdfac005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.603+0000 7fdfc9c00640 1 --2- 172.21.15.133:0/94470544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfac0039e0 0x7fdfac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.604+0000 7fdfc9200640 1 -- 172.21.15.133:0/94470544 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdfc40cf610 msgr2=0x7fdfac005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.604+0000 7fdfc9200640 1 --2- 172.21.15.133:0/94470544 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdfc40cf610 0x7fdfac005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.604+0000 7fdfc9200640 1 -- 172.21.15.133:0/94470544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfac0039e0 msgr2=0x7fdfac003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.604+0000 7fdfc9200640 1 --2- 172.21.15.133:0/94470544 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfac0039e0 0x7fdfac003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.604+0000 7fdfc9200640 1 -- 172.21.15.133:0/94470544 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac003520 con 0x7fdfac004800 2024-03-07T06:51:53.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.605+0000 7fdfc9200640 1 --2- 172.21.15.133:0/94470544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfac004800 0x7fdfac004c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009ac0 tx=0x7fdfac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801d17feebb27a05 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.605+0000 7fdfc3e00640 1 -- 172.21.15.133:0/94470544 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0039070 con 0x7fdfac004800 2024-03-07T06:51:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.605+0000 7fdfc3e00640 1 -- 172.21.15.133:0/94470544 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb002c6b0 con 0x7fdfac004800 2024-03-07T06:51:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.605+0000 7fdfc3e00640 1 -- 172.21.15.133:0/94470544 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0009690 con 0x7fdfac004800 2024-03-07T06:51:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.605+0000 7fdfcc600640 1 -- 172.21.15.133:0/94470544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfac004800 msgr2=0x7fdfac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.605+0000 7fdfcc600640 1 --2- 172.21.15.133:0/94470544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfac004800 0x7fdfac004c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009ac0 tx=0x7fdfac005830 comp rx=0 tx=0).stop 2024-03-07T06:51:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.605+0000 7fdfca600640 1 -- 172.21.15.133:0/94470544 reap_dead start 2024-03-07T06:51:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.606+0000 7fdfcc600640 1 -- 172.21.15.133:0/94470544 shutdown_connections 2024-03-07T06:51:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.606+0000 7fdfcc600640 1 -- 172.21.15.133:0/94470544 >> 172.21.15.133:0/94470544 conn(0x7fdfc40b1d30 msgr2=0x7fdfc40b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.606+0000 7fdfcc600640 1 -- 172.21.15.133:0/94470544 shutdown_connections 2024-03-07T06:51:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.606+0000 7fdfcc600640 1 -- 172.21.15.133:0/94470544 wait complete. 2024-03-07T06:51:53.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.607+0000 7fdfcc600640 1 Processor -- start 2024-03-07T06:51:53.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.607+0000 7fdfcc600640 1 -- start start 2024-03-07T06:51:53.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.608+0000 7fdfcc600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdfac004800 0x7fdfc413ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.608+0000 7fdfcc600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfc41f30e0 0x7fdfc41f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.608+0000 7fdfcc600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfc41f3560 0x7fdfc41f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.608+0000 7fdfcc600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc4145a50 con 0x7fdfc41f3560 2024-03-07T06:51:53.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.608+0000 7fdfcc600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc406ed60 con 0x7fdfc41f30e0 2024-03-07T06:51:53.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.608+0000 7fdfcc600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc4145bd0 con 0x7fdfac004800 2024-03-07T06:51:53.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.609+0000 7fdfca600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfc41f3560 0x7fdfc41f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.609+0000 7fdfca600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfc41f3560 0x7fdfc41f7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37138/0 (socket says 172.21.15.133:37138) 2024-03-07T06:51:53.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.609+0000 7fdfca600640 1 -- 172.21.15.133:0/2750941253 learned_addr learned my addr 172.21.15.133:0/2750941253 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:53.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.609+0000 7fdfc9c00640 1 --2- 172.21.15.133:0/2750941253 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdfac004800 0x7fdfc413ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.609+0000 7fdfc9200640 1 --2- 172.21.15.133:0/2750941253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfc41f30e0 0x7fdfc41f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.609+0000 7fdfca600640 1 -- 172.21.15.133:0/2750941253 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdfac004800 msgr2=0x7fdfc413ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:53.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.609+0000 7fdfca600640 1 --2- 172.21.15.133:0/2750941253 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdfac004800 0x7fdfc413ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.609+0000 7fdfca600640 1 -- 172.21.15.133:0/2750941253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfc41f30e0 msgr2=0x7fdfc41f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:53.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.609+0000 7fdfca600640 1 --2- 172.21.15.133:0/2750941253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfc41f30e0 0x7fdfc41f6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.609+0000 7fdfca600640 1 -- 172.21.15.133:0/2750941253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc40b3d40 con 0x7fdfc41f3560 2024-03-07T06:51:53.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.609+0000 7fdfca600640 1 --2- 172.21.15.133:0/2750941253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfc41f3560 0x7fdfc41f7450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc007fd0 tx=0x7fdfbc00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:53.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.609+0000 7fdfc1600640 1 -- 172.21.15.133:0/2750941253 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc003970 con 0x7fdfc41f3560 2024-03-07T06:51:53.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.610+0000 7fdfc1600640 1 -- 172.21.15.133:0/2750941253 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc018070 con 0x7fdfc41f3560 2024-03-07T06:51:53.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.610+0000 7fdfc1600640 1 -- 172.21.15.133:0/2750941253 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc003d70 con 0x7fdfc41f3560 2024-03-07T06:51:53.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.610+0000 7fdfcc600640 1 -- 172.21.15.133:0/2750941253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc40b4020 con 0x7fdfc41f3560 2024-03-07T06:51:53.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.610+0000 7fdfcc600640 1 -- 172.21.15.133:0/2750941253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc40b45e0 con 0x7fdfc41f3560 2024-03-07T06:51:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.611+0000 7fdfcc600640 1 -- 172.21.15.133:0/2750941253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc4062c90 con 0x7fdfc41f3560 2024-03-07T06:51:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.612+0000 7fdfc1600640 1 -- 172.21.15.133:0/2750941253 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fdfbc0050d0 con 0x7fdfc41f3560 2024-03-07T06:51:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.613+0000 7fdfc1600640 1 --2- 172.21.15.133:0/2750941253 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdfa406b500 0x7fdfa406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:53.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.613+0000 7fdfc1600640 1 -- 172.21.15.133:0/2750941253 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfbc089ef0 con 0x7fdfc41f3560 2024-03-07T06:51:53.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.613+0000 7fdfc9c00640 1 --2- 172.21.15.133:0/2750941253 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdfa406b500 0x7fdfa406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:53.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.614+0000 7fdfc9c00640 1 --2- 172.21.15.133:0/2750941253 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdfa406b500 0x7fdfa406d9c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80017b0 tx=0x7fdfb8000ed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:53.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.617+0000 7fdfc1600640 1 -- 172.21.15.133:0/2750941253 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fdfbc058cd0 con 0x7fdfc41f3560 2024-03-07T06:51:53.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.809+0000 7fdfcc600640 1 -- 172.21.15.133:0/2750941253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdfc4144d70 con 0x7fdfc41f3560 2024-03-07T06:51:53.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.810+0000 7fdfc1600640 1 -- 172.21.15.133:0/2750941253 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdfbc0585d0 con 0x7fdfc41f3560 2024-03-07T06:51:53.810 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:51:53.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.813+0000 7fdfcc600640 1 -- 172.21.15.133:0/2750941253 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdfa406b500 msgr2=0x7fdfa406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:53.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.813+0000 7fdfcc600640 1 --2- 172.21.15.133:0/2750941253 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdfa406b500 0x7fdfa406d9c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdfb80017b0 tx=0x7fdfb8000ed0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.813+0000 7fdfcc600640 1 -- 172.21.15.133:0/2750941253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfc41f3560 msgr2=0x7fdfc41f7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:53.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.813+0000 7fdfcc600640 1 --2- 172.21.15.133:0/2750941253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfc41f3560 0x7fdfc41f7450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc007fd0 tx=0x7fdfbc00ec10 comp rx=0 tx=0).stop 2024-03-07T06:51:53.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.813+0000 7fdfcc600640 1 -- 172.21.15.133:0/2750941253 shutdown_connections 2024-03-07T06:51:53.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.813+0000 7fdfcc600640 1 --2- 172.21.15.133:0/2750941253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfc41f3560 0x7fdfc41f7450 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.813+0000 7fdfcc600640 1 --2- 172.21.15.133:0/2750941253 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdfa406b500 0x7fdfa406d9c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.813+0000 7fdfcc600640 1 --2- 172.21.15.133:0/2750941253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdfc41f30e0 0x7fdfc41f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.813+0000 7fdfcc600640 1 --2- 172.21.15.133:0/2750941253 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdfac004800 0x7fdfc413ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:53.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.813+0000 7fdfcc600640 1 -- 172.21.15.133:0/2750941253 >> 172.21.15.133:0/2750941253 conn(0x7fdfc40b1d30 msgr2=0x7fdfc415e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:53.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.813+0000 7fdfcc600640 1 -- 172.21.15.133:0/2750941253 shutdown_connections 2024-03-07T06:51:53.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:53.813+0000 7fdfcc600640 1 -- 172.21.15.133:0/2750941253 wait complete. 2024-03-07T06:51:53.831 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:51:54.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.042+0000 7f5ddf800640 1 Processor -- start 2024-03-07T06:51:54.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.042+0000 7f5ddf800640 1 -- start start 2024-03-07T06:51:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.043+0000 7f5ddf800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5dd815b370 0x7f5dd80cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.043+0000 7f5ddf800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5dd806d180 0x7f5dd80cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.043+0000 7f5ddf800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd806d740 con 0x7f5dd815b750 2024-03-07T06:51:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.043+0000 7f5ddf800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd806da20 con 0x7f5dd815b370 2024-03-07T06:51:54.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.043+0000 7f5ddf800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd806dd00 con 0x7f5dd806d180 2024-03-07T06:51:54.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.044+0000 7f5dd7e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5dd806d180 0x7f5dd80cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.044+0000 7f5dd7e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5dd806d180 0x7f5dd80cf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:59952/0 (socket says 172.21.15.133:59952) 2024-03-07T06:51:54.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.044+0000 7f5dd7e00640 1 -- 172.21.15.133:0/3510447 learned_addr learned my addr 172.21.15.133:0/3510447 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:54.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.044+0000 7f5ddce00640 1 --2- 172.21.15.133:0/3510447 >> v2:172.21.15.161:3300/0 conn(0x7f5dd815b370 0x7f5dd80cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.045+0000 7f5dd7e00640 1 -- 172.21.15.133:0/3510447 >> v2:172.21.15.161:3300/0 conn(0x7f5dd815b370 msgr2=0x7f5dd80cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:54.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.045+0000 7f5dd7e00640 1 --2- 172.21.15.133:0/3510447 >> v2:172.21.15.161:3300/0 conn(0x7f5dd815b370 0x7f5dd80cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.045+0000 7f5dd7e00640 1 -- 172.21.15.133:0/3510447 >> v1:172.21.15.161:6789/0 conn(0x7f5dd815b750 legacy=0x7f5dd815bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.045+0000 7f5dd7400640 1 -- 172.21.15.133:0/3510447 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472569228 0 0) 0x7f5dd806d740 con 0x7f5dd815b750 2024-03-07T06:51:54.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.045+0000 7f5dd7e00640 1 -- 172.21.15.133:0/3510447 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd806dfe0 con 0x7f5dd806d180 2024-03-07T06:51:54.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.046+0000 7f5dd7e00640 1 --2- 172.21.15.133:0/3510447 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5dd806d180 0x7f5dd80cf490 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009870 tx=0x7f5dc002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=725d18fa32b2e3d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:54.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.047+0000 7f5dd7400640 1 -- 172.21.15.133:0/3510447 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0034070 con 0x7f5dd806d180 2024-03-07T06:51:54.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.047+0000 7f5dd7400640 1 -- 172.21.15.133:0/3510447 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc0030840 con 0x7f5dd806d180 2024-03-07T06:51:54.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.047+0000 7f5dd7400640 1 -- 172.21.15.133:0/3510447 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0030c10 con 0x7f5dd806d180 2024-03-07T06:51:54.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.047+0000 7f5ddf800640 1 -- 172.21.15.133:0/3510447 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5dd806d180 msgr2=0x7f5dd80cf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.047+0000 7f5ddf800640 1 --2- 172.21.15.133:0/3510447 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5dd806d180 0x7f5dd80cf490 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009870 tx=0x7f5dc002b980 comp rx=0 tx=0).stop 2024-03-07T06:51:54.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.048+0000 7f5ddf800640 1 -- 172.21.15.133:0/3510447 shutdown_connections 2024-03-07T06:51:54.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.048+0000 7f5ddf800640 1 --2- 172.21.15.133:0/3510447 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5dd806d180 0x7f5dd80cf490 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.048+0000 7f5ddf800640 1 --2- 172.21.15.133:0/3510447 >> v2:172.21.15.161:3300/0 conn(0x7f5dd815b370 0x7f5dd80cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.048+0000 7f5ddf800640 1 -- 172.21.15.133:0/3510447 >> 172.21.15.133:0/3510447 conn(0x7f5dd80b1d30 msgr2=0x7f5dd80b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:54.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.048+0000 7f5ddf800640 1 -- 172.21.15.133:0/3510447 shutdown_connections 2024-03-07T06:51:54.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.048+0000 7f5ddf800640 1 -- 172.21.15.133:0/3510447 wait complete. 2024-03-07T06:51:54.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.049+0000 7f5ddf800640 1 Processor -- start 2024-03-07T06:51:54.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.049+0000 7f5ddf800640 1 -- start start 2024-03-07T06:51:54.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.050+0000 7f5ddf800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dd806d180 0x7f5dd80cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.050+0000 7f5ddf800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5dd815b370 0x7f5dd80c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.050+0000 7f5ddf800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dd815b750 0x7f5dd80ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.050+0000 7f5ddf800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd80b4050 con 0x7f5dd815b750 2024-03-07T06:51:54.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.050+0000 7f5ddf800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd80b3ed0 con 0x7f5dd806d180 2024-03-07T06:51:54.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.050+0000 7f5ddf800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd80b41d0 con 0x7f5dd815b370 2024-03-07T06:51:54.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.051+0000 7f5ddd800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dd815b750 0x7f5dd80ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.051+0000 7f5ddd800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dd815b750 0x7f5dd80ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37140/0 (socket says 172.21.15.133:37140) 2024-03-07T06:51:54.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.051+0000 7f5ddd800640 1 -- 172.21.15.133:0/4038193471 learned_addr learned my addr 172.21.15.133:0/4038193471 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:54.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.051+0000 7f5dd7e00640 1 --2- 172.21.15.133:0/4038193471 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5dd815b370 0x7f5dd80c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.051+0000 7f5ddce00640 1 --2- 172.21.15.133:0/4038193471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dd806d180 0x7f5dd80cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.051+0000 7f5ddd800640 1 -- 172.21.15.133:0/4038193471 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5dd815b370 msgr2=0x7f5dd80c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.051+0000 7f5ddd800640 1 --2- 172.21.15.133:0/4038193471 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5dd815b370 0x7f5dd80c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.051+0000 7f5ddd800640 1 -- 172.21.15.133:0/4038193471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dd806d180 msgr2=0x7f5dd80cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.051+0000 7f5ddd800640 1 --2- 172.21.15.133:0/4038193471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dd806d180 0x7f5dd80cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.051+0000 7f5ddd800640 1 -- 172.21.15.133:0/4038193471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd80cc650 con 0x7f5dd815b750 2024-03-07T06:51:54.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.052+0000 7f5ddd800640 1 --2- 172.21.15.133:0/4038193471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dd815b750 0x7f5dd80ca190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000c7b0 tx=0x7f5dd000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:54.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.052+0000 7f5dd4c00640 1 -- 172.21.15.133:0/4038193471 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0003ba0 con 0x7f5dd815b750 2024-03-07T06:51:54.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.052+0000 7f5dd4c00640 1 -- 172.21.15.133:0/4038193471 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd0003d40 con 0x7f5dd815b750 2024-03-07T06:51:54.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.052+0000 7f5ddf800640 1 -- 172.21.15.133:0/4038193471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd80cc930 con 0x7f5dd815b750 2024-03-07T06:51:54.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.052+0000 7f5dd4c00640 1 -- 172.21.15.133:0/4038193471 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0005270 con 0x7f5dd815b750 2024-03-07T06:51:54.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.052+0000 7f5ddf800640 1 -- 172.21.15.133:0/4038193471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd80cc1d0 con 0x7f5dd815b750 2024-03-07T06:51:54.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.053+0000 7f5ddf800640 1 -- 172.21.15.133:0/4038193471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd80c4dc0 con 0x7f5dd815b750 2024-03-07T06:51:54.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.054+0000 7f5dd4c00640 1 -- 172.21.15.133:0/4038193471 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5dd0005410 con 0x7f5dd815b750 2024-03-07T06:51:54.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.055+0000 7f5dd4c00640 1 --2- 172.21.15.133:0/4038193471 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5db406b4d0 0x7f5db406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.055+0000 7f5dd4c00640 1 -- 172.21.15.133:0/4038193471 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5dd008a0e0 con 0x7f5dd815b750 2024-03-07T06:51:54.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.055+0000 7f5ddce00640 1 --2- 172.21.15.133:0/4038193471 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5db406b4d0 0x7f5db406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.056+0000 7f5ddce00640 1 --2- 172.21.15.133:0/4038193471 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5db406b4d0 0x7f5db406d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009ad0 tx=0x7f5dc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:54.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.059+0000 7f5dd4c00640 1 -- 172.21.15.133:0/4038193471 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5dd0058f60 con 0x7f5dd815b750 2024-03-07T06:51:54.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.257+0000 7f5ddf800640 1 -- 172.21.15.133:0/4038193471 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5dd8062c90 con 0x7f5dd815b750 2024-03-07T06:51:54.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.258+0000 7f5dd4c00640 1 -- 172.21.15.133:0/4038193471 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5dd0058860 con 0x7f5dd815b750 2024-03-07T06:51:54.258 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:51:54.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.261+0000 7f5ddf800640 1 -- 172.21.15.133:0/4038193471 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5db406b4d0 msgr2=0x7f5db406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.261+0000 7f5ddf800640 1 --2- 172.21.15.133:0/4038193471 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5db406b4d0 0x7f5db406d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009ad0 tx=0x7f5dc8008040 comp rx=0 tx=0).stop 2024-03-07T06:51:54.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.261+0000 7f5ddf800640 1 -- 172.21.15.133:0/4038193471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dd815b750 msgr2=0x7f5dd80ca190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.261+0000 7f5ddf800640 1 --2- 172.21.15.133:0/4038193471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dd815b750 0x7f5dd80ca190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000c7b0 tx=0x7f5dd000cbb0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.261+0000 7f5ddf800640 1 -- 172.21.15.133:0/4038193471 shutdown_connections 2024-03-07T06:51:54.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.261+0000 7f5ddf800640 1 --2- 172.21.15.133:0/4038193471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5dd815b750 0x7f5dd80ca190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.261+0000 7f5ddf800640 1 --2- 172.21.15.133:0/4038193471 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5dd815b370 0x7f5dd80c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.261+0000 7f5ddf800640 1 --2- 172.21.15.133:0/4038193471 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5db406b4d0 0x7f5db406d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.261+0000 7f5ddf800640 1 --2- 172.21.15.133:0/4038193471 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5dd806d180 0x7f5dd80cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.261+0000 7f5ddf800640 1 -- 172.21.15.133:0/4038193471 >> 172.21.15.133:0/4038193471 conn(0x7f5dd80b1d30 msgr2=0x7f5dd80b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:54.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.261+0000 7f5ddf800640 1 -- 172.21.15.133:0/4038193471 shutdown_connections 2024-03-07T06:51:54.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.261+0000 7f5ddf800640 1 -- 172.21.15.133:0/4038193471 wait complete. 2024-03-07T06:51:54.280 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:54.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.488+0000 7fd4b8600640 1 Processor -- start 2024-03-07T06:51:54.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.488+0000 7fd4b8600640 1 -- start start 2024-03-07T06:51:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.489+0000 7fd4b8600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4b015b1b0 0x7fd4b00b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.489+0000 7fd4b8600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b006f140 con 0x7fd4b00af950 2024-03-07T06:51:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.489+0000 7fd4b8600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b006f420 con 0x7fd4b00ce5f0 2024-03-07T06:51:54.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.489+0000 7fd4b8600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b0071610 con 0x7fd4b015b1b0 2024-03-07T06:51:54.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.490+0000 7fd4b5200640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd4b00af950 0x7fd4b00cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57046/0 (socket says 172.21.15.133:57046) 2024-03-07T06:51:54.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.490+0000 7fd4b5200640 1 -- 172.21.15.133:0/1952987343 learned_addr learned my addr 172.21.15.133:0/1952987343 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:54.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.490+0000 7fd4b5c00640 1 --2- 172.21.15.133:0/1952987343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4b015b1b0 0x7fd4b00b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.490+0000 7fd4afe00640 1 -- 172.21.15.133:0/1952987343 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428572423 0 0) 0x7fd4b006f140 con 0x7fd4b00af950 2024-03-07T06:51:54.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.490+0000 7fd4afe00640 1 -- 172.21.15.133:0/1952987343 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd498003520 con 0x7fd4b00af950 2024-03-07T06:51:54.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.491+0000 7fd4afe00640 1 -- 172.21.15.133:0/1952987343 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 863967190 0 0) 0x7fd498003520 con 0x7fd4b00af950 2024-03-07T06:51:54.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.491+0000 7fd4afe00640 1 -- 172.21.15.133:0/1952987343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4b015b1b0 msgr2=0x7fd4b00b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:54.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.491+0000 7fd4afe00640 1 --2- 172.21.15.133:0/1952987343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4b015b1b0 0x7fd4b00b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.491+0000 7fd4afe00640 1 -- 172.21.15.133:0/1952987343 >> v1:172.21.15.161:6789/0 conn(0x7fd4b00ce5f0 legacy=0x7fd4b0071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.491+0000 7fd4afe00640 1 -- 172.21.15.133:0/1952987343 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b0071860 con 0x7fd4b00af950 2024-03-07T06:51:54.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.491+0000 7fd4afe00640 1 -- 172.21.15.133:0/1952987343 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fd49c001670 con 0x7fd4b00af950 2024-03-07T06:51:54.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.491+0000 7fd4afe00640 1 -- 172.21.15.133:0/1952987343 >> v1:172.21.15.133:6789/0 conn(0x7fd4b00af950 legacy=0x7fd4b00cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.491+0000 7fd4afe00640 1 --2- 172.21.15.133:0/1952987343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4980039e0 0x7fd498003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.491+0000 7fd4afe00640 1 --2- 172.21.15.133:0/1952987343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd498004800 0x7fd498004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.491+0000 7fd4afe00640 1 --2- 172.21.15.133:0/1952987343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4b015b1b0 0x7fd498005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.491+0000 7fd4afe00640 1 -- 172.21.15.133:0/1952987343 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b0071610 con 0x7fd4980039e0 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.491+0000 7fd4afe00640 1 -- 172.21.15.133:0/1952987343 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49800baf0 con 0x7fd498004800 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.491+0000 7fd4afe00640 1 -- 172.21.15.133:0/1952987343 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49800bcf0 con 0x7fd4b015b1b0 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.491+0000 7fd4b5c00640 1 --2- 172.21.15.133:0/1952987343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4980039e0 0x7fd498003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.492+0000 7fd4b5200640 1 --2- 172.21.15.133:0/1952987343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd498004800 0x7fd498004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.492+0000 7fd4b6600640 1 --2- 172.21.15.133:0/1952987343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4b015b1b0 0x7fd498005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.492+0000 7fd4b5c00640 1 -- 172.21.15.133:0/1952987343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4b015b1b0 msgr2=0x7fd498005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.492+0000 7fd4b5c00640 1 --2- 172.21.15.133:0/1952987343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4b015b1b0 0x7fd498005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.492+0000 7fd4b5c00640 1 -- 172.21.15.133:0/1952987343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd498004800 msgr2=0x7fd498004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.492+0000 7fd4b5c00640 1 --2- 172.21.15.133:0/1952987343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd498004800 0x7fd498004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.492+0000 7fd4b5c00640 1 -- 172.21.15.133:0/1952987343 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd498003520 con 0x7fd4980039e0 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.492+0000 7fd4b6600640 1 --2- 172.21.15.133:0/1952987343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4b015b1b0 0x7fd498005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:54.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.492+0000 7fd4b5200640 1 --2- 172.21.15.133:0/1952987343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd498004800 0x7fd498004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.493+0000 7fd4b5c00640 1 --2- 172.21.15.133:0/1952987343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4980039e0 0x7fd498003e00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009ac0 tx=0x7fd498005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5c983e52ad5592 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.493+0000 7fd4afe00640 1 -- 172.21.15.133:0/1952987343 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4039070 con 0x7fd4980039e0 2024-03-07T06:51:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.493+0000 7fd4afe00640 1 -- 172.21.15.133:0/1952987343 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a402c6b0 con 0x7fd4980039e0 2024-03-07T06:51:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.493+0000 7fd4afe00640 1 -- 172.21.15.133:0/1952987343 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4009690 con 0x7fd4980039e0 2024-03-07T06:51:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.494+0000 7fd4b8600640 1 -- 172.21.15.133:0/1952987343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4980039e0 msgr2=0x7fd498003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.494+0000 7fd4b8600640 1 --2- 172.21.15.133:0/1952987343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4980039e0 0x7fd498003e00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009ac0 tx=0x7fd498005830 comp rx=0 tx=0).stop 2024-03-07T06:51:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.494+0000 7fd4b6600640 1 -- 172.21.15.133:0/1952987343 reap_dead start 2024-03-07T06:51:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.494+0000 7fd4b8600640 1 -- 172.21.15.133:0/1952987343 shutdown_connections 2024-03-07T06:51:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.494+0000 7fd4b8600640 1 -- 172.21.15.133:0/1952987343 >> 172.21.15.133:0/1952987343 conn(0x7fd4b00b3120 msgr2=0x7fd4b00b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.494+0000 7fd4b8600640 1 -- 172.21.15.133:0/1952987343 shutdown_connections 2024-03-07T06:51:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.494+0000 7fd4b8600640 1 -- 172.21.15.133:0/1952987343 wait complete. 2024-03-07T06:51:54.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.496+0000 7fd4b8600640 1 Processor -- start 2024-03-07T06:51:54.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.496+0000 7fd4b8600640 1 -- start start 2024-03-07T06:51:54.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.497+0000 7fd4b8600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4980039e0 0x7fd4b015e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.497+0000 7fd4b8600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01575b0 0x7fd4b0156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.497+0000 7fd4b8600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b009b970 0x7fd4b009be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.497+0000 7fd4b8600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b015b590 con 0x7fd4b01575b0 2024-03-07T06:51:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.497+0000 7fd4b8600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b006ee30 con 0x7fd4b009b970 2024-03-07T06:51:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.497+0000 7fd4b8600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b015b710 con 0x7fd4980039e0 2024-03-07T06:51:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.497+0000 7fd4b5200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01575b0 0x7fd4b0156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.497+0000 7fd4b5200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01575b0 0x7fd4b0156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37162/0 (socket says 172.21.15.133:37162) 2024-03-07T06:51:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.497+0000 7fd4b5200640 1 -- 172.21.15.133:0/608856505 learned_addr learned my addr 172.21.15.133:0/608856505 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.497+0000 7fd4b5c00640 1 --2- 172.21.15.133:0/608856505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4980039e0 0x7fd4b015e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.498+0000 7fd4b5200640 1 -- 172.21.15.133:0/608856505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4980039e0 msgr2=0x7fd4b015e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.498+0000 7fd4b5200640 1 --2- 172.21.15.133:0/608856505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4980039e0 0x7fd4b015e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.498+0000 7fd4b6600640 1 --2- 172.21.15.133:0/608856505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b009b970 0x7fd4b009be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.498+0000 7fd4b5200640 1 -- 172.21.15.133:0/608856505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b009b970 msgr2=0x7fd4b009be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.498+0000 7fd4b5200640 1 --2- 172.21.15.133:0/608856505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b009b970 0x7fd4b009be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.498+0000 7fd4b5200640 1 -- 172.21.15.133:0/608856505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b009c640 con 0x7fd4b01575b0 2024-03-07T06:51:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.498+0000 7fd4b5200640 1 --2- 172.21.15.133:0/608856505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01575b0 0x7fd4b0156c20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00d910 tx=0x7fd49c00ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.498+0000 7fd4ad600640 1 -- 172.21.15.133:0/608856505 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c019070 con 0x7fd4b01575b0 2024-03-07T06:51:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.498+0000 7fd4ad600640 1 -- 172.21.15.133:0/608856505 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c0074e0 con 0x7fd4b01575b0 2024-03-07T06:51:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.498+0000 7fd4b8600640 1 -- 172.21.15.133:0/608856505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b009c920 con 0x7fd4b01575b0 2024-03-07T06:51:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.498+0000 7fd4ad600640 1 -- 172.21.15.133:0/608856505 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c007690 con 0x7fd4b01575b0 2024-03-07T06:51:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.498+0000 7fd4b8600640 1 -- 172.21.15.133:0/608856505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b009c360 con 0x7fd4b01575b0 2024-03-07T06:51:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.500+0000 7fd4b8600640 1 -- 172.21.15.133:0/608856505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b0159eb0 con 0x7fd4b01575b0 2024-03-07T06:51:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.500+0000 7fd4ad600640 1 -- 172.21.15.133:0/608856505 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd49c01c070 con 0x7fd4b01575b0 2024-03-07T06:51:54.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.501+0000 7fd4ad600640 1 --2- 172.21.15.133:0/608856505 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd49006b500 0x7fd49006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.501+0000 7fd4ad600640 1 -- 172.21.15.133:0/608856505 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd49c089da0 con 0x7fd4b01575b0 2024-03-07T06:51:54.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.502+0000 7fd4b5c00640 1 --2- 172.21.15.133:0/608856505 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd49006b500 0x7fd49006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.503+0000 7fd4b5c00640 1 --2- 172.21.15.133:0/608856505 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd49006b500 0x7fd49006d9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4036000 tx=0x7fd4a4030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:54.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.506+0000 7fd4ad600640 1 -- 172.21.15.133:0/608856505 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd4b0159eb0 con 0x7fd4b01575b0 2024-03-07T06:51:54.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.701+0000 7fd4b8600640 1 -- 172.21.15.133:0/608856505 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4b0062c90 con 0x7fd4b01575b0 2024-03-07T06:51:54.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.703+0000 7fd4ad600640 1 -- 172.21.15.133:0/608856505 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16653 (secure 0 0 0) 0x7fd49c058b90 con 0x7fd4b01575b0 2024-03-07T06:51:54.703 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:54.703 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":21,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:16.174650+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:54.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.705+0000 7fd4b8600640 1 -- 172.21.15.133:0/608856505 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd49006b500 msgr2=0x7fd49006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.705+0000 7fd4b8600640 1 --2- 172.21.15.133:0/608856505 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd49006b500 0x7fd49006d9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4036000 tx=0x7fd4a4030000 comp rx=0 tx=0).stop 2024-03-07T06:51:54.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.705+0000 7fd4b8600640 1 -- 172.21.15.133:0/608856505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01575b0 msgr2=0x7fd4b0156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.705+0000 7fd4b8600640 1 --2- 172.21.15.133:0/608856505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01575b0 0x7fd4b0156c20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00d910 tx=0x7fd49c00ddd0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.706+0000 7fd4b8600640 1 -- 172.21.15.133:0/608856505 shutdown_connections 2024-03-07T06:51:54.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.706+0000 7fd4b8600640 1 --2- 172.21.15.133:0/608856505 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd49006b500 0x7fd49006d9c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.706+0000 7fd4b8600640 1 --2- 172.21.15.133:0/608856505 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd4b009b970 0x7fd4b009be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.706+0000 7fd4b8600640 1 --2- 172.21.15.133:0/608856505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4b01575b0 0x7fd4b0156c20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.706+0000 7fd4b8600640 1 --2- 172.21.15.133:0/608856505 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd4980039e0 0x7fd4b015e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.706+0000 7fd4b8600640 1 -- 172.21.15.133:0/608856505 >> 172.21.15.133:0/608856505 conn(0x7fd4b00b3120 msgr2=0x7fd4b0071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:54.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.706+0000 7fd4b8600640 1 -- 172.21.15.133:0/608856505 shutdown_connections 2024-03-07T06:51:54.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.706+0000 7fd4b8600640 1 -- 172.21.15.133:0/608856505 wait complete. 2024-03-07T06:51:54.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:51:12.858333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:51:13.615348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:51:54.726 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:51:54.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.937+0000 7ff399e00640 1 Processor -- start 2024-03-07T06:51:54.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.937+0000 7ff399e00640 1 -- start start 2024-03-07T06:51:54.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.939+0000 7ff399e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3940cf620 0x7ff3940b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.939+0000 7ff399e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff3940ae570 0x7ff3940ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.939+0000 7ff399e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff39406f450 con 0x7ff39406e3f0 2024-03-07T06:51:54.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.939+0000 7ff399e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3940aef10 con 0x7ff3940ae570 2024-03-07T06:51:54.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.939+0000 7ff399e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39406f170 con 0x7ff3940cf620 2024-03-07T06:51:54.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.939+0000 7ff393400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff39406e3f0 0x7ff39406e880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57054/0 (socket says 172.21.15.133:57054) 2024-03-07T06:51:54.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.939+0000 7ff393400640 1 -- 172.21.15.133:0/3902732874 learned_addr learned my addr 172.21.15.133:0/3902732874 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:54.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.939+0000 7ff392a00640 1 --2- 172.21.15.133:0/3902732874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3940cf620 0x7ff3940b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.939+0000 7ff392000640 1 --2- 172.21.15.133:0/3902732874 >> v2:172.21.15.161:3300/0 conn(0x7ff3940ae570 0x7ff3940ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.939+0000 7ff391600640 1 -- 172.21.15.133:0/3902732874 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026711680 0 0) 0x7ff39406f450 con 0x7ff39406e3f0 2024-03-07T06:51:54.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.939+0000 7ff391600640 1 -- 172.21.15.133:0/3902732874 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff378003520 con 0x7ff39406e3f0 2024-03-07T06:51:54.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff391600640 1 -- 172.21.15.133:0/3902732874 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3701328751 0 0) 0x7ff378003520 con 0x7ff39406e3f0 2024-03-07T06:51:54.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff391600640 1 -- 172.21.15.133:0/3902732874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3940cf620 msgr2=0x7ff3940b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:54.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff391600640 1 --2- 172.21.15.133:0/3902732874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3940cf620 0x7ff3940b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff391600640 1 -- 172.21.15.133:0/3902732874 >> v2:172.21.15.161:3300/0 conn(0x7ff3940ae570 msgr2=0x7ff3940ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:54.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff391600640 1 --2- 172.21.15.133:0/3902732874 >> v2:172.21.15.161:3300/0 conn(0x7ff3940ae570 0x7ff3940ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff391600640 1 -- 172.21.15.133:0/3902732874 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff394071ad0 con 0x7ff39406e3f0 2024-03-07T06:51:54.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff391600640 1 -- 172.21.15.133:0/3902732874 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7ff38c0019f0 con 0x7ff39406e3f0 2024-03-07T06:51:54.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff391600640 1 -- 172.21.15.133:0/3902732874 >> v1:172.21.15.133:6789/0 conn(0x7ff39406e3f0 legacy=0x7ff39406e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff391600640 1 --2- 172.21.15.133:0/3902732874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3940cf620 0x7ff3780039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff391600640 1 --2- 172.21.15.133:0/3902732874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff378003f30 0x7ff378004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff391600640 1 --2- 172.21.15.133:0/3902732874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff378004d40 0x7ff3780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff391600640 1 -- 172.21.15.133:0/3902732874 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39406f170 con 0x7ff378004d40 2024-03-07T06:51:54.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff391600640 1 -- 172.21.15.133:0/3902732874 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37800b9c0 con 0x7ff378003f30 2024-03-07T06:51:54.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff391600640 1 -- 172.21.15.133:0/3902732874 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37800bb90 con 0x7ff3940cf620 2024-03-07T06:51:54.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.940+0000 7ff393400640 1 --2- 172.21.15.133:0/3902732874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff378004d40 0x7ff3780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.941+0000 7ff392a00640 1 --2- 172.21.15.133:0/3902732874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3940cf620 0x7ff3780039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.941+0000 7ff392000640 1 --2- 172.21.15.133:0/3902732874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff378003f30 0x7ff378004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.941+0000 7ff393400640 1 -- 172.21.15.133:0/3902732874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3940cf620 msgr2=0x7ff3780039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:54.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.941+0000 7ff393400640 1 --2- 172.21.15.133:0/3902732874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3940cf620 0x7ff3780039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.941+0000 7ff393400640 1 -- 172.21.15.133:0/3902732874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff378003f30 msgr2=0x7ff378004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:54.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.941+0000 7ff393400640 1 --2- 172.21.15.133:0/3902732874 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff378003f30 0x7ff378004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.941+0000 7ff393400640 1 -- 172.21.15.133:0/3902732874 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff378003520 con 0x7ff378004d40 2024-03-07T06:51:54.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.942+0000 7ff393400640 1 --2- 172.21.15.133:0/3902732874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff378004d40 0x7ff3780051c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00a8d0 tx=0x7ff378005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39120c34bb776429 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:54.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.942+0000 7ff391600640 1 -- 172.21.15.133:0/3902732874 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c00a4c0 con 0x7ff378004d40 2024-03-07T06:51:54.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.942+0000 7ff391600640 1 -- 172.21.15.133:0/3902732874 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38c034650 con 0x7ff378004d40 2024-03-07T06:51:54.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.942+0000 7ff391600640 1 -- 172.21.15.133:0/3902732874 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c034a80 con 0x7ff378004d40 2024-03-07T06:51:54.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.943+0000 7ff399e00640 1 -- 172.21.15.133:0/3902732874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff378004d40 msgr2=0x7ff3780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.943+0000 7ff399e00640 1 --2- 172.21.15.133:0/3902732874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff378004d40 0x7ff3780051c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00a8d0 tx=0x7ff378005830 comp rx=0 tx=0).stop 2024-03-07T06:51:54.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.943+0000 7ff393400640 1 -- 172.21.15.133:0/3902732874 reap_dead start 2024-03-07T06:51:54.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.943+0000 7ff399e00640 1 -- 172.21.15.133:0/3902732874 shutdown_connections 2024-03-07T06:51:54.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.943+0000 7ff399e00640 1 -- 172.21.15.133:0/3902732874 >> 172.21.15.133:0/3902732874 conn(0x7ff3940b1d30 msgr2=0x7ff3940b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:54.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.943+0000 7ff399e00640 1 -- 172.21.15.133:0/3902732874 shutdown_connections 2024-03-07T06:51:54.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.943+0000 7ff399e00640 1 -- 172.21.15.133:0/3902732874 wait complete. 2024-03-07T06:51:54.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.945+0000 7ff399e00640 1 Processor -- start 2024-03-07T06:51:54.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.945+0000 7ff399e00640 1 -- start start 2024-03-07T06:51:54.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff399e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3941f0f40 0x7ff3941f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff399e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3941f1320 0x7ff39413ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff399e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3940b3f80 0x7ff3940b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff399e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3940b2e50 con 0x7ff3941f0f40 2024-03-07T06:51:54.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff399e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff394072090 con 0x7ff3940b3f80 2024-03-07T06:51:54.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff399e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3940b2fd0 con 0x7ff3941f1320 2024-03-07T06:51:54.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff392a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3941f0f40 0x7ff3941f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff392a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3941f0f40 0x7ff3941f4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37188/0 (socket says 172.21.15.133:37188) 2024-03-07T06:51:54.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff392a00640 1 -- 172.21.15.133:0/2981264518 learned_addr learned my addr 172.21.15.133:0/2981264518 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:54.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff392a00640 1 -- 172.21.15.133:0/2981264518 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3941f1320 msgr2=0x7ff39413ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff392a00640 1 --2- 172.21.15.133:0/2981264518 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3941f1320 0x7ff39413ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff393400640 1 --2- 172.21.15.133:0/2981264518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3940b3f80 0x7ff3940b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff392a00640 1 -- 172.21.15.133:0/2981264518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3940b3f80 msgr2=0x7ff3940b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:54.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff392a00640 1 --2- 172.21.15.133:0/2981264518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3940b3f80 0x7ff3940b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:54.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.946+0000 7ff392a00640 1 -- 172.21.15.133:0/2981264518 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39413d360 con 0x7ff3941f0f40 2024-03-07T06:51:54.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.947+0000 7ff392a00640 1 --2- 172.21.15.133:0/2981264518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3941f0f40 0x7ff3941f4620 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff384011600 tx=0x7ff384011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:54.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.947+0000 7ff38b400640 1 -- 172.21.15.133:0/2981264518 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38400f920 con 0x7ff3941f0f40 2024-03-07T06:51:54.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.947+0000 7ff38b400640 1 -- 172.21.15.133:0/2981264518 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38400fac0 con 0x7ff3941f0f40 2024-03-07T06:51:54.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.947+0000 7ff399e00640 1 -- 172.21.15.133:0/2981264518 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3941f7550 con 0x7ff3941f0f40 2024-03-07T06:51:54.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.947+0000 7ff38b400640 1 -- 172.21.15.133:0/2981264518 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38400fe70 con 0x7ff3941f0f40 2024-03-07T06:51:54.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.947+0000 7ff399e00640 1 -- 172.21.15.133:0/2981264518 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3941f7b10 con 0x7ff3941f0f40 2024-03-07T06:51:54.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.948+0000 7ff399e00640 1 -- 172.21.15.133:0/2981264518 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff394062c90 con 0x7ff3941f0f40 2024-03-07T06:51:54.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.949+0000 7ff38b400640 1 -- 172.21.15.133:0/2981264518 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff384015460 con 0x7ff3941f0f40 2024-03-07T06:51:54.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.950+0000 7ff38b400640 1 --2- 172.21.15.133:0/2981264518 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff37406b4d0 0x7ff37406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:54.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.951+0000 7ff38b400640 1 -- 172.21.15.133:0/2981264518 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff38408ab60 con 0x7ff3941f0f40 2024-03-07T06:51:54.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.951+0000 7ff392000640 1 --2- 172.21.15.133:0/2981264518 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff37406b4d0 0x7ff37406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:54.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.952+0000 7ff392000640 1 --2- 172.21.15.133:0/2981264518 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff37406b4d0 0x7ff37406d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff3941f1700 tx=0x7ff380008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:54.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:54.955+0000 7ff38b400640 1 -- 172.21.15.133:0/2981264518 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff384019030 con 0x7ff3941f0f40 2024-03-07T06:51:55.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.151+0000 7ff399e00640 1 -- 172.21.15.133:0/2981264518 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3940ae350 con 0x7ff3941f0f40 2024-03-07T06:51:55.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.152+0000 7ff38b400640 1 -- 172.21.15.133:0/2981264518 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff384059a70 con 0x7ff3941f0f40 2024-03-07T06:51:55.152 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:51:55.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.155+0000 7ff399e00640 1 -- 172.21.15.133:0/2981264518 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff37406b4d0 msgr2=0x7ff37406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:55.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.155+0000 7ff399e00640 1 --2- 172.21.15.133:0/2981264518 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff37406b4d0 0x7ff37406d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff3941f1700 tx=0x7ff380008040 comp rx=0 tx=0).stop 2024-03-07T06:51:55.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.155+0000 7ff399e00640 1 -- 172.21.15.133:0/2981264518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3941f0f40 msgr2=0x7ff3941f4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:55.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.155+0000 7ff399e00640 1 --2- 172.21.15.133:0/2981264518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3941f0f40 0x7ff3941f4620 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff384011600 tx=0x7ff384011ac0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.155+0000 7ff399e00640 1 -- 172.21.15.133:0/2981264518 shutdown_connections 2024-03-07T06:51:55.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.155+0000 7ff399e00640 1 --2- 172.21.15.133:0/2981264518 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff37406b4d0 0x7ff37406d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.155+0000 7ff399e00640 1 --2- 172.21.15.133:0/2981264518 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3940b3f80 0x7ff3940b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.155+0000 7ff399e00640 1 --2- 172.21.15.133:0/2981264518 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3941f1320 0x7ff39413ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.155+0000 7ff399e00640 1 --2- 172.21.15.133:0/2981264518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff3941f0f40 0x7ff3941f4620 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.155+0000 7ff399e00640 1 -- 172.21.15.133:0/2981264518 >> 172.21.15.133:0/2981264518 conn(0x7ff3940b1d30 msgr2=0x7ff394072400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:55.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.155+0000 7ff399e00640 1 -- 172.21.15.133:0/2981264518 shutdown_connections 2024-03-07T06:51:55.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.155+0000 7ff399e00640 1 -- 172.21.15.133:0/2981264518 wait complete. 2024-03-07T06:51:55.173 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:51:55.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.373+0000 7f2208600640 1 Processor -- start 2024-03-07T06:51:55.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.374+0000 7f2208600640 1 -- start start 2024-03-07T06:51:55.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.375+0000 7f2208600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2200145670 0x7f22000cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:55.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.375+0000 7f2208600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220006e150 con 0x7f22000cdbf0 2024-03-07T06:51:55.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.375+0000 7f2208600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220006e430 con 0x7f220015cf90 2024-03-07T06:51:55.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.375+0000 7f2208600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220006ed60 con 0x7f2200145670 2024-03-07T06:51:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.376+0000 7f2205c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2200145670 0x7f22000cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.376+0000 7f2206600640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f22000cdbf0 0x7f22000719a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57064/0 (socket says 172.21.15.133:57064) 2024-03-07T06:51:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.376+0000 7f2205c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2200145670 0x7f22000cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37204/0 (socket says 172.21.15.133:37204) 2024-03-07T06:51:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.376+0000 7f2206600640 1 -- 172.21.15.133:0/1202301577 learned_addr learned my addr 172.21.15.133:0/1202301577 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.376+0000 7f2205c00640 1 -- 172.21.15.133:0/1202301577 >> v1:172.21.15.161:6789/0 conn(0x7f220015cf90 legacy=0x7f22000cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.376+0000 7f2205c00640 1 -- 172.21.15.133:0/1202301577 >> v1:172.21.15.133:6789/0 conn(0x7f22000cdbf0 legacy=0x7f22000719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:55.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.376+0000 7f2205c00640 1 -- 172.21.15.133:0/1202301577 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220006e710 con 0x7f2200145670 2024-03-07T06:51:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.377+0000 7f2205c00640 1 --2- 172.21.15.133:0/1202301577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2200145670 0x7f22000cf610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f21f40097d0 tx=0x7f21f402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f42c69e14f19b92 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.378+0000 7f21ffe00640 1 -- 172.21.15.133:0/1202301577 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4035020 con 0x7f2200145670 2024-03-07T06:51:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.378+0000 7f21ffe00640 1 -- 172.21.15.133:0/1202301577 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f4031810 con 0x7f2200145670 2024-03-07T06:51:55.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.378+0000 7f21ffe00640 1 -- 172.21.15.133:0/1202301577 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4031be0 con 0x7f2200145670 2024-03-07T06:51:55.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.378+0000 7f2208600640 1 -- 172.21.15.133:0/1202301577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2200145670 msgr2=0x7f22000cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:55.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.378+0000 7f2208600640 1 --2- 172.21.15.133:0/1202301577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2200145670 0x7f22000cf610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f21f40097d0 tx=0x7f21f402b8b0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.378+0000 7f2208600640 1 -- 172.21.15.133:0/1202301577 shutdown_connections 2024-03-07T06:51:55.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.378+0000 7f2208600640 1 --2- 172.21.15.133:0/1202301577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2200145670 0x7f22000cf610 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.378+0000 7f2208600640 1 -- 172.21.15.133:0/1202301577 >> 172.21.15.133:0/1202301577 conn(0x7f22000b1d30 msgr2=0x7f22000b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:55.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.378+0000 7f2208600640 1 -- 172.21.15.133:0/1202301577 shutdown_connections 2024-03-07T06:51:55.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.378+0000 7f2208600640 1 -- 172.21.15.133:0/1202301577 wait complete. 2024-03-07T06:51:55.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.380+0000 7f2208600640 1 Processor -- start 2024-03-07T06:51:55.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.380+0000 7f2208600640 1 -- start start 2024-03-07T06:51:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.381+0000 7f2208600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22000cdbf0 0x7f22000cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.381+0000 7f2208600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2200145670 0x7f22001304b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.381+0000 7f2208600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f220015cf90 0x7f22001309f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:55.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.381+0000 7f2208600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2200145a50 con 0x7f220015cf90 2024-03-07T06:51:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.381+0000 7f2208600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220015d420 con 0x7f22000cdbf0 2024-03-07T06:51:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.381+0000 7f2208600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2200145bd0 con 0x7f2200145670 2024-03-07T06:51:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.381+0000 7f2206600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f220015cf90 0x7f22001309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.381+0000 7f2206600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f220015cf90 0x7f22001309f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37206/0 (socket says 172.21.15.133:37206) 2024-03-07T06:51:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.381+0000 7f2206600640 1 -- 172.21.15.133:0/2427948715 learned_addr learned my addr 172.21.15.133:0/2427948715 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.382+0000 7f2205c00640 1 --2- 172.21.15.133:0/2427948715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22000cdbf0 0x7f22000cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.382+0000 7f2205200640 1 --2- 172.21.15.133:0/2427948715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2200145670 0x7f22001304b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.382+0000 7f2206600640 1 -- 172.21.15.133:0/2427948715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2200145670 msgr2=0x7f22001304b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.382+0000 7f2206600640 1 --2- 172.21.15.133:0/2427948715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2200145670 0x7f22001304b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.382+0000 7f2206600640 1 -- 172.21.15.133:0/2427948715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22000cdbf0 msgr2=0x7f22000cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.382+0000 7f2206600640 1 --2- 172.21.15.133:0/2427948715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22000cdbf0 0x7f22000cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.382+0000 7f2206600640 1 -- 172.21.15.133:0/2427948715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2200131130 con 0x7f220015cf90 2024-03-07T06:51:55.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.382+0000 7f2206600640 1 --2- 172.21.15.133:0/2427948715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f220015cf90 0x7f22001309f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f21f800caa0 tx=0x7f21f800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:55.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.382+0000 7f21fd600640 1 -- 172.21.15.133:0/2427948715 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8017070 con 0x7f220015cf90 2024-03-07T06:51:55.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.382+0000 7f21fd600640 1 -- 172.21.15.133:0/2427948715 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f800ac00 con 0x7f220015cf90 2024-03-07T06:51:55.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.382+0000 7f2208600640 1 -- 172.21.15.133:0/2427948715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2200131410 con 0x7f220015cf90 2024-03-07T06:51:55.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.382+0000 7f21fd600640 1 -- 172.21.15.133:0/2427948715 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8012400 con 0x7f220015cf90 2024-03-07T06:51:55.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.383+0000 7f2208600640 1 -- 172.21.15.133:0/2427948715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2200131cb0 con 0x7f220015cf90 2024-03-07T06:51:55.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.384+0000 7f2208600640 1 -- 172.21.15.133:0/2427948715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2200131800 con 0x7f220015cf90 2024-03-07T06:51:55.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.385+0000 7f21fd600640 1 -- 172.21.15.133:0/2427948715 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f21f801a070 con 0x7f220015cf90 2024-03-07T06:51:55.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.385+0000 7f21fd600640 1 --2- 172.21.15.133:0/2427948715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f21e406b4d0 0x7f21e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:55.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.386+0000 7f21fd600640 1 -- 172.21.15.133:0/2427948715 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21f8089e60 con 0x7f220015cf90 2024-03-07T06:51:55.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.386+0000 7f2205c00640 1 --2- 172.21.15.133:0/2427948715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f21e406b4d0 0x7f21e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:55.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.387+0000 7f2205c00640 1 --2- 172.21.15.133:0/2427948715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f21e406b4d0 0x7f21e406d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f21f4036000 tx=0x7f21f403c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:55.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.390+0000 7f21fd600640 1 -- 172.21.15.133:0/2427948715 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f21f80585f0 con 0x7f220015cf90 2024-03-07T06:51:55.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.593+0000 7f2208600640 1 -- 172.21.15.133:0/2427948715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2200062c90 con 0x7f220015cf90 2024-03-07T06:51:55.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.594+0000 7f21fd600640 1 -- 172.21.15.133:0/2427948715 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f21f805c690 con 0x7f220015cf90 2024-03-07T06:51:55.594 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:51:55.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.597+0000 7f2208600640 1 -- 172.21.15.133:0/2427948715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f21e406b4d0 msgr2=0x7f21e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:55.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.597+0000 7f2208600640 1 --2- 172.21.15.133:0/2427948715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f21e406b4d0 0x7f21e406d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f21f4036000 tx=0x7f21f403c040 comp rx=0 tx=0).stop 2024-03-07T06:51:55.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.597+0000 7f2208600640 1 -- 172.21.15.133:0/2427948715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f220015cf90 msgr2=0x7f22001309f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:55.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.597+0000 7f2208600640 1 --2- 172.21.15.133:0/2427948715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f220015cf90 0x7f22001309f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f21f800caa0 tx=0x7f21f800cf60 comp rx=0 tx=0).stop 2024-03-07T06:51:55.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.597+0000 7f2208600640 1 -- 172.21.15.133:0/2427948715 shutdown_connections 2024-03-07T06:51:55.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.597+0000 7f2208600640 1 --2- 172.21.15.133:0/2427948715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f220015cf90 0x7f22001309f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.597+0000 7f2208600640 1 --2- 172.21.15.133:0/2427948715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2200145670 0x7f22001304b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.597+0000 7f2208600640 1 --2- 172.21.15.133:0/2427948715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f21e406b4d0 0x7f21e406d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.597+0000 7f2208600640 1 --2- 172.21.15.133:0/2427948715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22000cdbf0 0x7f22000cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.597+0000 7f2208600640 1 -- 172.21.15.133:0/2427948715 >> 172.21.15.133:0/2427948715 conn(0x7f22000b1d30 msgr2=0x7f220015d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:55.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.597+0000 7f2208600640 1 -- 172.21.15.133:0/2427948715 shutdown_connections 2024-03-07T06:51:55.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.598+0000 7f2208600640 1 -- 172.21.15.133:0/2427948715 wait complete. 2024-03-07T06:51:55.615 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:51:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.822+0000 7f19da800640 1 Processor -- start 2024-03-07T06:51:55.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.823+0000 7f19da800640 1 -- start start 2024-03-07T06:51:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.824+0000 7f19da800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19d4145670 0x7f19d40cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.824+0000 7f19da800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f19d415cf90 0x7f19d40cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.824+0000 7f19da800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d406e150 con 0x7f19d40cda20 2024-03-07T06:51:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.824+0000 7f19da800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d406ed60 con 0x7f19d415cf90 2024-03-07T06:51:55.824 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.824+0000 7f19da800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d406e430 con 0x7f19d4145670 2024-03-07T06:51:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.825+0000 7f19d3400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19d4145670 0x7f19d40cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.825+0000 7f19d3400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19d4145670 0x7f19d40cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:60050/0 (socket says 172.21.15.133:60050) 2024-03-07T06:51:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.825+0000 7f19d3400640 1 -- 172.21.15.133:0/3873579196 learned_addr learned my addr 172.21.15.133:0/3873579196 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.825+0000 7f19caa00640 1 --2- 172.21.15.133:0/3873579196 >> v2:172.21.15.161:3300/0 conn(0x7f19d415cf90 0x7f19d40cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.825+0000 7f19d3400640 1 -- 172.21.15.133:0/3873579196 >> v2:172.21.15.161:3300/0 conn(0x7f19d415cf90 msgr2=0x7f19d40cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.825+0000 7f19d3400640 1 --2- 172.21.15.133:0/3873579196 >> v2:172.21.15.161:3300/0 conn(0x7f19d415cf90 0x7f19d40cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.825+0000 7f19d3400640 1 -- 172.21.15.133:0/3873579196 >> v1:172.21.15.161:6789/0 conn(0x7f19d40cda20 legacy=0x7f19d415fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:55.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.825+0000 7f19d3400640 1 -- 172.21.15.133:0/3873579196 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d406e710 con 0x7f19d4145670 2024-03-07T06:51:55.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.827+0000 7f19d3400640 1 --2- 172.21.15.133:0/3873579196 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19d4145670 0x7f19d40cf610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009870 tx=0x7f19c402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71901a86113bf977 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:55.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.827+0000 7f19d2a00640 1 -- 172.21.15.133:0/3873579196 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4035020 con 0x7f19d4145670 2024-03-07T06:51:55.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.827+0000 7f19d2a00640 1 -- 172.21.15.133:0/3873579196 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c4031640 con 0x7f19d4145670 2024-03-07T06:51:55.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.827+0000 7f19d2a00640 1 -- 172.21.15.133:0/3873579196 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c402d040 con 0x7f19d4145670 2024-03-07T06:51:55.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.828+0000 7f19da800640 1 -- 172.21.15.133:0/3873579196 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19d4145670 msgr2=0x7f19d40cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:55.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.828+0000 7f19da800640 1 --2- 172.21.15.133:0/3873579196 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19d4145670 0x7f19d40cf610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009870 tx=0x7f19c402b8f0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.828+0000 7f19da800640 1 -- 172.21.15.133:0/3873579196 shutdown_connections 2024-03-07T06:51:55.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.828+0000 7f19da800640 1 --2- 172.21.15.133:0/3873579196 >> v2:172.21.15.161:3300/0 conn(0x7f19d415cf90 0x7f19d40cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.828+0000 7f19da800640 1 --2- 172.21.15.133:0/3873579196 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19d4145670 0x7f19d40cf610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.828+0000 7f19da800640 1 -- 172.21.15.133:0/3873579196 >> 172.21.15.133:0/3873579196 conn(0x7f19d40b1d30 msgr2=0x7f19d40b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:55.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.828+0000 7f19da800640 1 -- 172.21.15.133:0/3873579196 shutdown_connections 2024-03-07T06:51:55.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.828+0000 7f19da800640 1 -- 172.21.15.133:0/3873579196 wait complete. 2024-03-07T06:51:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.830+0000 7f19da800640 1 Processor -- start 2024-03-07T06:51:55.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.830+0000 7f19da800640 1 -- start start 2024-03-07T06:51:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.831+0000 7f19da800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19d40cda20 0x7f19d41f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.831+0000 7f19da800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19d4145670 0x7f19d413fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.831+0000 7f19da800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19d415cf90 0x7f19d40ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.831+0000 7f19da800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4145a50 con 0x7f19d415cf90 2024-03-07T06:51:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.831+0000 7f19da800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d415d3f0 con 0x7f19d40cda20 2024-03-07T06:51:55.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.831+0000 7f19da800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4145bd0 con 0x7f19d4145670 2024-03-07T06:51:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.832+0000 7f19d3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19d415cf90 0x7f19d40ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.832+0000 7f19d3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19d415cf90 0x7f19d40ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37222/0 (socket says 172.21.15.133:37222) 2024-03-07T06:51:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.832+0000 7f19d3e00640 1 -- 172.21.15.133:0/903663075 learned_addr learned my addr 172.21.15.133:0/903663075 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.832+0000 7f19d3400640 1 --2- 172.21.15.133:0/903663075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19d40cda20 0x7f19d41f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.832+0000 7f19caa00640 1 --2- 172.21.15.133:0/903663075 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19d4145670 0x7f19d413fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.832+0000 7f19d3e00640 1 -- 172.21.15.133:0/903663075 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19d4145670 msgr2=0x7f19d413fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.832+0000 7f19d3e00640 1 --2- 172.21.15.133:0/903663075 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19d4145670 0x7f19d413fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.832+0000 7f19d3e00640 1 -- 172.21.15.133:0/903663075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19d40cda20 msgr2=0x7f19d41f15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.832+0000 7f19d3e00640 1 --2- 172.21.15.133:0/903663075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19d40cda20 0x7f19d41f15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.832+0000 7f19d3e00640 1 -- 172.21.15.133:0/903663075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d41f1ae0 con 0x7f19d415cf90 2024-03-07T06:51:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.832+0000 7f19d3e00640 1 --2- 172.21.15.133:0/903663075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19d415cf90 0x7f19d40ca840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00c730 tx=0x7f19cc00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:55.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.833+0000 7f19cbe00640 1 -- 172.21.15.133:0/903663075 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc016020 con 0x7f19d415cf90 2024-03-07T06:51:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.833+0000 7f19cbe00640 1 -- 172.21.15.133:0/903663075 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc006d50 con 0x7f19d415cf90 2024-03-07T06:51:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.833+0000 7f19cbe00640 1 -- 172.21.15.133:0/903663075 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc003bb0 con 0x7f19d415cf90 2024-03-07T06:51:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.833+0000 7f19da800640 1 -- 172.21.15.133:0/903663075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d40c7d70 con 0x7f19d415cf90 2024-03-07T06:51:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.833+0000 7f19da800640 1 -- 172.21.15.133:0/903663075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d40c85b0 con 0x7f19d415cf90 2024-03-07T06:51:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.834+0000 7f19da800640 1 -- 172.21.15.133:0/903663075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d4062c90 con 0x7f19d415cf90 2024-03-07T06:51:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.835+0000 7f19cbe00640 1 -- 172.21.15.133:0/903663075 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f19cc0050d0 con 0x7f19d415cf90 2024-03-07T06:51:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.836+0000 7f19cbe00640 1 --2- 172.21.15.133:0/903663075 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f19b006b4d0 0x7f19b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:55.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.837+0000 7f19cbe00640 1 -- 172.21.15.133:0/903663075 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f19cc089e20 con 0x7f19d415cf90 2024-03-07T06:51:55.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.837+0000 7f19d3400640 1 --2- 172.21.15.133:0/903663075 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f19b006b4d0 0x7f19b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:55.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.838+0000 7f19d3400640 1 --2- 172.21.15.133:0/903663075 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f19b006b4d0 0x7f19b006d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f19c4036000 tx=0x7f19c403c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:55.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:55.841+0000 7f19cbe00640 1 -- 172.21.15.133:0/903663075 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f19cc020020 con 0x7f19d415cf90 2024-03-07T06:51:56.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.036+0000 7f19da800640 1 -- 172.21.15.133:0/903663075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f19d41403a0 con 0x7f19d415cf90 2024-03-07T06:51:56.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.036+0000 7f19cbe00640 1 -- 172.21.15.133:0/903663075 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f19d40c7d70 con 0x7f19d415cf90 2024-03-07T06:51:56.036 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:51:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.039+0000 7f19da800640 1 -- 172.21.15.133:0/903663075 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f19b006b4d0 msgr2=0x7f19b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.039+0000 7f19da800640 1 --2- 172.21.15.133:0/903663075 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f19b006b4d0 0x7f19b006d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f19c4036000 tx=0x7f19c403c040 comp rx=0 tx=0).stop 2024-03-07T06:51:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.039+0000 7f19da800640 1 -- 172.21.15.133:0/903663075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19d415cf90 msgr2=0x7f19d40ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.039+0000 7f19da800640 1 --2- 172.21.15.133:0/903663075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19d415cf90 0x7f19d40ca840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f19cc00c730 tx=0x7f19cc00cb30 comp rx=0 tx=0).stop 2024-03-07T06:51:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.039+0000 7f19da800640 1 -- 172.21.15.133:0/903663075 shutdown_connections 2024-03-07T06:51:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.039+0000 7f19da800640 1 --2- 172.21.15.133:0/903663075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f19d415cf90 0x7f19d40ca840 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.039+0000 7f19da800640 1 --2- 172.21.15.133:0/903663075 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f19d4145670 0x7f19d413fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.039+0000 7f19da800640 1 --2- 172.21.15.133:0/903663075 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f19b006b4d0 0x7f19b006d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.039+0000 7f19da800640 1 --2- 172.21.15.133:0/903663075 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f19d40cda20 0x7f19d41f15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.039+0000 7f19da800640 1 -- 172.21.15.133:0/903663075 >> 172.21.15.133:0/903663075 conn(0x7f19d40b1d30 msgr2=0x7f19d415c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:56.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.039+0000 7f19da800640 1 -- 172.21.15.133:0/903663075 shutdown_connections 2024-03-07T06:51:56.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.039+0000 7f19da800640 1 -- 172.21.15.133:0/903663075 wait complete. 2024-03-07T06:51:56.059 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.263+0000 7fbca7e00640 1 Processor -- start 2024-03-07T06:51:56.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.263+0000 7fbca7e00640 1 -- start start 2024-03-07T06:51:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.264+0000 7fbca7e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbca815bd30 0x7fbca806e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.264+0000 7fbca7e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca8160760 con 0x7fbca8072f40 2024-03-07T06:51:56.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.264+0000 7fbca7e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca815f9a0 con 0x7fbca80cf5d0 2024-03-07T06:51:56.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.264+0000 7fbca7e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca815fc80 con 0x7fbca815bd30 2024-03-07T06:51:56.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.265+0000 7fbca7400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fbca8072f40 0x7fbca815c110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57078/0 (socket says 172.21.15.133:57078) 2024-03-07T06:51:56.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.265+0000 7fbca7400640 1 -- 172.21.15.133:0/1632599189 learned_addr learned my addr 172.21.15.133:0/1632599189 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:56.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.265+0000 7fbca6a00640 1 --2- 172.21.15.133:0/1632599189 >> v2:172.21.15.161:3300/0 conn(0x7fbca815bd30 0x7fbca806e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:56.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.265+0000 7fbca5600640 1 -- 172.21.15.133:0/1632599189 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694893763 0 0) 0x7fbca8160760 con 0x7fbca8072f40 2024-03-07T06:51:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.265+0000 7fbca5600640 1 -- 172.21.15.133:0/1632599189 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc8c003520 con 0x7fbca8072f40 2024-03-07T06:51:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca5600640 1 -- 172.21.15.133:0/1632599189 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 91566870 0 0) 0x7fbc8c003520 con 0x7fbca8072f40 2024-03-07T06:51:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca5600640 1 -- 172.21.15.133:0/1632599189 >> v2:172.21.15.161:3300/0 conn(0x7fbca815bd30 msgr2=0x7fbca806e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca5600640 1 --2- 172.21.15.133:0/1632599189 >> v2:172.21.15.161:3300/0 conn(0x7fbca815bd30 0x7fbca806e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca5600640 1 -- 172.21.15.133:0/1632599189 >> v1:172.21.15.161:6789/0 conn(0x7fbca80cf5d0 legacy=0x7fbca8072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca5600640 1 -- 172.21.15.133:0/1632599189 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca815ff60 con 0x7fbca8072f40 2024-03-07T06:51:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca5600640 1 -- 172.21.15.133:0/1632599189 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fbca00019f0 con 0x7fbca8072f40 2024-03-07T06:51:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca5600640 1 -- 172.21.15.133:0/1632599189 >> v1:172.21.15.133:6789/0 conn(0x7fbca8072f40 legacy=0x7fbca815c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca5600640 1 --2- 172.21.15.133:0/1632599189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc8c0039b0 0x7fbc8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca5600640 1 --2- 172.21.15.133:0/1632599189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8c004800 0x7fbc8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca5600640 1 --2- 172.21.15.133:0/1632599189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8c0051c0 0x7fbc8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca5600640 1 -- 172.21.15.133:0/1632599189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca815fc80 con 0x7fbc8c004800 2024-03-07T06:51:56.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca5600640 1 -- 172.21.15.133:0/1632599189 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c00c060 con 0x7fbc8c0051c0 2024-03-07T06:51:56.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca5600640 1 -- 172.21.15.133:0/1632599189 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8c00c260 con 0x7fbc8c0039b0 2024-03-07T06:51:56.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca6000640 1 --2- 172.21.15.133:0/1632599189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8c004800 0x7fbc8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:56.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca6a00640 1 --2- 172.21.15.133:0/1632599189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc8c0039b0 0x7fbc8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:56.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.266+0000 7fbca7400640 1 --2- 172.21.15.133:0/1632599189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8c0051c0 0x7fbc8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:56.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.267+0000 7fbca6000640 1 -- 172.21.15.133:0/1632599189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc8c0039b0 msgr2=0x7fbc8c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:56.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.267+0000 7fbca6000640 1 --2- 172.21.15.133:0/1632599189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc8c0039b0 0x7fbc8c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.267+0000 7fbca6000640 1 -- 172.21.15.133:0/1632599189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8c0051c0 msgr2=0x7fbc8c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:56.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.267+0000 7fbca6000640 1 --2- 172.21.15.133:0/1632599189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8c0051c0 0x7fbc8c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.267+0000 7fbca6000640 1 -- 172.21.15.133:0/1632599189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8c003520 con 0x7fbc8c004800 2024-03-07T06:51:56.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.267+0000 7fbca7400640 1 -- 172.21.15.133:0/1632599189 reap_dead start 2024-03-07T06:51:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.268+0000 7fbca6000640 1 --2- 172.21.15.133:0/1632599189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8c004800 0x7fbc8c004c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbc94001750 tx=0x7fbc8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70dd40a2d809d422 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.268+0000 7fbca5600640 1 -- 172.21.15.133:0/1632599189 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94001370 con 0x7fbc8c004800 2024-03-07T06:51:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.268+0000 7fbca5600640 1 -- 172.21.15.133:0/1632599189 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc940052d0 con 0x7fbc8c004800 2024-03-07T06:51:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.268+0000 7fbca5600640 1 -- 172.21.15.133:0/1632599189 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94003830 con 0x7fbc8c004800 2024-03-07T06:51:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.268+0000 7fbca7e00640 1 -- 172.21.15.133:0/1632599189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8c004800 msgr2=0x7fbc8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.268+0000 7fbca7e00640 1 --2- 172.21.15.133:0/1632599189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8c004800 0x7fbc8c004c80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbc94001750 tx=0x7fbc8c005830 comp rx=0 tx=0).stop 2024-03-07T06:51:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.269+0000 7fbca7e00640 1 -- 172.21.15.133:0/1632599189 shutdown_connections 2024-03-07T06:51:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.269+0000 7fbca7e00640 1 --2- 172.21.15.133:0/1632599189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbc8c004800 0x7fbc8c004c80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.269+0000 7fbca7e00640 1 -- 172.21.15.133:0/1632599189 >> 172.21.15.133:0/1632599189 conn(0x7fbca80b2e90 msgr2=0x7fbca80b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.269+0000 7fbca7e00640 1 -- 172.21.15.133:0/1632599189 shutdown_connections 2024-03-07T06:51:56.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.269+0000 7fbca7e00640 1 -- 172.21.15.133:0/1632599189 wait complete. 2024-03-07T06:51:56.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.270+0000 7fbca7e00640 1 Processor -- start 2024-03-07T06:51:56.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.270+0000 7fbca7e00640 1 -- start start 2024-03-07T06:51:56.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.271+0000 7fbca7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8c004800 0x7fbca814c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.271+0000 7fbca7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbca814b7c0 0x7fbca8152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.271+0000 7fbca7e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbca8156f30 0x7fbca81573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.271+0000 7fbca7e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8072830 con 0x7fbca814b7c0 2024-03-07T06:51:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.271+0000 7fbca7e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca806f130 con 0x7fbc8c004800 2024-03-07T06:51:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.271+0000 7fbca7e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80729b0 con 0x7fbca8156f30 2024-03-07T06:51:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.272+0000 7fbca6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbca814b7c0 0x7fbca8152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.272+0000 7fbca6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbca814b7c0 0x7fbca8152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37246/0 (socket says 172.21.15.133:37246) 2024-03-07T06:51:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.272+0000 7fbca6000640 1 -- 172.21.15.133:0/3620792760 learned_addr learned my addr 172.21.15.133:0/3620792760 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.272+0000 7fbca7400640 1 --2- 172.21.15.133:0/3620792760 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbca8156f30 0x7fbca81573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.272+0000 7fbca6000640 1 -- 172.21.15.133:0/3620792760 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbca8156f30 msgr2=0x7fbca81573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.272+0000 7fbca6000640 1 --2- 172.21.15.133:0/3620792760 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbca8156f30 0x7fbca81573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.272+0000 7fbca6000640 1 -- 172.21.15.133:0/3620792760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8c004800 msgr2=0x7fbca814c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:56.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.272+0000 7fbca6000640 1 --2- 172.21.15.133:0/3620792760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8c004800 0x7fbca814c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.272+0000 7fbca6000640 1 -- 172.21.15.133:0/3620792760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca8153310 con 0x7fbca814b7c0 2024-03-07T06:51:56.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.272+0000 7fbca6000640 1 --2- 172.21.15.133:0/3620792760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbca814b7c0 0x7fbca8152dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbc94000c00 tx=0x7fbc94001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:56.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.273+0000 7fbc9ea00640 1 -- 172.21.15.133:0/3620792760 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94041020 con 0x7fbca814b7c0 2024-03-07T06:51:56.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.273+0000 7fbc9ea00640 1 -- 172.21.15.133:0/3620792760 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc94005480 con 0x7fbca814b7c0 2024-03-07T06:51:56.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.273+0000 7fbc9ea00640 1 -- 172.21.15.133:0/3620792760 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc94007a30 con 0x7fbca814b7c0 2024-03-07T06:51:56.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.273+0000 7fbca7e00640 1 -- 172.21.15.133:0/3620792760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca814bc40 con 0x7fbca814b7c0 2024-03-07T06:51:56.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.273+0000 7fbca7e00640 1 -- 172.21.15.133:0/3620792760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca815c500 con 0x7fbca814b7c0 2024-03-07T06:51:56.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.274+0000 7fbca7e00640 1 -- 172.21.15.133:0/3620792760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca8062c90 con 0x7fbca814b7c0 2024-03-07T06:51:56.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.275+0000 7fbc9ea00640 1 -- 172.21.15.133:0/3620792760 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbc94044020 con 0x7fbca814b7c0 2024-03-07T06:51:56.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.275+0000 7fbc9ea00640 1 --2- 172.21.15.133:0/3620792760 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbc8806b500 0x7fbc8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.276+0000 7fbc9ea00640 1 -- 172.21.15.133:0/3620792760 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fbc940abcb0 con 0x7fbca814b7c0 2024-03-07T06:51:56.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.276+0000 7fbca6a00640 1 --2- 172.21.15.133:0/3620792760 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbc8806b500 0x7fbc8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:56.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.277+0000 7fbca6a00640 1 --2- 172.21.15.133:0/3620792760 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbc8806b500 0x7fbc8806d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbca81f2f10 tx=0x7fbc98008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:56.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.280+0000 7fbc9ea00640 1 -- 172.21.15.133:0/3620792760 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbc9404b030 con 0x7fbca814b7c0 2024-03-07T06:51:56.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.488+0000 7fbca7e00640 1 -- 172.21.15.133:0/3620792760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbca8160480 con 0x7fbca814b7c0 2024-03-07T06:51:56.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.491+0000 7fbc9ea00640 1 -- 172.21.15.133:0/3620792760 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16901 (secure 0 0 0) 0x7fbc940303c0 con 0x7fbca814b7c0 2024-03-07T06:51:56.492 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:56.492 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:56.413096+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:56.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.494+0000 7fbca7e00640 1 -- 172.21.15.133:0/3620792760 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbc8806b500 msgr2=0x7fbc8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:56.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.494+0000 7fbca7e00640 1 --2- 172.21.15.133:0/3620792760 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbc8806b500 0x7fbc8806d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbca81f2f10 tx=0x7fbc98008040 comp rx=0 tx=0).stop 2024-03-07T06:51:56.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.494+0000 7fbca7e00640 1 -- 172.21.15.133:0/3620792760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbca814b7c0 msgr2=0x7fbca8152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:56.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.494+0000 7fbca7e00640 1 --2- 172.21.15.133:0/3620792760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbca814b7c0 0x7fbca8152dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbc94000c00 tx=0x7fbc94001120 comp rx=0 tx=0).stop 2024-03-07T06:51:56.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.494+0000 7fbca7e00640 1 -- 172.21.15.133:0/3620792760 shutdown_connections 2024-03-07T06:51:56.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.494+0000 7fbca7e00640 1 --2- 172.21.15.133:0/3620792760 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbca8156f30 0x7fbca81573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.494+0000 7fbca7e00640 1 --2- 172.21.15.133:0/3620792760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbca814b7c0 0x7fbca8152dd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.494+0000 7fbca7e00640 1 --2- 172.21.15.133:0/3620792760 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbc8806b500 0x7fbc8806d9c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.494+0000 7fbca7e00640 1 --2- 172.21.15.133:0/3620792760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc8c004800 0x7fbca814c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.495+0000 7fbca7e00640 1 -- 172.21.15.133:0/3620792760 >> 172.21.15.133:0/3620792760 conn(0x7fbca80b2e90 msgr2=0x7fbca80aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:56.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.495+0000 7fbca7e00640 1 -- 172.21.15.133:0/3620792760 shutdown_connections 2024-03-07T06:51:56.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.495+0000 7fbca7e00640 1 -- 172.21.15.133:0/3620792760 wait complete. 2024-03-07T06:51:56.513 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:51:12.858333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:51:13.615348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:51:56.514 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:51:56.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.722+0000 7f7281a00640 1 Processor -- start 2024-03-07T06:51:56.722 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.722+0000 7f7281a00640 1 -- start start 2024-03-07T06:51:56.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.724+0000 7f7281a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f727c145670 0x7f727c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.724+0000 7f7281a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f727c15cf90 0x7f727c0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.724+0000 7f7281a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f727c0cda20 0x7f727c15fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.724+0000 7f7281a00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727c06ed60 con 0x7f727c0cda20 2024-03-07T06:51:56.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.724+0000 7f7281a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727c06e150 con 0x7f727c15cf90 2024-03-07T06:51:56.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.724+0000 7f7281a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727c06e430 con 0x7f727c145670 2024-03-07T06:51:56.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.724+0000 7f727b400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f727c0cda20 0x7f727c15fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:56.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.724+0000 7f727b400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f727c0cda20 0x7f727c15fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37252/0 (socket says 172.21.15.133:37252) 2024-03-07T06:51:56.724 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.724+0000 7f727b400640 1 -- 172.21.15.133:0/1348025641 learned_addr learned my addr 172.21.15.133:0/1348025641 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:56.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.725+0000 7f727aa00640 1 --2- 172.21.15.133:0/1348025641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f727c145670 0x7f727c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:56.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.725+0000 7f7272000640 1 --2- 172.21.15.133:0/1348025641 >> v2:172.21.15.161:3300/0 conn(0x7f727c15cf90 0x7f727c0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:56.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.725+0000 7f727b400640 1 -- 172.21.15.133:0/1348025641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f727c145670 msgr2=0x7f727c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:56.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.725+0000 7f727b400640 1 --2- 172.21.15.133:0/1348025641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f727c145670 0x7f727c0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.725 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.725+0000 7f727b400640 1 -- 172.21.15.133:0/1348025641 >> v2:172.21.15.161:3300/0 conn(0x7f727c15cf90 msgr2=0x7f727c0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:56.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.725+0000 7f727b400640 1 --2- 172.21.15.133:0/1348025641 >> v2:172.21.15.161:3300/0 conn(0x7f727c15cf90 0x7f727c0cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.725+0000 7f727b400640 1 -- 172.21.15.133:0/1348025641 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c06e710 con 0x7f727c0cda20 2024-03-07T06:51:56.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.725+0000 7f727aa00640 1 --2- 172.21.15.133:0/1348025641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f727c145670 0x7f727c0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:56.726 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.726+0000 7f727b400640 1 --2- 172.21.15.133:0/1348025641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f727c0cda20 0x7f727c15fb00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f727c0b2b70 tx=0x7f727402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d42c88481ec203e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:56.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.726+0000 7f727a000640 1 -- 172.21.15.133:0/1348025641 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727402c9e0 con 0x7f727c0cda20 2024-03-07T06:51:56.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.726+0000 7f727a000640 1 -- 172.21.15.133:0/1348025641 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727402cb80 con 0x7f727c0cda20 2024-03-07T06:51:56.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.726+0000 7f727a000640 1 -- 172.21.15.133:0/1348025641 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72740343f0 con 0x7f727c0cda20 2024-03-07T06:51:56.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.727+0000 7f7281a00640 1 -- 172.21.15.133:0/1348025641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f727c0cda20 msgr2=0x7f727c15fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:56.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.727+0000 7f7281a00640 1 --2- 172.21.15.133:0/1348025641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f727c0cda20 0x7f727c15fb00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f727c0b2b70 tx=0x7f727402be40 comp rx=0 tx=0).stop 2024-03-07T06:51:56.727 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.727+0000 7f7281a00640 1 -- 172.21.15.133:0/1348025641 shutdown_connections 2024-03-07T06:51:56.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.727+0000 7f7281a00640 1 --2- 172.21.15.133:0/1348025641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f727c0cda20 0x7f727c15fb00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.727+0000 7f7281a00640 1 --2- 172.21.15.133:0/1348025641 >> v2:172.21.15.161:3300/0 conn(0x7f727c15cf90 0x7f727c0cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.727+0000 7f7281a00640 1 --2- 172.21.15.133:0/1348025641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f727c145670 0x7f727c0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.727+0000 7f7281a00640 1 -- 172.21.15.133:0/1348025641 >> 172.21.15.133:0/1348025641 conn(0x7f727c0b1d30 msgr2=0x7f727c0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:56.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.727+0000 7f7281a00640 1 -- 172.21.15.133:0/1348025641 shutdown_connections 2024-03-07T06:51:56.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.727+0000 7f7281a00640 1 -- 172.21.15.133:0/1348025641 wait complete. 2024-03-07T06:51:56.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.729+0000 7f7281a00640 1 Processor -- start 2024-03-07T06:51:56.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.729+0000 7f7281a00640 1 -- start start 2024-03-07T06:51:56.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f7281a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f727c0cda20 0x7f727c1402d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f7281a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f727c145670 0x7f727c15b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f7281a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f727c15cf90 0x7f727c13e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f7281a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c145a50 con 0x7f727c0cda20 2024-03-07T06:51:56.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f7281a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c15d3f0 con 0x7f727c145670 2024-03-07T06:51:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f7281a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727c145bd0 con 0x7f727c15cf90 2024-03-07T06:51:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f727aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f727c0cda20 0x7f727c1402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f727aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f727c0cda20 0x7f727c1402d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37268/0 (socket says 172.21.15.133:37268) 2024-03-07T06:51:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f727aa00640 1 -- 172.21.15.133:0/513072441 learned_addr learned my addr 172.21.15.133:0/513072441 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f727b400640 1 --2- 172.21.15.133:0/513072441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f727c15cf90 0x7f727c13e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f7272000640 1 --2- 172.21.15.133:0/513072441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f727c145670 0x7f727c15b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f727aa00640 1 -- 172.21.15.133:0/513072441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f727c15cf90 msgr2=0x7f727c13e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f727aa00640 1 --2- 172.21.15.133:0/513072441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f727c15cf90 0x7f727c13e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f727aa00640 1 -- 172.21.15.133:0/513072441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f727c145670 msgr2=0x7f727c15b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f727aa00640 1 --2- 172.21.15.133:0/513072441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f727c145670 0x7f727c15b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.731+0000 7f727aa00640 1 -- 172.21.15.133:0/513072441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c1400d0 con 0x7f727c0cda20 2024-03-07T06:51:56.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.732+0000 7f727aa00640 1 --2- 172.21.15.133:0/513072441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f727c0cda20 0x7f727c1402d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f726c00c7f0 tx=0x7f726c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:56.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.732+0000 7f7273400640 1 -- 172.21.15.133:0/513072441 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c015070 con 0x7f727c0cda20 2024-03-07T06:51:56.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.732+0000 7f7273400640 1 -- 172.21.15.133:0/513072441 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726c007820 con 0x7f727c0cda20 2024-03-07T06:51:56.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.732+0000 7f7281a00640 1 -- 172.21.15.133:0/513072441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727c13da50 con 0x7f727c0cda20 2024-03-07T06:51:56.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.732+0000 7f7273400640 1 -- 172.21.15.133:0/513072441 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c007c10 con 0x7f727c0cda20 2024-03-07T06:51:56.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.732+0000 7f7281a00640 1 -- 172.21.15.133:0/513072441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727c13e010 con 0x7f727c0cda20 2024-03-07T06:51:56.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.733+0000 7f7281a00640 1 -- 172.21.15.133:0/513072441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727c062c90 con 0x7f727c0cda20 2024-03-07T06:51:56.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.734+0000 7f7273400640 1 -- 172.21.15.133:0/513072441 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f726c018070 con 0x7f727c0cda20 2024-03-07T06:51:56.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.735+0000 7f7273400640 1 --2- 172.21.15.133:0/513072441 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f725806b4d0 0x7f725806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:56.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.735+0000 7f7273400640 1 -- 172.21.15.133:0/513072441 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f726c08a330 con 0x7f727c0cda20 2024-03-07T06:51:56.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.736+0000 7f7272000640 1 --2- 172.21.15.133:0/513072441 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f725806b4d0 0x7f725806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:56.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.737+0000 7f7272000640 1 --2- 172.21.15.133:0/513072441 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f725806b4d0 0x7f725806d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f727c13a0b0 tx=0x7f7264008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:56.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.740+0000 7f7273400640 1 -- 172.21.15.133:0/513072441 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f727c1400d0 con 0x7f727c0cda20 2024-03-07T06:51:56.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.938+0000 7f7281a00640 1 -- 172.21.15.133:0/513072441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f727c15f1b0 con 0x7f727c0cda20 2024-03-07T06:51:56.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.938+0000 7f7273400640 1 -- 172.21.15.133:0/513072441 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f726c059140 con 0x7f727c0cda20 2024-03-07T06:51:56.938 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:51:56.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.941+0000 7f7281a00640 1 -- 172.21.15.133:0/513072441 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f725806b4d0 msgr2=0x7f725806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:56.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.941+0000 7f7281a00640 1 --2- 172.21.15.133:0/513072441 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f725806b4d0 0x7f725806d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f727c13a0b0 tx=0x7f7264008040 comp rx=0 tx=0).stop 2024-03-07T06:51:56.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.941+0000 7f7281a00640 1 -- 172.21.15.133:0/513072441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f727c0cda20 msgr2=0x7f727c1402d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:56.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.941+0000 7f7281a00640 1 --2- 172.21.15.133:0/513072441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f727c0cda20 0x7f727c1402d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f726c00c7f0 tx=0x7f726c00ccb0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.941+0000 7f7281a00640 1 -- 172.21.15.133:0/513072441 shutdown_connections 2024-03-07T06:51:56.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.941+0000 7f7281a00640 1 --2- 172.21.15.133:0/513072441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f727c15cf90 0x7f727c13e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.941+0000 7f7281a00640 1 --2- 172.21.15.133:0/513072441 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f725806b4d0 0x7f725806d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.941+0000 7f7281a00640 1 --2- 172.21.15.133:0/513072441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f727c145670 0x7f727c15b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.941+0000 7f7281a00640 1 --2- 172.21.15.133:0/513072441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f727c0cda20 0x7f727c1402d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.941+0000 7f7281a00640 1 -- 172.21.15.133:0/513072441 >> 172.21.15.133:0/513072441 conn(0x7f727c0b1d30 msgr2=0x7f727c15c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.941+0000 7f7281a00640 1 -- 172.21.15.133:0/513072441 shutdown_connections 2024-03-07T06:51:56.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:56.941+0000 7f7281a00640 1 -- 172.21.15.133:0/513072441 wait complete. 2024-03-07T06:51:56.960 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:51:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.172+0000 7febd1400640 1 Processor -- start 2024-03-07T06:51:57.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.172+0000 7febd1400640 1 -- start start 2024-03-07T06:51:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.174+0000 7febd1400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7febcc0729b0 0x7febcc166c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.174+0000 7febd1400640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febcc06e150 con 0x7febcc145670 2024-03-07T06:51:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.174+0000 7febd1400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febcc06e430 con 0x7febcc0cf610 2024-03-07T06:51:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.174+0000 7febd1400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febcc06ed60 con 0x7febcc0729b0 2024-03-07T06:51:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.174+0000 7febca800640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7febcc145670 0x7febcc15cf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57086/0 (socket says 172.21.15.133:57086) 2024-03-07T06:51:57.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.174+0000 7febca800640 1 -- 172.21.15.133:0/1414061676 learned_addr learned my addr 172.21.15.133:0/1414061676 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 -- 172.21.15.133:0/1414061676 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892545273 0 0) 0x7febcc06e150 con 0x7febcc145670 2024-03-07T06:51:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 -- 172.21.15.133:0/1414061676 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febb0003520 con 0x7febcc145670 2024-03-07T06:51:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febcb200640 1 --2- 172.21.15.133:0/1414061676 >> v2:172.21.15.161:3300/0 conn(0x7febcc0729b0 0x7febcc166c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 -- 172.21.15.133:0/1414061676 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 986694886 0 0) 0x7febb0003520 con 0x7febcc145670 2024-03-07T06:51:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 -- 172.21.15.133:0/1414061676 >> v2:172.21.15.161:3300/0 conn(0x7febcc0729b0 msgr2=0x7febcc166c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 --2- 172.21.15.133:0/1414061676 >> v2:172.21.15.161:3300/0 conn(0x7febcc0729b0 0x7febcc166c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 -- 172.21.15.133:0/1414061676 >> v1:172.21.15.161:6789/0 conn(0x7febcc0cf610 legacy=0x7febcc0cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:57.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 -- 172.21.15.133:0/1414061676 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc06e710 con 0x7febcc145670 2024-03-07T06:51:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 -- 172.21.15.133:0/1414061676 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7febbc001670 con 0x7febcc145670 2024-03-07T06:51:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 -- 172.21.15.133:0/1414061676 >> v1:172.21.15.133:6789/0 conn(0x7febcc145670 legacy=0x7febcc15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 --2- 172.21.15.133:0/1414061676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febb0003a00 0x7febb0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 --2- 172.21.15.133:0/1414061676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb0004800 0x7febb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 --2- 172.21.15.133:0/1414061676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb00052e0 0x7febb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 -- 172.21.15.133:0/1414061676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc06ed60 con 0x7febb0003a00 2024-03-07T06:51:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 -- 172.21.15.133:0/1414061676 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb000c200 con 0x7febb00052e0 2024-03-07T06:51:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febc9400640 1 -- 172.21.15.133:0/1414061676 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb000c400 con 0x7febb0004800 2024-03-07T06:51:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.175+0000 7febca800640 1 --2- 172.21.15.133:0/1414061676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febb0003a00 0x7febb0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.176+0000 7febca800640 1 -- 172.21.15.133:0/1414061676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb0004800 msgr2=0x7febb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:57.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.176+0000 7febca800640 1 --2- 172.21.15.133:0/1414061676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febb0004800 0x7febb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.176+0000 7febca800640 1 -- 172.21.15.133:0/1414061676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb00052e0 msgr2=0x7febb000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:57.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.176+0000 7febca800640 1 --2- 172.21.15.133:0/1414061676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb00052e0 0x7febb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.176+0000 7febca800640 1 -- 172.21.15.133:0/1414061676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb0003520 con 0x7febb0003a00 2024-03-07T06:51:57.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.176+0000 7febcb200640 1 -- 172.21.15.133:0/1414061676 reap_dead start 2024-03-07T06:51:57.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.177+0000 7febca800640 1 --2- 172.21.15.133:0/1414061676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febb0003a00 0x7febb0003e20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c0e0 tx=0x7febb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c271e2c27f8cae server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:57.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.177+0000 7febc9400640 1 -- 172.21.15.133:0/1414061676 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc007c20 con 0x7febb0003a00 2024-03-07T06:51:57.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.177+0000 7febc9400640 1 -- 172.21.15.133:0/1414061676 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc030070 con 0x7febb0003a00 2024-03-07T06:51:57.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.177+0000 7febc9400640 1 -- 172.21.15.133:0/1414061676 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc005d70 con 0x7febb0003a00 2024-03-07T06:51:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.178+0000 7febd1400640 1 -- 172.21.15.133:0/1414061676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febb0003a00 msgr2=0x7febb0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.178+0000 7febd1400640 1 --2- 172.21.15.133:0/1414061676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febb0003a00 0x7febb0003e20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c0e0 tx=0x7febb0005830 comp rx=0 tx=0).stop 2024-03-07T06:51:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.178+0000 7febd1400640 1 -- 172.21.15.133:0/1414061676 shutdown_connections 2024-03-07T06:51:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.178+0000 7febd1400640 1 --2- 172.21.15.133:0/1414061676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febb0003a00 0x7febb0003e20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.178+0000 7febd1400640 1 -- 172.21.15.133:0/1414061676 >> 172.21.15.133:0/1414061676 conn(0x7febcc0b1d30 msgr2=0x7febcc0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.178+0000 7febd1400640 1 -- 172.21.15.133:0/1414061676 shutdown_connections 2024-03-07T06:51:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.178+0000 7febd1400640 1 -- 172.21.15.133:0/1414061676 wait complete. 2024-03-07T06:51:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.180+0000 7febd1400640 1 Processor -- start 2024-03-07T06:51:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.180+0000 7febd1400640 1 -- start start 2024-03-07T06:51:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febd1400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb0003a00 0x7febcc0cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febd1400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febcc0b2f50 0x7febcc0ab420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febd1400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febcc0ab960 0x7febcc138250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febd1400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc145bd0 con 0x7febcc0ab960 2024-03-07T06:51:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febd1400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc145a50 con 0x7febb0003a00 2024-03-07T06:51:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febd1400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febcc15cf90 con 0x7febcc0b2f50 2024-03-07T06:51:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febcb200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febcc0ab960 0x7febcc138250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febcb200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febcc0ab960 0x7febcc138250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37276/0 (socket says 172.21.15.133:37276) 2024-03-07T06:51:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febcb200640 1 -- 172.21.15.133:0/3167021911 learned_addr learned my addr 172.21.15.133:0/3167021911 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febc9e00640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febcc0b2f50 0x7febcc0ab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febca800640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb0003a00 0x7febcc0cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febcb200640 1 -- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febcc0b2f50 msgr2=0x7febcc0ab420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febcb200640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febcc0b2f50 0x7febcc0ab420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febcb200640 1 -- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb0003a00 msgr2=0x7febcc0cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febcb200640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb0003a00 0x7febcc0cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febcb200640 1 -- 172.21.15.133:0/3167021911 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc0b3410 con 0x7febcc0ab960 2024-03-07T06:51:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febc9e00640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febcc0b2f50 0x7febcc0ab420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:57.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.181+0000 7febca800640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb0003a00 0x7febcc0cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.182+0000 7febcb200640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febcc0ab960 0x7febcc138250 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7febcc15c510 tx=0x7febc4007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.182+0000 7febc2a00640 1 -- 172.21.15.133:0/3167021911 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc400ec80 con 0x7febcc0ab960 2024-03-07T06:51:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.182+0000 7febc2a00640 1 -- 172.21.15.133:0/3167021911 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc4019070 con 0x7febcc0ab960 2024-03-07T06:51:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.182+0000 7febc2a00640 1 -- 172.21.15.133:0/3167021911 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4014660 con 0x7febcc0ab960 2024-03-07T06:51:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.182+0000 7febd1400640 1 -- 172.21.15.133:0/3167021911 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febcc0b36f0 con 0x7febcc0ab960 2024-03-07T06:51:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.182+0000 7febd1400640 1 -- 172.21.15.133:0/3167021911 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc0b3cb0 con 0x7febcc0ab960 2024-03-07T06:51:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.183+0000 7febd1400640 1 -- 172.21.15.133:0/3167021911 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febcc062c90 con 0x7febcc0ab960 2024-03-07T06:51:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.184+0000 7febc2a00640 1 -- 172.21.15.133:0/3167021911 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7febc4014800 con 0x7febcc0ab960 2024-03-07T06:51:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.185+0000 7febc2a00640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7febac06b4d0 0x7febac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.185+0000 7febc2a00640 1 -- 172.21.15.133:0/3167021911 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7febc408a710 con 0x7febcc0ab960 2024-03-07T06:51:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.186+0000 7febca800640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7febac06b4d0 0x7febac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:57.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.187+0000 7febca800640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7febac06b4d0 0x7febac06d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7febbc034000 tx=0x7febbc039040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:57.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.189+0000 7febc2a00640 1 -- 172.21.15.133:0/3167021911 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7febc4059470 con 0x7febcc0ab960 2024-03-07T06:51:57.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.385+0000 7febd1400640 1 -- 172.21.15.133:0/3167021911 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7febcc0aade0 con 0x7febcc0ab960 2024-03-07T06:51:57.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.386+0000 7febc2a00640 1 -- 172.21.15.133:0/3167021911 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7febc400ee60 con 0x7febcc0ab960 2024-03-07T06:51:57.386 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:51:57.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.388+0000 7febd1400640 1 -- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7febac06b4d0 msgr2=0x7febac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.388+0000 7febd1400640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7febac06b4d0 0x7febac06d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7febbc034000 tx=0x7febbc039040 comp rx=0 tx=0).stop 2024-03-07T06:51:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.388+0000 7febd1400640 1 -- 172.21.15.133:0/3167021911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febcc0ab960 msgr2=0x7febcc138250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.388+0000 7febd1400640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febcc0ab960 0x7febcc138250 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7febcc15c510 tx=0x7febc4007f40 comp rx=0 tx=0).stop 2024-03-07T06:51:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.389+0000 7febd1400640 1 -- 172.21.15.133:0/3167021911 shutdown_connections 2024-03-07T06:51:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.389+0000 7febd1400640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7febcc0ab960 0x7febcc138250 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.389+0000 7febd1400640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febcc0b2f50 0x7febcc0ab420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.389+0000 7febd1400640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7febac06b4d0 0x7febac06d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.389+0000 7febd1400640 1 --2- 172.21.15.133:0/3167021911 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febb0003a00 0x7febcc0cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.389+0000 7febd1400640 1 -- 172.21.15.133:0/3167021911 >> 172.21.15.133:0/3167021911 conn(0x7febcc0b1d30 msgr2=0x7febcc0719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:57.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.389+0000 7febd1400640 1 -- 172.21.15.133:0/3167021911 shutdown_connections 2024-03-07T06:51:57.390 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.389+0000 7febd1400640 1 -- 172.21.15.133:0/3167021911 wait complete. 2024-03-07T06:51:57.408 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:51:57.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.622+0000 7ff334800640 1 Processor -- start 2024-03-07T06:51:57.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.622+0000 7ff334800640 1 -- start start 2024-03-07T06:51:57.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.624+0000 7ff334800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c162950 0x7ff32c15edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.624+0000 7ff334800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32c15f410 con 0x7ff32c0b4130 2024-03-07T06:51:57.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.624+0000 7ff334800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32c161e70 con 0x7ff32c140ed0 2024-03-07T06:51:57.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.624+0000 7ff334800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c1624c0 con 0x7ff32c162950 2024-03-07T06:51:57.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.624+0000 7ff331400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff32c0b4130 0x7ff32c15e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57090/0 (socket says 172.21.15.133:57090) 2024-03-07T06:51:57.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.624+0000 7ff331400640 1 -- 172.21.15.133:0/1614452539 learned_addr learned my addr 172.21.15.133:0/1614452539 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:57.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff332800640 1 --2- 172.21.15.133:0/1614452539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c162950 0x7ff32c15edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:57.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff330a00640 1 -- 172.21.15.133:0/1614452539 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925870017 0 0) 0x7ff32c15f410 con 0x7ff32c0b4130 2024-03-07T06:51:57.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff330a00640 1 -- 172.21.15.133:0/1614452539 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff318003520 con 0x7ff32c0b4130 2024-03-07T06:51:57.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff330a00640 1 -- 172.21.15.133:0/1614452539 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2272381343 0 0) 0x7ff318003520 con 0x7ff32c0b4130 2024-03-07T06:51:57.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff330a00640 1 -- 172.21.15.133:0/1614452539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c162950 msgr2=0x7ff32c15edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:57.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff330a00640 1 --2- 172.21.15.133:0/1614452539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c162950 0x7ff32c15edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff330a00640 1 -- 172.21.15.133:0/1614452539 >> v1:172.21.15.161:6789/0 conn(0x7ff32c140ed0 legacy=0x7ff32c0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:57.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff330a00640 1 -- 172.21.15.133:0/1614452539 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c162150 con 0x7ff32c0b4130 2024-03-07T06:51:57.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff330a00640 1 -- 172.21.15.133:0/1614452539 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7ff31c001670 con 0x7ff32c0b4130 2024-03-07T06:51:57.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff332800640 1 --2- 172.21.15.133:0/1614452539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c162950 0x7ff32c15edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:51:57.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff330a00640 1 -- 172.21.15.133:0/1614452539 >> v1:172.21.15.133:6789/0 conn(0x7ff32c0b4130 legacy=0x7ff32c15e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:57.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff330a00640 1 --2- 172.21.15.133:0/1614452539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3180039e0 0x7ff318003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff330a00640 1 --2- 172.21.15.133:0/1614452539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff318004800 0x7ff318004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff330a00640 1 --2- 172.21.15.133:0/1614452539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c162950 0x7ff318005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.625+0000 7ff330a00640 1 -- 172.21.15.133:0/1614452539 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c1624c0 con 0x7ff318004800 2024-03-07T06:51:57.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.626+0000 7ff330a00640 1 -- 172.21.15.133:0/1614452539 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31800bb50 con 0x7ff3180039e0 2024-03-07T06:51:57.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.626+0000 7ff330a00640 1 -- 172.21.15.133:0/1614452539 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31800bd00 con 0x7ff32c162950 2024-03-07T06:51:57.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.626+0000 7ff331400640 1 --2- 172.21.15.133:0/1614452539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff318004800 0x7ff318004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:57.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.626+0000 7ff331e00640 1 --2- 172.21.15.133:0/1614452539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3180039e0 0x7ff318003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:57.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.626+0000 7ff331400640 1 -- 172.21.15.133:0/1614452539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c162950 msgr2=0x7ff318005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:57.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.626+0000 7ff331400640 1 --2- 172.21.15.133:0/1614452539 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c162950 0x7ff318005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.626+0000 7ff331400640 1 -- 172.21.15.133:0/1614452539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3180039e0 msgr2=0x7ff318003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:57.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.626+0000 7ff331400640 1 --2- 172.21.15.133:0/1614452539 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff3180039e0 0x7ff318003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.626+0000 7ff331400640 1 -- 172.21.15.133:0/1614452539 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff318003520 con 0x7ff318004800 2024-03-07T06:51:57.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.627+0000 7ff331400640 1 --2- 172.21.15.133:0/1614452539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff318004800 0x7ff318004c80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff31c007fd0 tx=0x7ff318005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b4af2c63dfcafa server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.627+0000 7ff330a00640 1 -- 172.21.15.133:0/1614452539 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c007c20 con 0x7ff318004800 2024-03-07T06:51:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.627+0000 7ff330a00640 1 -- 172.21.15.133:0/1614452539 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c030070 con 0x7ff318004800 2024-03-07T06:51:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.627+0000 7ff330a00640 1 -- 172.21.15.133:0/1614452539 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c005d30 con 0x7ff318004800 2024-03-07T06:51:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.628+0000 7ff334800640 1 -- 172.21.15.133:0/1614452539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff318004800 msgr2=0x7ff318004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.628+0000 7ff334800640 1 --2- 172.21.15.133:0/1614452539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff318004800 0x7ff318004c80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff31c007fd0 tx=0x7ff318005830 comp rx=0 tx=0).stop 2024-03-07T06:51:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.628+0000 7ff332800640 1 -- 172.21.15.133:0/1614452539 reap_dead start 2024-03-07T06:51:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.628+0000 7ff334800640 1 -- 172.21.15.133:0/1614452539 shutdown_connections 2024-03-07T06:51:57.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.628+0000 7ff334800640 1 -- 172.21.15.133:0/1614452539 >> 172.21.15.133:0/1614452539 conn(0x7ff32c070400 msgr2=0x7ff32c070810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:57.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.628+0000 7ff334800640 1 -- 172.21.15.133:0/1614452539 shutdown_connections 2024-03-07T06:51:57.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.628+0000 7ff334800640 1 -- 172.21.15.133:0/1614452539 wait complete. 2024-03-07T06:51:57.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.630+0000 7ff334800640 1 Processor -- start 2024-03-07T06:51:57.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.630+0000 7ff334800640 1 -- start start 2024-03-07T06:51:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.631+0000 7ff334800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff318004800 0x7ff32c1f4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.631+0000 7ff334800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c13e930 0x7ff32c1ea640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.631+0000 7ff334800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c158120 0x7ff32c1e8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.631+0000 7ff334800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c15ef40 con 0x7ff32c158120 2024-03-07T06:51:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.631+0000 7ff334800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c15edc0 con 0x7ff318004800 2024-03-07T06:51:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.631+0000 7ff334800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c15f0c0 con 0x7ff32c13e930 2024-03-07T06:51:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.631+0000 7ff332800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c158120 0x7ff32c1e8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.631+0000 7ff332800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c158120 0x7ff32c1e8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37304/0 (socket says 172.21.15.133:37304) 2024-03-07T06:51:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.631+0000 7ff332800640 1 -- 172.21.15.133:0/2885426086 learned_addr learned my addr 172.21.15.133:0/2885426086 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.632+0000 7ff331400640 1 --2- 172.21.15.133:0/2885426086 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c13e930 0x7ff32c1ea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:57.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.632+0000 7ff331e00640 1 --2- 172.21.15.133:0/2885426086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff318004800 0x7ff32c1f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.632+0000 7ff332800640 1 -- 172.21.15.133:0/2885426086 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c13e930 msgr2=0x7ff32c1ea640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.632+0000 7ff332800640 1 --2- 172.21.15.133:0/2885426086 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c13e930 0x7ff32c1ea640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.632+0000 7ff332800640 1 -- 172.21.15.133:0/2885426086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff318004800 msgr2=0x7ff32c1f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.632+0000 7ff332800640 1 --2- 172.21.15.133:0/2885426086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff318004800 0x7ff32c1f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.632+0000 7ff332800640 1 -- 172.21.15.133:0/2885426086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c142a20 con 0x7ff32c158120 2024-03-07T06:51:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.632+0000 7ff332800640 1 --2- 172.21.15.133:0/2885426086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c158120 0x7ff32c1e8590 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff32c162d60 tx=0x7ff32800e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.632+0000 7ff322000640 1 -- 172.21.15.133:0/2885426086 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32800f040 con 0x7ff32c158120 2024-03-07T06:51:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.632+0000 7ff322000640 1 -- 172.21.15.133:0/2885426086 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff328019070 con 0x7ff32c158120 2024-03-07T06:51:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.632+0000 7ff334800640 1 -- 172.21.15.133:0/2885426086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32c142d00 con 0x7ff32c158120 2024-03-07T06:51:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.632+0000 7ff322000640 1 -- 172.21.15.133:0/2885426086 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff328013810 con 0x7ff32c158120 2024-03-07T06:51:57.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.633+0000 7ff334800640 1 -- 172.21.15.133:0/2885426086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c1432c0 con 0x7ff32c158120 2024-03-07T06:51:57.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.634+0000 7ff334800640 1 -- 172.21.15.133:0/2885426086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32c067c80 con 0x7ff32c158120 2024-03-07T06:51:57.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.634+0000 7ff322000640 1 -- 172.21.15.133:0/2885426086 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff3280139b0 con 0x7ff32c158120 2024-03-07T06:51:57.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.635+0000 7ff322000640 1 --2- 172.21.15.133:0/2885426086 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff31006b500 0x7ff31006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:57.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.635+0000 7ff322000640 1 -- 172.21.15.133:0/2885426086 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff328089af0 con 0x7ff32c158120 2024-03-07T06:51:57.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.636+0000 7ff331e00640 1 --2- 172.21.15.133:0/2885426086 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff31006b500 0x7ff31006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:57.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.637+0000 7ff331e00640 1 --2- 172.21.15.133:0/2885426086 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff31006b500 0x7ff31006d9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff324007fd0 tx=0x7ff324009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:57.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.639+0000 7ff322000640 1 -- 172.21.15.133:0/2885426086 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff328058850 con 0x7ff32c158120 2024-03-07T06:51:57.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.837+0000 7ff334800640 1 -- 172.21.15.133:0/2885426086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff32c062c90 con 0x7ff32c158120 2024-03-07T06:51:57.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.838+0000 7ff322000640 1 -- 172.21.15.133:0/2885426086 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff328058150 con 0x7ff32c158120 2024-03-07T06:51:57.837 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:51:57.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.840+0000 7ff334800640 1 -- 172.21.15.133:0/2885426086 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff31006b500 msgr2=0x7ff31006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:57.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.840+0000 7ff334800640 1 --2- 172.21.15.133:0/2885426086 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff31006b500 0x7ff31006d9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff324007fd0 tx=0x7ff324009040 comp rx=0 tx=0).stop 2024-03-07T06:51:57.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.840+0000 7ff334800640 1 -- 172.21.15.133:0/2885426086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c158120 msgr2=0x7ff32c1e8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:57.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.840+0000 7ff334800640 1 --2- 172.21.15.133:0/2885426086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c158120 0x7ff32c1e8590 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff32c162d60 tx=0x7ff32800e8b0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.841+0000 7ff334800640 1 -- 172.21.15.133:0/2885426086 shutdown_connections 2024-03-07T06:51:57.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.841+0000 7ff334800640 1 --2- 172.21.15.133:0/2885426086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c158120 0x7ff32c1e8590 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.841+0000 7ff334800640 1 --2- 172.21.15.133:0/2885426086 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c13e930 0x7ff32c1ea640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.841+0000 7ff334800640 1 --2- 172.21.15.133:0/2885426086 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff31006b500 0x7ff31006d9c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.841+0000 7ff334800640 1 --2- 172.21.15.133:0/2885426086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff318004800 0x7ff32c1f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:57.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.841+0000 7ff334800640 1 -- 172.21.15.133:0/2885426086 >> 172.21.15.133:0/2885426086 conn(0x7ff32c070400 msgr2=0x7ff32c160c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:57.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.841+0000 7ff334800640 1 -- 172.21.15.133:0/2885426086 shutdown_connections 2024-03-07T06:51:57.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:57.841+0000 7ff334800640 1 -- 172.21.15.133:0/2885426086 wait complete. 2024-03-07T06:51:57.870 INFO:tasks.cephfs_test_runner:Starting test: test_dir_deletion (tasks.cephfs.test_strays.TestStrays) That when deleting a bunch of dentries and the containing 2024-03-07T06:51:57.871 INFO:tasks.cephfs_test_runner:test_dir_deletion (tasks.cephfs.test_strays.TestStrays) 2024-03-07T06:51:57.871 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:51:58.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.087+0000 7f9118400640 1 Processor -- start 2024-03-07T06:51:58.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.087+0000 7f9118400640 1 -- start start 2024-03-07T06:51:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.089+0000 7f9118400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f91100b43f0 0x7f9110070e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.089+0000 7f9118400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91100ae470 0x7f91100ae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.089+0000 7f9118400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9110147ea0 con 0x7f91100b6120 2024-03-07T06:51:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.089+0000 7f9118400640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9110071390 con 0x7f91100b43f0 2024-03-07T06:51:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.089+0000 7f9118400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91100b4960 con 0x7f91100ae470 2024-03-07T06:51:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.089+0000 7f9115000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f91100b43f0 0x7f9110070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.089+0000 7f9115000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f91100b43f0 0x7f9110070e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37314/0 (socket says 172.21.15.133:37314) 2024-03-07T06:51:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.089+0000 7f9115000640 1 -- 172.21.15.133:0/90843167 learned_addr learned my addr 172.21.15.133:0/90843167 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.089+0000 7f9116400640 1 --2- 172.21.15.133:0/90843167 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91100ae470 0x7f91100ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.089+0000 7f9115000640 1 -- 172.21.15.133:0/90843167 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91100ae470 msgr2=0x7f91100ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:58.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.089+0000 7f9115000640 1 --2- 172.21.15.133:0/90843167 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91100ae470 0x7f91100ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.089+0000 7f9115000640 1 -- 172.21.15.133:0/90843167 >> v1:172.21.15.161:6789/0 conn(0x7f91100b6120 legacy=0x7f91100b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:58.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.089+0000 7f9115000640 1 -- 172.21.15.133:0/90843167 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100b4c40 con 0x7f91100b43f0 2024-03-07T06:51:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.091+0000 7f9115000640 1 --2- 172.21.15.133:0/90843167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91100b43f0 0x7f9110070e50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009870 tx=0x7f90fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fa91b99d045ef server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.091+0000 7f910fe00640 1 -- 172.21.15.133:0/90843167 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc035020 con 0x7f91100b43f0 2024-03-07T06:51:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.091+0000 7f910fe00640 1 -- 172.21.15.133:0/90843167 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc031640 con 0x7f91100b43f0 2024-03-07T06:51:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.091+0000 7f910fe00640 1 -- 172.21.15.133:0/90843167 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc02d040 con 0x7f91100b43f0 2024-03-07T06:51:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.091+0000 7f9118400640 1 -- 172.21.15.133:0/90843167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91100b43f0 msgr2=0x7f9110070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.091+0000 7f9118400640 1 --2- 172.21.15.133:0/90843167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91100b43f0 0x7f9110070e50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009870 tx=0x7f90fc02b8f0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.092+0000 7f9118400640 1 -- 172.21.15.133:0/90843167 shutdown_connections 2024-03-07T06:51:58.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.092+0000 7f9118400640 1 --2- 172.21.15.133:0/90843167 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91100ae470 0x7f91100ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.092+0000 7f9118400640 1 --2- 172.21.15.133:0/90843167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91100b43f0 0x7f9110070e50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.092+0000 7f9118400640 1 -- 172.21.15.133:0/90843167 >> 172.21.15.133:0/90843167 conn(0x7f9110071550 msgr2=0x7f9110071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:58.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.092+0000 7f9118400640 1 -- 172.21.15.133:0/90843167 shutdown_connections 2024-03-07T06:51:58.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.092+0000 7f9118400640 1 -- 172.21.15.133:0/90843167 wait complete. 2024-03-07T06:51:58.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.093+0000 7f9118400640 1 Processor -- start 2024-03-07T06:51:58.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.094+0000 7f9118400640 1 -- start start 2024-03-07T06:51:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.094+0000 7f9118400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91100ae470 0x7f91101e8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.094+0000 7f9118400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91100b43f0 0x7f91100ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.095+0000 7f9118400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91100b6120 0x7f9110137380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.095+0000 7f9118400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100aee40 con 0x7f91100ae470 2024-03-07T06:51:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.095+0000 7f9118400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100b5200 con 0x7f91100b43f0 2024-03-07T06:51:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.095+0000 7f9118400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91100aefc0 con 0x7f91100b6120 2024-03-07T06:51:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.095+0000 7f9115a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91100ae470 0x7f91101e8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.095+0000 7f9115a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91100ae470 0x7f91101e8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37328/0 (socket says 172.21.15.133:37328) 2024-03-07T06:51:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.095+0000 7f9115a00640 1 -- 172.21.15.133:0/2011376738 learned_addr learned my addr 172.21.15.133:0/2011376738 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.095+0000 7f9115a00640 1 -- 172.21.15.133:0/2011376738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91100b6120 msgr2=0x7f9110137380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.095+0000 7f9115a00640 1 --2- 172.21.15.133:0/2011376738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91100b6120 0x7f9110137380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.095+0000 7f9115a00640 1 -- 172.21.15.133:0/2011376738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91100b43f0 msgr2=0x7f91100ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.095+0000 7f9115a00640 1 --2- 172.21.15.133:0/2011376738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91100b43f0 0x7f91100ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.095+0000 7f9115a00640 1 -- 172.21.15.133:0/2011376738 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91101e06f0 con 0x7f91100ae470 2024-03-07T06:51:58.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.095+0000 7f9115a00640 1 --2- 172.21.15.133:0/2011376738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91100ae470 0x7f91101e8530 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f910400b3f0 tx=0x7f910400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:58.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.096+0000 7f910d600640 1 -- 172.21.15.133:0/2011376738 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104015020 con 0x7f91100ae470 2024-03-07T06:51:58.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.096+0000 7f910d600640 1 -- 172.21.15.133:0/2011376738 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9104011830 con 0x7f91100ae470 2024-03-07T06:51:58.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.096+0000 7f910d600640 1 -- 172.21.15.133:0/2011376738 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104011c20 con 0x7f91100ae470 2024-03-07T06:51:58.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.096+0000 7f9118400640 1 -- 172.21.15.133:0/2011376738 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91101e09d0 con 0x7f91100ae470 2024-03-07T06:51:58.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.096+0000 7f9118400640 1 -- 172.21.15.133:0/2011376738 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91101e0f90 con 0x7f91100ae470 2024-03-07T06:51:58.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.097+0000 7f9118400640 1 -- 172.21.15.133:0/2011376738 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91101e12d0 con 0x7f91100ae470 2024-03-07T06:51:58.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.098+0000 7f910d600640 1 -- 172.21.15.133:0/2011376738 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9104018020 con 0x7f91100ae470 2024-03-07T06:51:58.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.099+0000 7f910d600640 1 --2- 172.21.15.133:0/2011376738 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f90f006b500 0x7f90f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.099+0000 7f910d600640 1 -- 172.21.15.133:0/2011376738 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f910408a360 con 0x7f91100ae470 2024-03-07T06:51:58.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.100+0000 7f9115000640 1 --2- 172.21.15.133:0/2011376738 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f90f006b500 0x7f90f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.101+0000 7f9115000640 1 --2- 172.21.15.133:0/2011376738 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f90f006b500 0x7f90f006d9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f90fc036000 tx=0x7f90fc030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:58.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.103+0000 7f910d600640 1 -- 172.21.15.133:0/2011376738 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f91101e12d0 con 0x7f91100ae470 2024-03-07T06:51:58.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.300+0000 7f9118400640 1 -- 172.21.15.133:0/2011376738 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9110147000 con 0x7f91100ae470 2024-03-07T06:51:58.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.304+0000 7f910d600640 1 -- 172.21.15.133:0/2011376738 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16901 (secure 0 0 0) 0x7f9110147000 con 0x7f91100ae470 2024-03-07T06:51:58.304 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:51:58.304 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:56.413096+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:51:58.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.307+0000 7f9118400640 1 -- 172.21.15.133:0/2011376738 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f90f006b500 msgr2=0x7f90f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.307+0000 7f9118400640 1 --2- 172.21.15.133:0/2011376738 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f90f006b500 0x7f90f006d9c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f90fc036000 tx=0x7f90fc030000 comp rx=0 tx=0).stop 2024-03-07T06:51:58.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.307+0000 7f9118400640 1 -- 172.21.15.133:0/2011376738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91100ae470 msgr2=0x7f91101e8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.307+0000 7f9118400640 1 --2- 172.21.15.133:0/2011376738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91100ae470 0x7f91101e8530 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f910400b3f0 tx=0x7f910400b8b0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.307+0000 7f9118400640 1 -- 172.21.15.133:0/2011376738 shutdown_connections 2024-03-07T06:51:58.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.307+0000 7f9118400640 1 --2- 172.21.15.133:0/2011376738 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91100b6120 0x7f9110137380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.307+0000 7f9118400640 1 --2- 172.21.15.133:0/2011376738 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f90f006b500 0x7f90f006d9c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.307+0000 7f9118400640 1 --2- 172.21.15.133:0/2011376738 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91100b43f0 0x7f91100ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.307+0000 7f9118400640 1 --2- 172.21.15.133:0/2011376738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91100ae470 0x7f91101e8530 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.307+0000 7f9118400640 1 -- 172.21.15.133:0/2011376738 >> 172.21.15.133:0/2011376738 conn(0x7f9110071550 msgr2=0x7f91101e8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:58.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.307+0000 7f9118400640 1 -- 172.21.15.133:0/2011376738 shutdown_connections 2024-03-07T06:51:58.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.307+0000 7f9118400640 1 -- 172.21.15.133:0/2011376738 wait complete. 2024-03-07T06:51:58.326 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:51:12.858333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:51:13.615348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:51:58.326 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:51:58.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.540+0000 7facea000640 1 Processor -- start 2024-03-07T06:51:58.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.540+0000 7facea000640 1 -- start start 2024-03-07T06:51:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.542+0000 7facea000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7face40ccff0 0x7face40cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.542+0000 7facea000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face4160720 con 0x7face4159880 2024-03-07T06:51:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.542+0000 7facea000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face40cde00 con 0x7face40b4f10 2024-03-07T06:51:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.542+0000 7facea000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face40cd560 con 0x7face40ccff0 2024-03-07T06:51:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.542+0000 7face8c00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7face4159880 0x7face4160010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57096/0 (socket says 172.21.15.133:57096) 2024-03-07T06:51:58.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.542+0000 7face8c00640 1 -- 172.21.15.133:0/195745324 learned_addr learned my addr 172.21.15.133:0/195745324 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.543+0000 7face3e00640 1 --2- 172.21.15.133:0/195745324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7face40ccff0 0x7face40cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.543+0000 7face3400640 1 -- 172.21.15.133:0/195745324 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688393465 0 0) 0x7face4160720 con 0x7face4159880 2024-03-07T06:51:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.543+0000 7face3400640 1 -- 172.21.15.133:0/195745324 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faccc003520 con 0x7face4159880 2024-03-07T06:51:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.543+0000 7face3400640 1 -- 172.21.15.133:0/195745324 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1478142675 0 0) 0x7faccc003520 con 0x7face4159880 2024-03-07T06:51:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.543+0000 7face3400640 1 -- 172.21.15.133:0/195745324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7face40ccff0 msgr2=0x7face40cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.543+0000 7face3400640 1 --2- 172.21.15.133:0/195745324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7face40ccff0 0x7face40cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.543+0000 7face3400640 1 -- 172.21.15.133:0/195745324 >> v1:172.21.15.161:6789/0 conn(0x7face40b4f10 legacy=0x7face415c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.543+0000 7face3400640 1 -- 172.21.15.133:0/195745324 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face40cd840 con 0x7face4159880 2024-03-07T06:51:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.543+0000 7face3400640 1 -- 172.21.15.133:0/195745324 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7facd8001670 con 0x7face4159880 2024-03-07T06:51:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.543+0000 7face3400640 1 -- 172.21.15.133:0/195745324 >> v1:172.21.15.133:6789/0 conn(0x7face4159880 legacy=0x7face4160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.543+0000 7face3400640 1 --2- 172.21.15.133:0/195745324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faccc0039e0 0x7faccc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.544+0000 7face3400640 1 --2- 172.21.15.133:0/195745324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faccc004800 0x7faccc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.544+0000 7face3400640 1 --2- 172.21.15.133:0/195745324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7face40ccff0 0x7faccc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.544+0000 7face3400640 1 -- 172.21.15.133:0/195745324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face40cd560 con 0x7faccc004800 2024-03-07T06:51:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.544+0000 7face3400640 1 -- 172.21.15.133:0/195745324 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc00baa0 con 0x7faccc0039e0 2024-03-07T06:51:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.544+0000 7face3400640 1 -- 172.21.15.133:0/195745324 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc00bca0 con 0x7face40ccff0 2024-03-07T06:51:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.544+0000 7face3e00640 1 --2- 172.21.15.133:0/195745324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faccc004800 0x7faccc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.544+0000 7face8c00640 1 --2- 172.21.15.133:0/195745324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faccc0039e0 0x7faccc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.544+0000 7face3e00640 1 -- 172.21.15.133:0/195745324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7face40ccff0 msgr2=0x7faccc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.544+0000 7face3e00640 1 --2- 172.21.15.133:0/195745324 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7face40ccff0 0x7faccc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.544+0000 7face3e00640 1 -- 172.21.15.133:0/195745324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faccc0039e0 msgr2=0x7faccc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.544+0000 7face3e00640 1 --2- 172.21.15.133:0/195745324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faccc0039e0 0x7faccc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.544+0000 7face3e00640 1 -- 172.21.15.133:0/195745324 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc003520 con 0x7faccc004800 2024-03-07T06:51:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.544+0000 7face8c00640 1 --2- 172.21.15.133:0/195745324 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faccc0039e0 0x7faccc003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:58.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.545+0000 7face3e00640 1 --2- 172.21.15.133:0/195745324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faccc004800 0x7faccc004c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7facd4009a90 tx=0x7faccc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f4e4fdf7f8e0f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:58.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.546+0000 7face3400640 1 -- 172.21.15.133:0/195745324 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4039070 con 0x7faccc004800 2024-03-07T06:51:58.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.546+0000 7face3400640 1 -- 172.21.15.133:0/195745324 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd402c6b0 con 0x7faccc004800 2024-03-07T06:51:58.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.546+0000 7face3400640 1 -- 172.21.15.133:0/195745324 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4009690 con 0x7faccc004800 2024-03-07T06:51:58.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.546+0000 7facea000640 1 -- 172.21.15.133:0/195745324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faccc004800 msgr2=0x7faccc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.546+0000 7facea000640 1 --2- 172.21.15.133:0/195745324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faccc004800 0x7faccc004c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7facd4009a90 tx=0x7faccc005830 comp rx=0 tx=0).stop 2024-03-07T06:51:58.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.546+0000 7face9600640 1 -- 172.21.15.133:0/195745324 reap_dead start 2024-03-07T06:51:58.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.546+0000 7facea000640 1 -- 172.21.15.133:0/195745324 shutdown_connections 2024-03-07T06:51:58.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.546+0000 7facea000640 1 -- 172.21.15.133:0/195745324 >> 172.21.15.133:0/195745324 conn(0x7face4071860 msgr2=0x7face4071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:58.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.546+0000 7facea000640 1 -- 172.21.15.133:0/195745324 shutdown_connections 2024-03-07T06:51:58.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.546+0000 7facea000640 1 -- 172.21.15.133:0/195745324 wait complete. 2024-03-07T06:51:58.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.548+0000 7facea000640 1 Processor -- start 2024-03-07T06:51:58.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.548+0000 7facea000640 1 -- start start 2024-03-07T06:51:58.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.549+0000 7facea000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faccc004800 0x7face415c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.549+0000 7facea000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7face414d8c0 0x7face40b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.549+0000 7facea000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7face40b0850 0x7face415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.549+0000 7facea000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face4159c60 con 0x7faccc004800 2024-03-07T06:51:58.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.549+0000 7facea000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face415faf0 con 0x7face414d8c0 2024-03-07T06:51:58.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.549+0000 7facea000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face40cc6f0 con 0x7face40b0850 2024-03-07T06:51:58.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.549+0000 7face8c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faccc004800 0x7face415c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.549+0000 7face8c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faccc004800 0x7face415c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37344/0 (socket says 172.21.15.133:37344) 2024-03-07T06:51:58.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.549+0000 7face8c00640 1 -- 172.21.15.133:0/2357895235 learned_addr learned my addr 172.21.15.133:0/2357895235 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:58.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.550+0000 7face3e00640 1 --2- 172.21.15.133:0/2357895235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7face414d8c0 0x7face40b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.550+0000 7face9600640 1 --2- 172.21.15.133:0/2357895235 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7face40b0850 0x7face415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.550+0000 7face8c00640 1 -- 172.21.15.133:0/2357895235 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7face40b0850 msgr2=0x7face415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.550+0000 7face8c00640 1 --2- 172.21.15.133:0/2357895235 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7face40b0850 0x7face415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.550+0000 7face8c00640 1 -- 172.21.15.133:0/2357895235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7face414d8c0 msgr2=0x7face40b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.550+0000 7face8c00640 1 --2- 172.21.15.133:0/2357895235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7face414d8c0 0x7face40b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.550+0000 7face8c00640 1 -- 172.21.15.133:0/2357895235 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face40adcb0 con 0x7faccc004800 2024-03-07T06:51:58.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.550+0000 7face8c00640 1 --2- 172.21.15.133:0/2357895235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faccc004800 0x7face415c2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7facd800d8f0 tx=0x7facd800ddb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:58.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.550+0000 7face0c00640 1 -- 172.21.15.133:0/2357895235 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd8019070 con 0x7faccc004800 2024-03-07T06:51:58.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.551+0000 7face0c00640 1 -- 172.21.15.133:0/2357895235 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd80074e0 con 0x7faccc004800 2024-03-07T06:51:58.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.551+0000 7face0c00640 1 -- 172.21.15.133:0/2357895235 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd80049e0 con 0x7faccc004800 2024-03-07T06:51:58.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.551+0000 7facea000640 1 -- 172.21.15.133:0/2357895235 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face40adf90 con 0x7faccc004800 2024-03-07T06:51:58.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.551+0000 7facea000640 1 -- 172.21.15.133:0/2357895235 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face40ae550 con 0x7faccc004800 2024-03-07T06:51:58.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.552+0000 7facea000640 1 -- 172.21.15.133:0/2357895235 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face40b0c60 con 0x7faccc004800 2024-03-07T06:51:58.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.553+0000 7face0c00640 1 -- 172.21.15.133:0/2357895235 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7facd801c070 con 0x7faccc004800 2024-03-07T06:51:58.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.554+0000 7face0c00640 1 --2- 172.21.15.133:0/2357895235 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7facc806b500 0x7facc806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.554+0000 7face0c00640 1 -- 172.21.15.133:0/2357895235 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7facd8089e40 con 0x7faccc004800 2024-03-07T06:51:58.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.555+0000 7face3e00640 1 --2- 172.21.15.133:0/2357895235 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7facc806b500 0x7facc806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.556+0000 7face3e00640 1 --2- 172.21.15.133:0/2357895235 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7facc806b500 0x7facc806d9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7facd4036000 tx=0x7facd4030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:58.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.558+0000 7face0c00640 1 -- 172.21.15.133:0/2357895235 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7face40b0c60 con 0x7faccc004800 2024-03-07T06:51:58.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.756+0000 7facea000640 1 -- 172.21.15.133:0/2357895235 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7face4062c90 con 0x7faccc004800 2024-03-07T06:51:58.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.757+0000 7face0c00640 1 -- 172.21.15.133:0/2357895235 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7facd8058c30 con 0x7faccc004800 2024-03-07T06:51:58.756 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:51:58.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.759+0000 7facea000640 1 -- 172.21.15.133:0/2357895235 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7facc806b500 msgr2=0x7facc806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.759+0000 7facea000640 1 --2- 172.21.15.133:0/2357895235 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7facc806b500 0x7facc806d9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7facd4036000 tx=0x7facd4030000 comp rx=0 tx=0).stop 2024-03-07T06:51:58.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.759+0000 7facea000640 1 -- 172.21.15.133:0/2357895235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faccc004800 msgr2=0x7face415c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.759+0000 7facea000640 1 --2- 172.21.15.133:0/2357895235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faccc004800 0x7face415c2f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7facd800d8f0 tx=0x7facd800ddb0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.759+0000 7facea000640 1 -- 172.21.15.133:0/2357895235 shutdown_connections 2024-03-07T06:51:58.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.759+0000 7facea000640 1 --2- 172.21.15.133:0/2357895235 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7face40b0850 0x7face415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.759+0000 7facea000640 1 --2- 172.21.15.133:0/2357895235 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7facc806b500 0x7facc806d9c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.759+0000 7facea000640 1 --2- 172.21.15.133:0/2357895235 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7face414d8c0 0x7face40b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.759+0000 7facea000640 1 --2- 172.21.15.133:0/2357895235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faccc004800 0x7face415c2f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.759+0000 7facea000640 1 -- 172.21.15.133:0/2357895235 >> 172.21.15.133:0/2357895235 conn(0x7face4071860 msgr2=0x7face40b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:58.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.759+0000 7facea000640 1 -- 172.21.15.133:0/2357895235 shutdown_connections 2024-03-07T06:51:58.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.759+0000 7facea000640 1 -- 172.21.15.133:0/2357895235 wait complete. 2024-03-07T06:51:58.777 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:51:58.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.979+0000 7fe770600640 1 Processor -- start 2024-03-07T06:51:58.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.979+0000 7fe770600640 1 -- start start 2024-03-07T06:51:58.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.980+0000 7fe770600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe7680b3ed0 0x7fe76815b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.980+0000 7fe770600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe76806d180 0x7fe76815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.980+0000 7fe770600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe76806d740 con 0x7fe76815b370 2024-03-07T06:51:58.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.980+0000 7fe770600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe76815c1d0 con 0x7fe7680b3ed0 2024-03-07T06:51:58.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.980+0000 7fe770600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76806da20 con 0x7fe76806d180 2024-03-07T06:51:58.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.981+0000 7fe76e600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe76806d180 0x7fe76815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.981+0000 7fe76e600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe76806d180 0x7fe76815bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:60136/0 (socket says 172.21.15.133:60136) 2024-03-07T06:51:58.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.981+0000 7fe76dc00640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fe76815b370 0x7fe7680cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.133:59770/0 (socket says 172.21.15.133:59770) 2024-03-07T06:51:58.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.981+0000 7fe76e600640 1 -- 172.21.15.133:0/3723546922 learned_addr learned my addr 172.21.15.133:0/3723546922 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:58.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.981+0000 7fe76d200640 1 --2- 172.21.15.133:0/3723546922 >> v2:172.21.15.161:3300/0 conn(0x7fe7680b3ed0 0x7fe76815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.982+0000 7fe767e00640 1 -- 172.21.15.133:0/3723546922 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472918219 0 0) 0x7fe76806d740 con 0x7fe76815b370 2024-03-07T06:51:58.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.982+0000 7fe767e00640 1 -- 172.21.15.133:0/3723546922 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe750003520 con 0x7fe76815b370 2024-03-07T06:51:58.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.982+0000 7fe76e600640 1 -- 172.21.15.133:0/3723546922 >> v2:172.21.15.161:3300/0 conn(0x7fe7680b3ed0 msgr2=0x7fe76815b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:58.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.982+0000 7fe76e600640 1 --2- 172.21.15.133:0/3723546922 >> v2:172.21.15.161:3300/0 conn(0x7fe7680b3ed0 0x7fe76815b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.982+0000 7fe76e600640 1 -- 172.21.15.133:0/3723546922 >> v1:172.21.15.161:6789/0 conn(0x7fe76815b370 legacy=0x7fe7680cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.982+0000 7fe76e600640 1 -- 172.21.15.133:0/3723546922 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76806dd00 con 0x7fe76806d180 2024-03-07T06:51:58.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.983+0000 7fe76e600640 1 --2- 172.21.15.133:0/3723546922 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe76806d180 0x7fe76815bc90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe760009e50 tx=0x7fe76002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73590198638d3e25 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:58.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.984+0000 7fe767e00640 1 -- 172.21.15.133:0/3723546922 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76002cb00 con 0x7fe76806d180 2024-03-07T06:51:58.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.984+0000 7fe767e00640 1 -- 172.21.15.133:0/3723546922 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76002cca0 con 0x7fe76806d180 2024-03-07T06:51:58.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.984+0000 7fe767e00640 1 -- 172.21.15.133:0/3723546922 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7600345f0 con 0x7fe76806d180 2024-03-07T06:51:58.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.984+0000 7fe770600640 1 -- 172.21.15.133:0/3723546922 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe76806d180 msgr2=0x7fe76815bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.984+0000 7fe770600640 1 --2- 172.21.15.133:0/3723546922 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe76806d180 0x7fe76815bc90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe760009e50 tx=0x7fe76002bf60 comp rx=0 tx=0).stop 2024-03-07T06:51:58.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.985+0000 7fe770600640 1 -- 172.21.15.133:0/3723546922 shutdown_connections 2024-03-07T06:51:58.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.985+0000 7fe770600640 1 --2- 172.21.15.133:0/3723546922 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe76806d180 0x7fe76815bc90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.985+0000 7fe770600640 1 --2- 172.21.15.133:0/3723546922 >> v2:172.21.15.161:3300/0 conn(0x7fe7680b3ed0 0x7fe76815b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.985+0000 7fe770600640 1 -- 172.21.15.133:0/3723546922 >> 172.21.15.133:0/3723546922 conn(0x7fe7680b1d30 msgr2=0x7fe7680b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:58.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.985+0000 7fe770600640 1 -- 172.21.15.133:0/3723546922 shutdown_connections 2024-03-07T06:51:58.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.985+0000 7fe770600640 1 -- 172.21.15.133:0/3723546922 wait complete. 2024-03-07T06:51:58.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.986+0000 7fe770600640 1 Processor -- start 2024-03-07T06:51:58.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.986+0000 7fe770600640 1 -- start start 2024-03-07T06:51:58.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.987+0000 7fe770600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe76806d180 0x7fe768167100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.987+0000 7fe770600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7680b3ed0 0x7fe768140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.987+0000 7fe770600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe76815b370 0x7fe76813ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.987+0000 7fe770600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680cf660 con 0x7fe76806d180 2024-03-07T06:51:58.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.987+0000 7fe770600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76806e2c0 con 0x7fe7680b3ed0 2024-03-07T06:51:58.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.987+0000 7fe770600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7680cf7e0 con 0x7fe76815b370 2024-03-07T06:51:58.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.988+0000 7fe76dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe76806d180 0x7fe768167100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.988+0000 7fe76dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe76806d180 0x7fe768167100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37356/0 (socket says 172.21.15.133:37356) 2024-03-07T06:51:58.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.988+0000 7fe76dc00640 1 -- 172.21.15.133:0/1602856133 learned_addr learned my addr 172.21.15.133:0/1602856133 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:58.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.988+0000 7fe76d200640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7680b3ed0 0x7fe768140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.988+0000 7fe76e600640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe76815b370 0x7fe76813ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.988+0000 7fe76dc00640 1 -- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe76815b370 msgr2=0x7fe76813ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.988+0000 7fe76dc00640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe76815b370 0x7fe76813ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.988+0000 7fe76dc00640 1 -- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7680b3ed0 msgr2=0x7fe768140320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:58.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.988+0000 7fe76dc00640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7680b3ed0 0x7fe768140320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.988+0000 7fe76dc00640 1 -- 172.21.15.133:0/1602856133 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7681664f0 con 0x7fe76806d180 2024-03-07T06:51:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.988+0000 7fe76d200640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7680b3ed0 0x7fe768140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:51:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.988+0000 7fe76dc00640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe76806d180 0x7fe768167100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c390 tx=0x7fe75c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.989+0000 7fe765600640 1 -- 172.21.15.133:0/1602856133 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0074e0 con 0x7fe76806d180 2024-03-07T06:51:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.989+0000 7fe765600640 1 -- 172.21.15.133:0/1602856133 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c0047e0 con 0x7fe76806d180 2024-03-07T06:51:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.989+0000 7fe770600640 1 -- 172.21.15.133:0/1602856133 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7681651b0 con 0x7fe76806d180 2024-03-07T06:51:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.989+0000 7fe765600640 1 -- 172.21.15.133:0/1602856133 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c004bd0 con 0x7fe76806d180 2024-03-07T06:51:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.989+0000 7fe770600640 1 -- 172.21.15.133:0/1602856133 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe768165a10 con 0x7fe76806d180 2024-03-07T06:51:58.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.990+0000 7fe770600640 1 -- 172.21.15.133:0/1602856133 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe768062c90 con 0x7fe76806d180 2024-03-07T06:51:58.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.991+0000 7fe765600640 1 -- 172.21.15.133:0/1602856133 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe75c01a020 con 0x7fe76806d180 2024-03-07T06:51:58.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.992+0000 7fe765600640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe74806b4d0 0x7fe74806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:58.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.992+0000 7fe765600640 1 -- 172.21.15.133:0/1602856133 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe75c089b10 con 0x7fe76806d180 2024-03-07T06:51:58.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.992+0000 7fe76d200640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe74806b4d0 0x7fe74806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:58.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.993+0000 7fe76d200640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe74806b4d0 0x7fe74806d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe768140020 tx=0x7fe754008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:58.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:58.996+0000 7fe765600640 1 -- 172.21.15.133:0/1602856133 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe75c010300 con 0x7fe76806d180 2024-03-07T06:51:59.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.190+0000 7fe770600640 1 -- 172.21.15.133:0/1602856133 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe76814f380 con 0x7fe76806d180 2024-03-07T06:51:59.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.191+0000 7fe765600640 1 -- 172.21.15.133:0/1602856133 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe75c058950 con 0x7fe76806d180 2024-03-07T06:51:59.191 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:51:59.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.194+0000 7fe770600640 1 -- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe74806b4d0 msgr2=0x7fe74806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.194+0000 7fe770600640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe74806b4d0 0x7fe74806d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe768140020 tx=0x7fe754008040 comp rx=0 tx=0).stop 2024-03-07T06:51:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.194+0000 7fe770600640 1 -- 172.21.15.133:0/1602856133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe76806d180 msgr2=0x7fe768167100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.194+0000 7fe770600640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe76806d180 0x7fe768167100 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00c390 tx=0x7fe75c00c850 comp rx=0 tx=0).stop 2024-03-07T06:51:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.194+0000 7fe770600640 1 -- 172.21.15.133:0/1602856133 shutdown_connections 2024-03-07T06:51:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.194+0000 7fe770600640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe76815b370 0x7fe76813ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.194+0000 7fe770600640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe74806b4d0 0x7fe74806d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.194+0000 7fe770600640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe7680b3ed0 0x7fe768140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.194+0000 7fe770600640 1 --2- 172.21.15.133:0/1602856133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe76806d180 0x7fe768167100 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.194+0000 7fe770600640 1 -- 172.21.15.133:0/1602856133 >> 172.21.15.133:0/1602856133 conn(0x7fe7680b1d30 msgr2=0x7fe76815a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.194+0000 7fe770600640 1 -- 172.21.15.133:0/1602856133 shutdown_connections 2024-03-07T06:51:59.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.194+0000 7fe770600640 1 -- 172.21.15.133:0/1602856133 wait complete. 2024-03-07T06:51:59.196 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:51:59.201 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:51:59.212 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:51:59.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.453+0000 7fe349a00640 1 Processor -- start 2024-03-07T06:51:59.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.453+0000 7fe349a00640 1 -- start start 2024-03-07T06:51:59.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.455+0000 7fe349a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe34415e350 0x7fe344161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.455+0000 7fe349a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe344071990 0x7fe3441623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.455+0000 7fe349a00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe344071ff0 con 0x7fe3440b7200 2024-03-07T06:51:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.455+0000 7fe349a00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3440d2160 con 0x7fe34415e350 2024-03-07T06:51:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.455+0000 7fe349a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3440d23b0 con 0x7fe344071990 2024-03-07T06:51:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.455+0000 7fe342a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe34415e350 0x7fe344161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.455+0000 7fe342a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe34415e350 0x7fe344161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37364/0 (socket says 172.21.15.133:37364) 2024-03-07T06:51:59.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.455+0000 7fe342a00640 1 -- 172.21.15.133:0/235322550 learned_addr learned my addr 172.21.15.133:0/235322550 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.455+0000 7fe343400640 1 --2- 172.21.15.133:0/235322550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe344071990 0x7fe3441623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.455+0000 7fe342a00640 1 -- 172.21.15.133:0/235322550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe344071990 msgr2=0x7fe3441623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.455+0000 7fe342a00640 1 --2- 172.21.15.133:0/235322550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe344071990 0x7fe3441623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.455+0000 7fe342a00640 1 -- 172.21.15.133:0/235322550 >> v1:172.21.15.133:6789/0 conn(0x7fe3440b7200 legacy=0x7fe344071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.456+0000 7fe342a00640 1 -- 172.21.15.133:0/235322550 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3440d2690 con 0x7fe34415e350 2024-03-07T06:51:59.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.456+0000 7fe342000640 1 -- 172.21.15.133:0/235322550 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011348178 0 0) 0x7fe344071ff0 con 0x7fe3440b7200 2024-03-07T06:51:59.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.457+0000 7fe342a00640 1 --2- 172.21.15.133:0/235322550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe34415e350 0x7fe344161eb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe334009870 tx=0x7fe33402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf46fa791cb36504 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:59.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.457+0000 7fe342000640 1 -- 172.21.15.133:0/235322550 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334034070 con 0x7fe34415e350 2024-03-07T06:51:59.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.457+0000 7fe342000640 1 -- 172.21.15.133:0/235322550 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe334030840 con 0x7fe34415e350 2024-03-07T06:51:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.457+0000 7fe342000640 1 -- 172.21.15.133:0/235322550 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334030c10 con 0x7fe34415e350 2024-03-07T06:51:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.457+0000 7fe349a00640 1 -- 172.21.15.133:0/235322550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe34415e350 msgr2=0x7fe344161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.457+0000 7fe349a00640 1 --2- 172.21.15.133:0/235322550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe34415e350 0x7fe344161eb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe334009870 tx=0x7fe33402b980 comp rx=0 tx=0).stop 2024-03-07T06:51:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.458+0000 7fe349a00640 1 -- 172.21.15.133:0/235322550 shutdown_connections 2024-03-07T06:51:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.458+0000 7fe349a00640 1 --2- 172.21.15.133:0/235322550 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe344071990 0x7fe3441623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.458+0000 7fe349a00640 1 --2- 172.21.15.133:0/235322550 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe34415e350 0x7fe344161eb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.458+0000 7fe349a00640 1 -- 172.21.15.133:0/235322550 >> 172.21.15.133:0/235322550 conn(0x7fe3440d0e70 msgr2=0x7fe3440d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.458+0000 7fe349a00640 1 -- 172.21.15.133:0/235322550 shutdown_connections 2024-03-07T06:51:59.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.458+0000 7fe349a00640 1 -- 172.21.15.133:0/235322550 wait complete. 2024-03-07T06:51:59.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.459+0000 7fe349a00640 1 Processor -- start 2024-03-07T06:51:59.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.459+0000 7fe349a00640 1 -- start start 2024-03-07T06:51:59.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.460+0000 7fe349a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe344071990 0x7fe3440cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.460+0000 7fe349a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3440b7200 0x7fe34415a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.460+0000 7fe349a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe34415e350 0x7fe34415bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.460+0000 7fe349a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3440d2e90 con 0x7fe3440b7200 2024-03-07T06:51:59.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.460+0000 7fe349a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34415e730 con 0x7fe344071990 2024-03-07T06:51:59.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.460+0000 7fe349a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3440d3010 con 0x7fe34415e350 2024-03-07T06:51:59.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.461+0000 7fe33be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3440b7200 0x7fe34415a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:59.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.461+0000 7fe33be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3440b7200 0x7fe34415a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37366/0 (socket says 172.21.15.133:37366) 2024-03-07T06:51:59.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.461+0000 7fe33be00640 1 -- 172.21.15.133:0/1361574010 learned_addr learned my addr 172.21.15.133:0/1361574010 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:59.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.461+0000 7fe342a00640 1 --2- 172.21.15.133:0/1361574010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe344071990 0x7fe3440cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:59.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.461+0000 7fe33be00640 1 -- 172.21.15.133:0/1361574010 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe34415e350 msgr2=0x7fe34415bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:59.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.461+0000 7fe33be00640 1 --2- 172.21.15.133:0/1361574010 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe34415e350 0x7fe34415bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.461+0000 7fe33be00640 1 -- 172.21.15.133:0/1361574010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe344071990 msgr2=0x7fe3440cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:59.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.461+0000 7fe33be00640 1 --2- 172.21.15.133:0/1361574010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe344071990 0x7fe3440cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.461+0000 7fe33be00640 1 -- 172.21.15.133:0/1361574010 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3441600c0 con 0x7fe3440b7200 2024-03-07T06:51:59.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.461+0000 7fe342a00640 1 --2- 172.21.15.133:0/1361574010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe344071990 0x7fe3440cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:59.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.461+0000 7fe33be00640 1 --2- 172.21.15.133:0/1361574010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3440b7200 0x7fe34415a860 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00c4b0 tx=0x7fe32c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:59.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.462+0000 7fe33aa00640 1 -- 172.21.15.133:0/1361574010 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c017070 con 0x7fe3440b7200 2024-03-07T06:51:59.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.462+0000 7fe33aa00640 1 -- 172.21.15.133:0/1361574010 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe32c005480 con 0x7fe3440b7200 2024-03-07T06:51:59.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.462+0000 7fe33aa00640 1 -- 172.21.15.133:0/1361574010 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32c003830 con 0x7fe3440b7200 2024-03-07T06:51:59.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.462+0000 7fe349a00640 1 -- 172.21.15.133:0/1361574010 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3440cc320 con 0x7fe3440b7200 2024-03-07T06:51:59.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.462+0000 7fe349a00640 1 -- 172.21.15.133:0/1361574010 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3440cc5c0 con 0x7fe3440b7200 2024-03-07T06:51:59.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.463+0000 7fe349a00640 1 -- 172.21.15.133:0/1361574010 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe344062c90 con 0x7fe3440b7200 2024-03-07T06:51:59.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.464+0000 7fe33aa00640 1 -- 172.21.15.133:0/1361574010 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe32c01a070 con 0x7fe3440b7200 2024-03-07T06:51:59.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.465+0000 7fe33aa00640 1 --2- 172.21.15.133:0/1361574010 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe32006b4d0 0x7fe32006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.465+0000 7fe33aa00640 1 -- 172.21.15.133:0/1361574010 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe32c089640 con 0x7fe3440b7200 2024-03-07T06:51:59.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.465+0000 7fe342a00640 1 --2- 172.21.15.133:0/1361574010 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe32006b4d0 0x7fe32006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:59.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.467+0000 7fe342a00640 1 --2- 172.21.15.133:0/1361574010 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe32006b4d0 0x7fe32006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe334036000 tx=0x7fe334008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:59.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.470+0000 7fe33aa00640 1 -- 172.21.15.133:0/1361574010 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe32c058430 con 0x7fe3440b7200 2024-03-07T06:51:59.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.661+0000 7fe349a00640 1 -- 172.21.15.133:0/1361574010 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3440b91e0 con 0x7fe3440b7200 2024-03-07T06:51:59.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.662+0000 7fe33aa00640 1 -- 172.21.15.133:0/1361574010 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe32c057d30 con 0x7fe3440b7200 2024-03-07T06:51:59.662 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:51:59.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.664+0000 7fe349a00640 1 -- 172.21.15.133:0/1361574010 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe32006b4d0 msgr2=0x7fe32006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:59.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.664+0000 7fe349a00640 1 --2- 172.21.15.133:0/1361574010 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe32006b4d0 0x7fe32006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe334036000 tx=0x7fe334008030 comp rx=0 tx=0).stop 2024-03-07T06:51:59.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.664+0000 7fe349a00640 1 -- 172.21.15.133:0/1361574010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3440b7200 msgr2=0x7fe34415a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:59.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.664+0000 7fe349a00640 1 --2- 172.21.15.133:0/1361574010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3440b7200 0x7fe34415a860 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00c4b0 tx=0x7fe32c00c970 comp rx=0 tx=0).stop 2024-03-07T06:51:59.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.665+0000 7fe349a00640 1 -- 172.21.15.133:0/1361574010 shutdown_connections 2024-03-07T06:51:59.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.665+0000 7fe349a00640 1 --2- 172.21.15.133:0/1361574010 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe34415e350 0x7fe34415bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.665+0000 7fe349a00640 1 --2- 172.21.15.133:0/1361574010 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3440b7200 0x7fe34415a860 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.665+0000 7fe349a00640 1 --2- 172.21.15.133:0/1361574010 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe32006b4d0 0x7fe32006d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.665+0000 7fe349a00640 1 --2- 172.21.15.133:0/1361574010 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe344071990 0x7fe3440cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.665+0000 7fe349a00640 1 -- 172.21.15.133:0/1361574010 >> 172.21.15.133:0/1361574010 conn(0x7fe3440d0e70 msgr2=0x7fe3440b8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:59.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.665+0000 7fe349a00640 1 -- 172.21.15.133:0/1361574010 shutdown_connections 2024-03-07T06:51:59.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.665+0000 7fe349a00640 1 -- 172.21.15.133:0/1361574010 wait complete. 2024-03-07T06:51:59.683 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_strays.TestStrays.test_dir_deletion' 2024-03-07T06:51:59.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.887+0000 7faf8e600640 1 Processor -- start 2024-03-07T06:51:59.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.887+0000 7faf8e600640 1 -- start start 2024-03-07T06:51:59.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.888+0000 7faf8e600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf8815b1b0 0x7faf880b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.888+0000 7faf8e600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faf880ce5f0 0x7faf88071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.889+0000 7faf8e600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf8806f140 con 0x7faf880af950 2024-03-07T06:51:59.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.889+0000 7faf8e600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf88071610 con 0x7faf880ce5f0 2024-03-07T06:51:59.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.889+0000 7faf8e600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8806f420 con 0x7faf8815b1b0 2024-03-07T06:51:59.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.889+0000 7faf86a00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7faf880af950 0x7faf880cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:57112/0 (socket says 172.21.15.133:57112) 2024-03-07T06:51:59.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.889+0000 7faf86a00640 1 -- 172.21.15.133:0/2085078138 learned_addr learned my addr 172.21.15.133:0/2085078138 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:51:59.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.889+0000 7faf87400640 1 --2- 172.21.15.133:0/2085078138 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf8815b1b0 0x7faf880b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:59.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.889+0000 7faf87e00640 1 --2- 172.21.15.133:0/2085078138 >> v2:172.21.15.161:3300/0 conn(0x7faf880ce5f0 0x7faf88071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:59.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.889+0000 7faf86000640 1 -- 172.21.15.133:0/2085078138 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3028987403 0 0) 0x7faf8806f140 con 0x7faf880af950 2024-03-07T06:51:59.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.889+0000 7faf86000640 1 -- 172.21.15.133:0/2085078138 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf6c003520 con 0x7faf880af950 2024-03-07T06:51:59.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86000640 1 -- 172.21.15.133:0/2085078138 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3583609858 0 0) 0x7faf6c003520 con 0x7faf880af950 2024-03-07T06:51:59.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86000640 1 -- 172.21.15.133:0/2085078138 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf8815b1b0 msgr2=0x7faf880b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:59.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86000640 1 --2- 172.21.15.133:0/2085078138 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf8815b1b0 0x7faf880b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86000640 1 -- 172.21.15.133:0/2085078138 >> v2:172.21.15.161:3300/0 conn(0x7faf880ce5f0 msgr2=0x7faf88071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:59.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86000640 1 --2- 172.21.15.133:0/2085078138 >> v2:172.21.15.161:3300/0 conn(0x7faf880ce5f0 0x7faf88071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86000640 1 -- 172.21.15.133:0/2085078138 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf88071860 con 0x7faf880af950 2024-03-07T06:51:59.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86000640 1 -- 172.21.15.133:0/2085078138 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7faf74001670 con 0x7faf880af950 2024-03-07T06:51:59.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86000640 1 -- 172.21.15.133:0/2085078138 >> v1:172.21.15.133:6789/0 conn(0x7faf880af950 legacy=0x7faf880cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:59.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86000640 1 --2- 172.21.15.133:0/2085078138 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf8815b1b0 0x7faf6c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86000640 1 --2- 172.21.15.133:0/2085078138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf6c003f80 0x7faf6c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86000640 1 --2- 172.21.15.133:0/2085078138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf6c004d40 0x7faf6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86000640 1 -- 172.21.15.133:0/2085078138 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf88071610 con 0x7faf6c003f80 2024-03-07T06:51:59.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86000640 1 -- 172.21.15.133:0/2085078138 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8806f420 con 0x7faf6c004d40 2024-03-07T06:51:59.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86000640 1 -- 172.21.15.133:0/2085078138 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c00dc80 con 0x7faf8815b1b0 2024-03-07T06:51:59.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86a00640 1 --2- 172.21.15.133:0/2085078138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf6c003f80 0x7faf6c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:59.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf87400640 1 --2- 172.21.15.133:0/2085078138 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf8815b1b0 0x7faf6c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:59.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf87e00640 1 --2- 172.21.15.133:0/2085078138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf6c004d40 0x7faf6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:59.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86a00640 1 -- 172.21.15.133:0/2085078138 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf8815b1b0 msgr2=0x7faf6c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:59.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86a00640 1 --2- 172.21.15.133:0/2085078138 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf8815b1b0 0x7faf6c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86a00640 1 -- 172.21.15.133:0/2085078138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf6c004d40 msgr2=0x7faf6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:51:59.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86a00640 1 --2- 172.21.15.133:0/2085078138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf6c004d40 0x7faf6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf86a00640 1 -- 172.21.15.133:0/2085078138 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c003520 con 0x7faf6c003f80 2024-03-07T06:51:59.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.890+0000 7faf87e00640 1 --2- 172.21.15.133:0/2085078138 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf6c004d40 0x7faf6c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:59.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.891+0000 7faf86a00640 1 --2- 172.21.15.133:0/2085078138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf6c003f80 0x7faf6c004800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf7400c190 tx=0x7faf6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde4f0b0ab5b4834 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:59.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.892+0000 7faf86000640 1 -- 172.21.15.133:0/2085078138 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf74007c60 con 0x7faf6c003f80 2024-03-07T06:51:59.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.892+0000 7faf86000640 1 -- 172.21.15.133:0/2085078138 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf74030070 con 0x7faf6c003f80 2024-03-07T06:51:59.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.892+0000 7faf86000640 1 -- 172.21.15.133:0/2085078138 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf74005d30 con 0x7faf6c003f80 2024-03-07T06:51:59.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.892+0000 7faf8e600640 1 -- 172.21.15.133:0/2085078138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf6c003f80 msgr2=0x7faf6c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:59.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.892+0000 7faf8e600640 1 --2- 172.21.15.133:0/2085078138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf6c003f80 0x7faf6c004800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf7400c190 tx=0x7faf6c005830 comp rx=0 tx=0).stop 2024-03-07T06:51:59.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.892+0000 7faf87e00640 1 -- 172.21.15.133:0/2085078138 reap_dead start 2024-03-07T06:51:59.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.892+0000 7faf8e600640 1 -- 172.21.15.133:0/2085078138 shutdown_connections 2024-03-07T06:51:59.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.892+0000 7faf8e600640 1 -- 172.21.15.133:0/2085078138 >> 172.21.15.133:0/2085078138 conn(0x7faf880b3120 msgr2=0x7faf880b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:51:59.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.892+0000 7faf8e600640 1 -- 172.21.15.133:0/2085078138 shutdown_connections 2024-03-07T06:51:59.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.892+0000 7faf8e600640 1 -- 172.21.15.133:0/2085078138 wait complete. 2024-03-07T06:51:59.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.894+0000 7faf8e600640 1 Processor -- start 2024-03-07T06:51:59.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.894+0000 7faf8e600640 1 -- start start 2024-03-07T06:51:59.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.895+0000 7faf8e600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf6c003f80 0x7faf8809a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.895+0000 7faf8e600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf881f71e0 0x7faf8809d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.895+0000 7faf8e600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf881519d0 0x7faf880724d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.895+0000 7faf8e600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8815b590 con 0x7faf6c003f80 2024-03-07T06:51:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.895+0000 7faf8e600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8806ee30 con 0x7faf881519d0 2024-03-07T06:51:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.895+0000 7faf8e600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8815b710 con 0x7faf881f71e0 2024-03-07T06:51:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.896+0000 7faf87400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf6c003f80 0x7faf8809a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.896+0000 7faf87400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf6c003f80 0x7faf8809a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37372/0 (socket says 172.21.15.133:37372) 2024-03-07T06:51:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.896+0000 7faf87400640 1 -- 172.21.15.133:0/1474967898 learned_addr learned my addr 172.21.15.133:0/1474967898 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:51:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.896+0000 7faf86a00640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf881f71e0 0x7faf8809d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.896+0000 7faf87e00640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf881519d0 0x7faf880724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.896+0000 7faf87400640 1 -- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf881f71e0 msgr2=0x7faf8809d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.896+0000 7faf87400640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf881f71e0 0x7faf8809d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.896+0000 7faf87400640 1 -- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf881519d0 msgr2=0x7faf880724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:51:59.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.896+0000 7faf87400640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf881519d0 0x7faf880724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:51:59.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.896+0000 7faf87400640 1 -- 172.21.15.133:0/1474967898 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf881f76e0 con 0x7faf6c003f80 2024-03-07T06:51:59.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.896+0000 7faf87e00640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf881519d0 0x7faf880724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:51:59.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.896+0000 7faf87400640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf6c003f80 0x7faf8809a9d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf78007f90 tx=0x7faf78011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:59.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.897+0000 7faf7f400640 1 -- 172.21.15.133:0/1474967898 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7801a020 con 0x7faf6c003f80 2024-03-07T06:51:59.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.897+0000 7faf7f400640 1 -- 172.21.15.133:0/1474967898 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf78016470 con 0x7faf6c003f80 2024-03-07T06:51:59.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.897+0000 7faf7f400640 1 -- 172.21.15.133:0/1474967898 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78016860 con 0x7faf6c003f80 2024-03-07T06:51:59.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.897+0000 7faf8e600640 1 -- 172.21.15.133:0/1474967898 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf881f79c0 con 0x7faf6c003f80 2024-03-07T06:51:59.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.897+0000 7faf8e600640 1 -- 172.21.15.133:0/1474967898 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf881f7f80 con 0x7faf6c003f80 2024-03-07T06:51:59.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.898+0000 7faf8e600640 1 -- 172.21.15.133:0/1474967898 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf88062c90 con 0x7faf6c003f80 2024-03-07T06:51:59.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.899+0000 7faf7f400640 1 -- 172.21.15.133:0/1474967898 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7faf78005d10 con 0x7faf6c003f80 2024-03-07T06:51:59.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.900+0000 7faf7f400640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7faf6806b4d0 0x7faf6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:51:59.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.900+0000 7faf7f400640 1 -- 172.21.15.133:0/1474967898 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7faf7808b140 con 0x7faf6c003f80 2024-03-07T06:51:59.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.900+0000 7faf86a00640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7faf6806b4d0 0x7faf6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:51:59.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.901+0000 7faf86a00640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7faf6806b4d0 0x7faf6806d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faf74034000 tx=0x7faf74007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:51:59.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:51:59.904+0000 7faf7f400640 1 -- 172.21.15.133:0/1474967898 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faf78024030 con 0x7faf6c003f80 2024-03-07T06:52:00.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.151+0000 7faf8e600640 1 -- 172.21.15.133:0/1474967898 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_strays.TestStrays.test_dir_deletion"]} v 0) v1 -- 0x7faf880cc660 con 0x7faf6c003f80 2024-03-07T06:52:00.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.454+0000 7faf7f400640 1 -- 172.21.15.133:0/1474967898 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_strays.TestStrays.test_dir_deletion"]}]=0 v66) v1 ==== 135+0+0 (secure 0 0 0) 0x7faf78059ea0 con 0x7faf6c003f80 2024-03-07T06:52:00.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.457+0000 7faf8e600640 1 -- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7faf6806b4d0 msgr2=0x7faf6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:00.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.457+0000 7faf8e600640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7faf6806b4d0 0x7faf6806d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faf74034000 tx=0x7faf74007000 comp rx=0 tx=0).stop 2024-03-07T06:52:00.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.457+0000 7faf8e600640 1 -- 172.21.15.133:0/1474967898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf6c003f80 msgr2=0x7faf8809a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:00.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.457+0000 7faf8e600640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf6c003f80 0x7faf8809a9d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faf78007f90 tx=0x7faf78011660 comp rx=0 tx=0).stop 2024-03-07T06:52:00.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.457+0000 7faf8e600640 1 -- 172.21.15.133:0/1474967898 shutdown_connections 2024-03-07T06:52:00.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.457+0000 7faf8e600640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7faf6806b4d0 0x7faf6806d990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:00.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.457+0000 7faf8e600640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faf881519d0 0x7faf880724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:00.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.457+0000 7faf8e600640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faf881f71e0 0x7faf8809d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:00.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.457+0000 7faf8e600640 1 --2- 172.21.15.133:0/1474967898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf6c003f80 0x7faf8809a9d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:00.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.457+0000 7faf8e600640 1 -- 172.21.15.133:0/1474967898 >> 172.21.15.133:0/1474967898 conn(0x7faf880b3120 msgr2=0x7faf88071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:00.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.457+0000 7faf8e600640 1 -- 172.21.15.133:0/1474967898 shutdown_connections 2024-03-07T06:52:00.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.457+0000 7faf8e600640 1 -- 172.21.15.133:0/1474967898 wait complete. 2024-03-07T06:52:00.480 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-07T06:52:00.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.691+0000 7effde800640 1 Processor -- start 2024-03-07T06:52:00.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.691+0000 7effde800640 1 -- start start 2024-03-07T06:52:00.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.693+0000 7effde800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7effd8144040 0x7effd80cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:00.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.693+0000 7effde800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7effd806ea40 0x7effd806ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:00.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.693+0000 7effde800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd80ced30 con 0x7effd8143950 2024-03-07T06:52:00.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.693+0000 7effde800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd80cce40 con 0x7effd8144040 2024-03-07T06:52:00.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.693+0000 7effde800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd806f390 con 0x7effd806ea40 2024-03-07T06:52:00.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.693+0000 7effdca00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7effd8144040 0x7effd80cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:00.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.693+0000 7effdca00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7effd8144040 0x7effd80cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37376/0 (socket says 172.21.15.133:37376) 2024-03-07T06:52:00.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.693+0000 7effdca00640 1 -- 172.21.15.133:0/1581675871 learned_addr learned my addr 172.21.15.133:0/1581675871 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:00.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.693+0000 7effdde00640 1 --2- 172.21.15.133:0/1581675871 >> v2:172.21.15.161:3300/0 conn(0x7effd806ea40 0x7effd806ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:00.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.694+0000 7effdca00640 1 -- 172.21.15.133:0/1581675871 >> v2:172.21.15.161:3300/0 conn(0x7effd806ea40 msgr2=0x7effd806ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:00.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.694+0000 7effdca00640 1 --2- 172.21.15.133:0/1581675871 >> v2:172.21.15.161:3300/0 conn(0x7effd806ea40 0x7effd806ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:00.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.694+0000 7effdca00640 1 -- 172.21.15.133:0/1581675871 >> v1:172.21.15.161:6789/0 conn(0x7effd8143950 legacy=0x7effd806d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:00.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.694+0000 7effdca00640 1 -- 172.21.15.133:0/1581675871 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd80cc5e0 con 0x7effd8144040 2024-03-07T06:52:00.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.695+0000 7effdca00640 1 --2- 172.21.15.133:0/1581675871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd8144040 0x7effd80cf4d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7effd80b2920 tx=0x7effc402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe60d2a293fadc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:00.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.695+0000 7effd7e00640 1 -- 172.21.15.133:0/1581675871 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4035020 con 0x7effd8144040 2024-03-07T06:52:00.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.695+0000 7effd7e00640 1 -- 172.21.15.133:0/1581675871 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc4031870 con 0x7effd8144040 2024-03-07T06:52:00.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.695+0000 7effd7e00640 1 -- 172.21.15.133:0/1581675871 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4031c40 con 0x7effd8144040 2024-03-07T06:52:00.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.696+0000 7effde800640 1 -- 172.21.15.133:0/1581675871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd8144040 msgr2=0x7effd80cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:00.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.696+0000 7effde800640 1 --2- 172.21.15.133:0/1581675871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd8144040 0x7effd80cf4d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7effd80b2920 tx=0x7effc402b910 comp rx=0 tx=0).stop 2024-03-07T06:52:00.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.696+0000 7effde800640 1 -- 172.21.15.133:0/1581675871 shutdown_connections 2024-03-07T06:52:00.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.696+0000 7effde800640 1 --2- 172.21.15.133:0/1581675871 >> v2:172.21.15.161:3300/0 conn(0x7effd806ea40 0x7effd806ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:00.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.696+0000 7effde800640 1 --2- 172.21.15.133:0/1581675871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd8144040 0x7effd80cf4d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:00.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.696+0000 7effde800640 1 -- 172.21.15.133:0/1581675871 >> 172.21.15.133:0/1581675871 conn(0x7effd8071860 msgr2=0x7effd8071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:00.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.696+0000 7effde800640 1 -- 172.21.15.133:0/1581675871 shutdown_connections 2024-03-07T06:52:00.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.696+0000 7effde800640 1 -- 172.21.15.133:0/1581675871 wait complete. 2024-03-07T06:52:00.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.698+0000 7effde800640 1 Processor -- start 2024-03-07T06:52:00.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.698+0000 7effde800640 1 -- start start 2024-03-07T06:52:00.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.699+0000 7effde800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7effd806ea40 0x7effd813dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:00.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.699+0000 7effde800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd8143950 0x7effd81f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:00.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.699+0000 7effde800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd8144040 0x7effd81f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:00.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.699+0000 7effde800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd80cefb0 con 0x7effd8144040 2024-03-07T06:52:00.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.699+0000 7effde800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd80ccb60 con 0x7effd8143950 2024-03-07T06:52:00.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.699+0000 7effde800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd8143d30 con 0x7effd806ea40 2024-03-07T06:52:00.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.699+0000 7effdde00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd8144040 0x7effd81f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:00.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.699+0000 7effdde00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd8144040 0x7effd81f77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37392/0 (socket says 172.21.15.133:37392) 2024-03-07T06:52:00.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.699+0000 7effdde00640 1 -- 172.21.15.133:0/1821600526 learned_addr learned my addr 172.21.15.133:0/1821600526 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:00.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effdca00640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd8143950 0x7effd81f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:00.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effdd400640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7effd806ea40 0x7effd813dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:00.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effdde00640 1 -- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7effd806ea40 msgr2=0x7effd813dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:00.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effdde00640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7effd806ea40 0x7effd813dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:00.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effdde00640 1 -- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd8143950 msgr2=0x7effd81f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:00.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effdde00640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd8143950 0x7effd81f72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:00.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effdde00640 1 -- 172.21.15.133:0/1821600526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd813e9b0 con 0x7effd8144040 2024-03-07T06:52:00.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effdca00640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd8143950 0x7effd81f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:00.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effdd400640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7effd806ea40 0x7effd813dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:00.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effdde00640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd8144040 0x7effd81f77e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7effd0007e60 tx=0x7effd000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:00.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effd5600640 1 -- 172.21.15.133:0/1821600526 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0014420 con 0x7effd8144040 2024-03-07T06:52:00.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effd5600640 1 -- 172.21.15.133:0/1821600526 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd0019070 con 0x7effd8144040 2024-03-07T06:52:00.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effde800640 1 -- 172.21.15.133:0/1821600526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd813ec90 con 0x7effd8144040 2024-03-07T06:52:00.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effd5600640 1 -- 172.21.15.133:0/1821600526 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd0014820 con 0x7effd8144040 2024-03-07T06:52:00.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.700+0000 7effde800640 1 -- 172.21.15.133:0/1821600526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd813f250 con 0x7effd8144040 2024-03-07T06:52:00.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.701+0000 7effde800640 1 -- 172.21.15.133:0/1821600526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd8067c80 con 0x7effd8144040 2024-03-07T06:52:00.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.702+0000 7effd5600640 1 -- 172.21.15.133:0/1821600526 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7effd00149c0 con 0x7effd8144040 2024-03-07T06:52:00.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.703+0000 7effd5600640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7effb806b4d0 0x7effb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:00.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.703+0000 7effd5600640 1 -- 172.21.15.133:0/1821600526 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7effd008aaf0 con 0x7effd8144040 2024-03-07T06:52:00.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.704+0000 7effdd400640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7effb806b4d0 0x7effb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:00.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.705+0000 7effdd400640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7effb806b4d0 0x7effb806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7effd80b1750 tx=0x7effcc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:00.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.708+0000 7effd5600640 1 -- 172.21.15.133:0/1821600526 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7effd00598e0 con 0x7effd8144040 2024-03-07T06:52:00.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.899+0000 7effde800640 1 -- 172.21.15.133:0/1821600526 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7effd8062c90 con 0x7effd8144040 2024-03-07T06:52:00.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.905+0000 7effd5600640 1 -- 172.21.15.133:0/1821600526 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7effd00591e0 con 0x7effd8144040 2024-03-07T06:52:00.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.907+0000 7effde800640 1 -- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7effb806b4d0 msgr2=0x7effb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:00.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.907+0000 7effde800640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7effb806b4d0 0x7effb806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7effd80b1750 tx=0x7effcc008040 comp rx=0 tx=0).stop 2024-03-07T06:52:00.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.907+0000 7effde800640 1 -- 172.21.15.133:0/1821600526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd8144040 msgr2=0x7effd81f77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:00.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.907+0000 7effde800640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd8144040 0x7effd81f77e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7effd0007e60 tx=0x7effd000e4b0 comp rx=0 tx=0).stop 2024-03-07T06:52:00.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.907+0000 7effde800640 1 -- 172.21.15.133:0/1821600526 shutdown_connections 2024-03-07T06:52:00.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.907+0000 7effde800640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7effd8144040 0x7effd81f77e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:00.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.907+0000 7effde800640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7effb806b4d0 0x7effb806d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:00.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.907+0000 7effde800640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7effd8143950 0x7effd81f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:00.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.907+0000 7effde800640 1 --2- 172.21.15.133:0/1821600526 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7effd806ea40 0x7effd813dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:00.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.907+0000 7effde800640 1 -- 172.21.15.133:0/1821600526 >> 172.21.15.133:0/1821600526 conn(0x7effd8071860 msgr2=0x7effd8141040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:00.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.908+0000 7effde800640 1 -- 172.21.15.133:0/1821600526 shutdown_connections 2024-03-07T06:52:00.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:00.908+0000 7effde800640 1 -- 172.21.15.133:0/1821600526 wait complete. 2024-03-07T06:52:00.925 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-07T06:52:00.925 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-07T06:52:01.040 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-07T06:52:01.160 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-07T06:52:01.160 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:52:01.160 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:52:01.215 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:52:01.215 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:52:01.271 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:52:01.271 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:52:01.325 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:52:01.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.536+0000 7f0e87800640 1 Processor -- start 2024-03-07T06:52:01.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.536+0000 7f0e87800640 1 -- start start 2024-03-07T06:52:01.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.538+0000 7f0e87800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0e8015b1b0 0x7f0e800b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:01.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.538+0000 7f0e87800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0e800af950 0x7f0e800cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:01.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.538+0000 7f0e87800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e800ce830 con 0x7f0e800ce420 2024-03-07T06:52:01.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.538+0000 7f0e87800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e80071610 con 0x7f0e8015b1b0 2024-03-07T06:52:01.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.538+0000 7f0e87800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8006f140 con 0x7f0e800af950 2024-03-07T06:52:01.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.538+0000 7f0e84e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0e8015b1b0 0x7f0e800b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:01.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.538+0000 7f0e84e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0e8015b1b0 0x7f0e800b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47278/0 (socket says 172.21.15.133:47278) 2024-03-07T06:52:01.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.538+0000 7f0e84e00640 1 -- 172.21.15.133:0/210366884 learned_addr learned my addr 172.21.15.133:0/210366884 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:01.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.538+0000 7f0e7fe00640 1 --2- 172.21.15.133:0/210366884 >> v2:172.21.15.161:3300/0 conn(0x7f0e800af950 0x7f0e800cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:01.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.538+0000 7f0e84e00640 1 -- 172.21.15.133:0/210366884 >> v2:172.21.15.161:3300/0 conn(0x7f0e800af950 msgr2=0x7f0e800cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:01.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.538+0000 7f0e84e00640 1 --2- 172.21.15.133:0/210366884 >> v2:172.21.15.161:3300/0 conn(0x7f0e800af950 0x7f0e800cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.538+0000 7f0e84e00640 1 -- 172.21.15.133:0/210366884 >> v1:172.21.15.133:6789/0 conn(0x7f0e800ce420 legacy=0x7f0e8006de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:01.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.538+0000 7f0e84e00640 1 -- 172.21.15.133:0/210366884 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8006f420 con 0x7f0e8015b1b0 2024-03-07T06:52:01.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.540+0000 7f0e84e00640 1 --2- 172.21.15.133:0/210366884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015b1b0 0x7f0e800b3c10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0e800ceac0 tx=0x7f0e7402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7d7fb640a6bbb server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:01.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.540+0000 7f0e7f400640 1 -- 172.21.15.133:0/210366884 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74035020 con 0x7f0e8015b1b0 2024-03-07T06:52:01.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.540+0000 7f0e7f400640 1 -- 172.21.15.133:0/210366884 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e74031870 con 0x7f0e8015b1b0 2024-03-07T06:52:01.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.540+0000 7f0e7f400640 1 -- 172.21.15.133:0/210366884 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74031c40 con 0x7f0e8015b1b0 2024-03-07T06:52:01.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.541+0000 7f0e87800640 1 -- 172.21.15.133:0/210366884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015b1b0 msgr2=0x7f0e800b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:01.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.541+0000 7f0e87800640 1 --2- 172.21.15.133:0/210366884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015b1b0 0x7f0e800b3c10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0e800ceac0 tx=0x7f0e7402b910 comp rx=0 tx=0).stop 2024-03-07T06:52:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.541+0000 7f0e87800640 1 -- 172.21.15.133:0/210366884 shutdown_connections 2024-03-07T06:52:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.541+0000 7f0e87800640 1 --2- 172.21.15.133:0/210366884 >> v2:172.21.15.161:3300/0 conn(0x7f0e800af950 0x7f0e800cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.541+0000 7f0e87800640 1 --2- 172.21.15.133:0/210366884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015b1b0 0x7f0e800b3c10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.541+0000 7f0e87800640 1 -- 172.21.15.133:0/210366884 >> 172.21.15.133:0/210366884 conn(0x7f0e800b3120 msgr2=0x7f0e800b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.541+0000 7f0e87800640 1 -- 172.21.15.133:0/210366884 shutdown_connections 2024-03-07T06:52:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.541+0000 7f0e87800640 1 -- 172.21.15.133:0/210366884 wait complete. 2024-03-07T06:52:01.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.543+0000 7f0e87800640 1 Processor -- start 2024-03-07T06:52:01.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.543+0000 7f0e87800640 1 -- start start 2024-03-07T06:52:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.544+0000 7f0e87800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e800af950 0x7f0e801482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.544+0000 7f0e87800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e800ce420 0x7f0e8009b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.544+0000 7f0e87800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015b1b0 0x7f0e800acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.544+0000 7f0e87800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8015b590 con 0x7f0e8015b1b0 2024-03-07T06:52:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.544+0000 7f0e87800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800cc880 con 0x7f0e800ce420 2024-03-07T06:52:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.544+0000 7f0e87800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8015b710 con 0x7f0e800af950 2024-03-07T06:52:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.544+0000 7f0e85800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015b1b0 0x7f0e800acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:01.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.544+0000 7f0e85800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015b1b0 0x7f0e800acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47294/0 (socket says 172.21.15.133:47294) 2024-03-07T06:52:01.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.544+0000 7f0e85800640 1 -- 172.21.15.133:0/2928174020 learned_addr learned my addr 172.21.15.133:0/2928174020 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:01.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.544+0000 7f0e7fe00640 1 --2- 172.21.15.133:0/2928174020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e800ce420 0x7f0e8009b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:01.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.545+0000 7f0e85800640 1 -- 172.21.15.133:0/2928174020 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e800af950 msgr2=0x7f0e801482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:01.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.545+0000 7f0e85800640 1 --2- 172.21.15.133:0/2928174020 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e800af950 0x7f0e801482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.545+0000 7f0e85800640 1 -- 172.21.15.133:0/2928174020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e800ce420 msgr2=0x7f0e8009b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:01.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.545+0000 7f0e85800640 1 --2- 172.21.15.133:0/2928174020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e800ce420 0x7f0e8009b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.545+0000 7f0e85800640 1 -- 172.21.15.133:0/2928174020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8009be60 con 0x7f0e8015b1b0 2024-03-07T06:52:01.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.545+0000 7f0e85800640 1 --2- 172.21.15.133:0/2928174020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015b1b0 0x7f0e800acd00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c7a0 tx=0x7f0e7800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:01.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.545+0000 7f0e7cc00640 1 -- 172.21.15.133:0/2928174020 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78003b70 con 0x7f0e8015b1b0 2024-03-07T06:52:01.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.545+0000 7f0e7cc00640 1 -- 172.21.15.133:0/2928174020 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e78003d30 con 0x7f0e8015b1b0 2024-03-07T06:52:01.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.545+0000 7f0e7cc00640 1 -- 172.21.15.133:0/2928174020 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7800aa30 con 0x7f0e8015b1b0 2024-03-07T06:52:01.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.545+0000 7f0e87800640 1 -- 172.21.15.133:0/2928174020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8009c140 con 0x7f0e8015b1b0 2024-03-07T06:52:01.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.545+0000 7f0e87800640 1 -- 172.21.15.133:0/2928174020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8009c700 con 0x7f0e8015b1b0 2024-03-07T06:52:01.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.546+0000 7f0e87800640 1 -- 172.21.15.133:0/2928174020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e800b54d0 con 0x7f0e8015b1b0 2024-03-07T06:52:01.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.547+0000 7f0e7cc00640 1 -- 172.21.15.133:0/2928174020 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0e7800abd0 con 0x7f0e8015b1b0 2024-03-07T06:52:01.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.548+0000 7f0e7cc00640 1 --2- 172.21.15.133:0/2928174020 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0e6006b4d0 0x7f0e6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:01.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.548+0000 7f0e7cc00640 1 -- 172.21.15.133:0/2928174020 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0e7808a1f0 con 0x7f0e8015b1b0 2024-03-07T06:52:01.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.549+0000 7f0e84e00640 1 --2- 172.21.15.133:0/2928174020 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0e6006b4d0 0x7f0e6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:01.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.550+0000 7f0e84e00640 1 --2- 172.21.15.133:0/2928174020 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0e6006b4d0 0x7f0e6006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0e74036000 tx=0x7f0e74030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:01.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.553+0000 7f0e7cc00640 1 -- 172.21.15.133:0/2928174020 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0e800ce830 con 0x7f0e8015b1b0 2024-03-07T06:52:01.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.733+0000 7f0e87800640 1 -- 172.21.15.133:0/2928174020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e800cc660 con 0x7f0e8015b1b0 2024-03-07T06:52:01.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.735+0000 7f0e7cc00640 1 -- 172.21.15.133:0/2928174020 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16901 (secure 0 0 0) 0x7f0e78058f50 con 0x7f0e8015b1b0 2024-03-07T06:52:01.735 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:01.735 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:56.413096+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:52:01.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.738+0000 7f0e87800640 1 -- 172.21.15.133:0/2928174020 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0e6006b4d0 msgr2=0x7f0e6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:01.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.738+0000 7f0e87800640 1 --2- 172.21.15.133:0/2928174020 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0e6006b4d0 0x7f0e6006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0e74036000 tx=0x7f0e74030000 comp rx=0 tx=0).stop 2024-03-07T06:52:01.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.738+0000 7f0e87800640 1 -- 172.21.15.133:0/2928174020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015b1b0 msgr2=0x7f0e800acd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:01.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.738+0000 7f0e87800640 1 --2- 172.21.15.133:0/2928174020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015b1b0 0x7f0e800acd00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c7a0 tx=0x7f0e7800cc60 comp rx=0 tx=0).stop 2024-03-07T06:52:01.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.738+0000 7f0e87800640 1 -- 172.21.15.133:0/2928174020 shutdown_connections 2024-03-07T06:52:01.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.738+0000 7f0e87800640 1 --2- 172.21.15.133:0/2928174020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8015b1b0 0x7f0e800acd00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.738+0000 7f0e87800640 1 --2- 172.21.15.133:0/2928174020 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0e6006b4d0 0x7f0e6006d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.738+0000 7f0e87800640 1 --2- 172.21.15.133:0/2928174020 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e800ce420 0x7f0e8009b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.738+0000 7f0e87800640 1 --2- 172.21.15.133:0/2928174020 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e800af950 0x7f0e801482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.738+0000 7f0e87800640 1 -- 172.21.15.133:0/2928174020 >> 172.21.15.133:0/2928174020 conn(0x7f0e800b3120 msgr2=0x7f0e80072790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.738+0000 7f0e87800640 1 -- 172.21.15.133:0/2928174020 shutdown_connections 2024-03-07T06:52:01.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.738+0000 7f0e87800640 1 -- 172.21.15.133:0/2928174020 wait complete. 2024-03-07T06:52:01.757 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:51:12.858333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:51:13.615348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:52:01.757 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:52:01.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.962+0000 7fe4bdc00640 1 Processor -- start 2024-03-07T06:52:01.962 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.962+0000 7fe4bdc00640 1 -- start start 2024-03-07T06:52:01.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.964+0000 7fe4bdc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe4b8159880 0x7fe4b80cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:01.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.964+0000 7fe4bdc00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b8160720 con 0x7fe4b80b4f10 2024-03-07T06:52:01.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.964+0000 7fe4bdc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b80cde00 con 0x7fe4b80ccff0 2024-03-07T06:52:01.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.964+0000 7fe4bdc00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b80cd560 con 0x7fe4b8159880 2024-03-07T06:52:01.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.964+0000 7fe4bd200640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe4b80b4f10 0x7fe4b815c360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47752/0 (socket says 172.21.15.133:47752) 2024-03-07T06:52:01.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.964+0000 7fe4bd200640 1 -- 172.21.15.133:0/3205338612 learned_addr learned my addr 172.21.15.133:0/3205338612 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:01.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.965+0000 7fe4b7e00640 1 --2- 172.21.15.133:0/3205338612 >> v2:172.21.15.161:3300/0 conn(0x7fe4b8159880 0x7fe4b80cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:01.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.965+0000 7fe4b6a00640 1 -- 172.21.15.133:0/3205338612 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800010080 0 0) 0x7fe4b8160720 con 0x7fe4b80b4f10 2024-03-07T06:52:01.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.965+0000 7fe4b6a00640 1 -- 172.21.15.133:0/3205338612 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe49c003520 con 0x7fe4b80b4f10 2024-03-07T06:52:01.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.965+0000 7fe4b6a00640 1 -- 172.21.15.133:0/3205338612 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1744114441 0 0) 0x7fe49c003520 con 0x7fe4b80b4f10 2024-03-07T06:52:01.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.965+0000 7fe4b6a00640 1 -- 172.21.15.133:0/3205338612 >> v2:172.21.15.161:3300/0 conn(0x7fe4b8159880 msgr2=0x7fe4b80cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:01.970 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.965+0000 7fe4b6a00640 1 --2- 172.21.15.133:0/3205338612 >> v2:172.21.15.161:3300/0 conn(0x7fe4b8159880 0x7fe4b80cf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.965+0000 7fe4b6a00640 1 -- 172.21.15.133:0/3205338612 >> v1:172.21.15.161:6789/0 conn(0x7fe4b80ccff0 legacy=0x7fe4b8160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.965+0000 7fe4b6a00640 1 -- 172.21.15.133:0/3205338612 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b80cd840 con 0x7fe4b80b4f10 2024-03-07T06:52:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.965+0000 7fe4b6a00640 1 -- 172.21.15.133:0/3205338612 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fe4b00019f0 con 0x7fe4b80b4f10 2024-03-07T06:52:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.966+0000 7fe4b6a00640 1 -- 172.21.15.133:0/3205338612 >> v1:172.21.15.133:6789/0 conn(0x7fe4b80b4f10 legacy=0x7fe4b815c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.966+0000 7fe4b6a00640 1 --2- 172.21.15.133:0/3205338612 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe49c0039b0 0x7fe49c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.966+0000 7fe4b6a00640 1 --2- 172.21.15.133:0/3205338612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe49c004800 0x7fe49c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.966+0000 7fe4b6a00640 1 --2- 172.21.15.133:0/3205338612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe49c0051c0 0x7fe49c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.966+0000 7fe4b6a00640 1 -- 172.21.15.133:0/3205338612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b80cd560 con 0x7fe49c0051c0 2024-03-07T06:52:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.966+0000 7fe4b6a00640 1 -- 172.21.15.133:0/3205338612 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c00c060 con 0x7fe49c004800 2024-03-07T06:52:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.966+0000 7fe4b6a00640 1 -- 172.21.15.133:0/3205338612 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c00c260 con 0x7fe49c0039b0 2024-03-07T06:52:01.971 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.966+0000 7fe4b7e00640 1 --2- 172.21.15.133:0/3205338612 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe49c0039b0 0x7fe49c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.966+0000 7fe4b7400640 1 --2- 172.21.15.133:0/3205338612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe49c004800 0x7fe49c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.966+0000 7fe4bd200640 1 --2- 172.21.15.133:0/3205338612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe49c0051c0 0x7fe49c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.967+0000 7fe4bd200640 1 -- 172.21.15.133:0/3205338612 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe49c0039b0 msgr2=0x7fe49c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.967+0000 7fe4bd200640 1 --2- 172.21.15.133:0/3205338612 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe49c0039b0 0x7fe49c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.967+0000 7fe4bd200640 1 -- 172.21.15.133:0/3205338612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe49c004800 msgr2=0x7fe49c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.967+0000 7fe4bd200640 1 --2- 172.21.15.133:0/3205338612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe49c004800 0x7fe49c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.967+0000 7fe4bd200640 1 -- 172.21.15.133:0/3205338612 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c003520 con 0x7fe49c0051c0 2024-03-07T06:52:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.968+0000 7fe4bd200640 1 -- 172.21.15.133:0/3205338612 reap_dead start 2024-03-07T06:52:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.968+0000 7fe4bd200640 1 --2- 172.21.15.133:0/3205338612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe49c0051c0 0x7fe49c00b9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000a850 tx=0x7fe49c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ddf5857ccc6ee82 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.968+0000 7fe4b6a00640 1 -- 172.21.15.133:0/3205338612 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b000a4a0 con 0x7fe49c0051c0 2024-03-07T06:52:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.968+0000 7fe4b6a00640 1 -- 172.21.15.133:0/3205338612 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b0034650 con 0x7fe49c0051c0 2024-03-07T06:52:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.968+0000 7fe4b6a00640 1 -- 172.21.15.133:0/3205338612 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0034a40 con 0x7fe49c0051c0 2024-03-07T06:52:01.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.969+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3205338612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe49c0051c0 msgr2=0x7fe49c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:01.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.969+0000 7fe4bdc00640 1 --2- 172.21.15.133:0/3205338612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe49c0051c0 0x7fe49c00b9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000a850 tx=0x7fe49c005830 comp rx=0 tx=0).stop 2024-03-07T06:52:01.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.969+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3205338612 shutdown_connections 2024-03-07T06:52:01.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.969+0000 7fe4bdc00640 1 --2- 172.21.15.133:0/3205338612 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe49c0051c0 0x7fe49c00b9c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.969+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3205338612 >> 172.21.15.133:0/3205338612 conn(0x7fe4b8071860 msgr2=0x7fe4b8071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:01.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.969+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3205338612 shutdown_connections 2024-03-07T06:52:01.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.969+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3205338612 wait complete. 2024-03-07T06:52:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.971+0000 7fe4bdc00640 1 Processor -- start 2024-03-07T06:52:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.971+0000 7fe4bdc00640 1 -- start start 2024-03-07T06:52:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.972+0000 7fe4bdc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe49c0051c0 0x7fe4b815e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.972+0000 7fe4bdc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4b80b4a00 0x7fe4b80a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.973+0000 7fe4bdc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4b80aa1f0 0x7fe4b8148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:01.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.973+0000 7fe4bdc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b8159c60 con 0x7fe4b80aa1f0 2024-03-07T06:52:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.973+0000 7fe4bdc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b815faf0 con 0x7fe49c0051c0 2024-03-07T06:52:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.973+0000 7fe4bdc00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b80cc6f0 con 0x7fe4b80b4a00 2024-03-07T06:52:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.973+0000 7fe4bd200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4b80aa1f0 0x7fe4b8148120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.973+0000 7fe4bd200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4b80aa1f0 0x7fe4b8148120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47312/0 (socket says 172.21.15.133:47312) 2024-03-07T06:52:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.973+0000 7fe4bd200640 1 -- 172.21.15.133:0/3001376055 learned_addr learned my addr 172.21.15.133:0/3001376055 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.973+0000 7fe4b7400640 1 --2- 172.21.15.133:0/3001376055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4b80b4a00 0x7fe4b80a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.973+0000 7fe4b7e00640 1 --2- 172.21.15.133:0/3001376055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe49c0051c0 0x7fe4b815e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.973+0000 7fe4bd200640 1 -- 172.21.15.133:0/3001376055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4b80b4a00 msgr2=0x7fe4b80a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.973+0000 7fe4bd200640 1 --2- 172.21.15.133:0/3001376055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4b80b4a00 0x7fe4b80a9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.973+0000 7fe4bd200640 1 -- 172.21.15.133:0/3001376055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe49c0051c0 msgr2=0x7fe4b815e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.973+0000 7fe4bd200640 1 --2- 172.21.15.133:0/3001376055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe49c0051c0 0x7fe4b815e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.973+0000 7fe4bd200640 1 -- 172.21.15.133:0/3001376055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b80b3960 con 0x7fe4b80aa1f0 2024-03-07T06:52:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.974+0000 7fe4bd200640 1 --2- 172.21.15.133:0/3001376055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4b80aa1f0 0x7fe4b8148120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000a850 tx=0x7fe4b0001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:01.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.974+0000 7fe4afe00640 1 -- 172.21.15.133:0/3001376055 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0045070 con 0x7fe4b80aa1f0 2024-03-07T06:52:01.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.974+0000 7fe4afe00640 1 -- 172.21.15.133:0/3001376055 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b003a040 con 0x7fe4b80aa1f0 2024-03-07T06:52:01.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.974+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3001376055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b80b3c40 con 0x7fe4b80aa1f0 2024-03-07T06:52:01.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.974+0000 7fe4afe00640 1 -- 172.21.15.133:0/3001376055 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00019f0 con 0x7fe4b80aa1f0 2024-03-07T06:52:01.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.974+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3001376055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b80b4200 con 0x7fe4b80aa1f0 2024-03-07T06:52:01.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.975+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3001376055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b8067c80 con 0x7fe4b80aa1f0 2024-03-07T06:52:01.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.976+0000 7fe4afe00640 1 -- 172.21.15.133:0/3001376055 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe4b0048070 con 0x7fe4b80aa1f0 2024-03-07T06:52:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.977+0000 7fe4afe00640 1 --2- 172.21.15.133:0/3001376055 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe49806b4d0 0x7fe49806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.977+0000 7fe4afe00640 1 -- 172.21.15.133:0/3001376055 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe4b00abd60 con 0x7fe4b80aa1f0 2024-03-07T06:52:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.978+0000 7fe4b7e00640 1 --2- 172.21.15.133:0/3001376055 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe49806b4d0 0x7fe49806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:01.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.979+0000 7fe4b7e00640 1 --2- 172.21.15.133:0/3001376055 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe49806b4d0 0x7fe49806d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009a80 tx=0x7fe4a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:01.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:01.983+0000 7fe4afe00640 1 -- 172.21.15.133:0/3001376055 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe4b007a480 con 0x7fe4b80aa1f0 2024-03-07T06:52:02.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.179+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3001376055 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4b8062c90 con 0x7fe4b80aa1f0 2024-03-07T06:52:02.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.180+0000 7fe4afe00640 1 -- 172.21.15.133:0/3001376055 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe4b007e520 con 0x7fe4b80aa1f0 2024-03-07T06:52:02.180 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:52:02.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.183+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3001376055 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe49806b4d0 msgr2=0x7fe49806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:02.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.183+0000 7fe4bdc00640 1 --2- 172.21.15.133:0/3001376055 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe49806b4d0 0x7fe49806d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009a80 tx=0x7fe4a8008040 comp rx=0 tx=0).stop 2024-03-07T06:52:02.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.183+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3001376055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4b80aa1f0 msgr2=0x7fe4b8148120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:02.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.183+0000 7fe4bdc00640 1 --2- 172.21.15.133:0/3001376055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4b80aa1f0 0x7fe4b8148120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000a850 tx=0x7fe4b0001dc0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.183+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3001376055 shutdown_connections 2024-03-07T06:52:02.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.183+0000 7fe4bdc00640 1 --2- 172.21.15.133:0/3001376055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4b80aa1f0 0x7fe4b8148120 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.183+0000 7fe4bdc00640 1 --2- 172.21.15.133:0/3001376055 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe4b80b4a00 0x7fe4b80a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.183+0000 7fe4bdc00640 1 --2- 172.21.15.133:0/3001376055 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe49806b4d0 0x7fe49806d990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.183+0000 7fe4bdc00640 1 --2- 172.21.15.133:0/3001376055 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe49c0051c0 0x7fe4b815e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.183+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3001376055 >> 172.21.15.133:0/3001376055 conn(0x7fe4b8071860 msgr2=0x7fe4b8158a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:02.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.183+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3001376055 shutdown_connections 2024-03-07T06:52:02.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.183+0000 7fe4bdc00640 1 -- 172.21.15.133:0/3001376055 wait complete. 2024-03-07T06:52:02.201 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:52:02.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.412+0000 7fc6f7200640 1 Processor -- start 2024-03-07T06:52:02.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.412+0000 7fc6f7200640 1 -- start start 2024-03-07T06:52:02.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.413+0000 7fc6f7200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f0071050 0x7fc6f00d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:02.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.413+0000 7fc6f7200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f0071610 con 0x7fc6f0070bc0 2024-03-07T06:52:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.413+0000 7fc6f7200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f00718f0 con 0x7fc6f0162110 2024-03-07T06:52:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.413+0000 7fc6f7200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0071bd0 con 0x7fc6f0071050 2024-03-07T06:52:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.414+0000 7fc6ef400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc6f0070bc0 0x7fc6f00d26f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47756/0 (socket says 172.21.15.133:47756) 2024-03-07T06:52:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.414+0000 7fc6ef400640 1 -- 172.21.15.133:0/3377706682 learned_addr learned my addr 172.21.15.133:0/3377706682 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.414+0000 7fc6f5200640 1 --2- 172.21.15.133:0/3377706682 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f0071050 0x7fc6f00d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.414+0000 7fc6eea00640 1 -- 172.21.15.133:0/3377706682 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195388956 0 0) 0x7fc6f0071610 con 0x7fc6f0070bc0 2024-03-07T06:52:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.414+0000 7fc6eea00640 1 -- 172.21.15.133:0/3377706682 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6d4003520 con 0x7fc6f0070bc0 2024-03-07T06:52:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.414+0000 7fc6eea00640 1 -- 172.21.15.133:0/3377706682 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 963584992 0 0) 0x7fc6d4003520 con 0x7fc6f0070bc0 2024-03-07T06:52:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6eea00640 1 -- 172.21.15.133:0/3377706682 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f0071050 msgr2=0x7fc6f00d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6eea00640 1 --2- 172.21.15.133:0/3377706682 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f0071050 0x7fc6f00d2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6eea00640 1 -- 172.21.15.133:0/3377706682 >> v1:172.21.15.161:6789/0 conn(0x7fc6f0162110 legacy=0x7fc6f01624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6eea00640 1 -- 172.21.15.133:0/3377706682 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f0071eb0 con 0x7fc6f0070bc0 2024-03-07T06:52:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6eea00640 1 -- 172.21.15.133:0/3377706682 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fc6d8001670 con 0x7fc6f0070bc0 2024-03-07T06:52:02.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6eea00640 1 -- 172.21.15.133:0/3377706682 >> v1:172.21.15.133:6789/0 conn(0x7fc6f0070bc0 legacy=0x7fc6f00d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:02.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6eea00640 1 --2- 172.21.15.133:0/3377706682 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f0071050 0x7fc6d40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:02.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6eea00640 1 --2- 172.21.15.133:0/3377706682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d4003fb0 0x7fc6d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:02.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6eea00640 1 --2- 172.21.15.133:0/3377706682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6d4004d40 0x7fc6d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:02.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6eea00640 1 -- 172.21.15.133:0/3377706682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0071bd0 con 0x7fc6d4003fb0 2024-03-07T06:52:02.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6eea00640 1 -- 172.21.15.133:0/3377706682 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d400bbb0 con 0x7fc6d4004d40 2024-03-07T06:52:02.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6eea00640 1 -- 172.21.15.133:0/3377706682 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d400bdb0 con 0x7fc6f0071050 2024-03-07T06:52:02.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6ef400640 1 --2- 172.21.15.133:0/3377706682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d4003fb0 0x7fc6d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6efe00640 1 --2- 172.21.15.133:0/3377706682 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f0071050 0x7fc6d40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6f5200640 1 --2- 172.21.15.133:0/3377706682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6d4004d40 0x7fc6d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6ef400640 1 -- 172.21.15.133:0/3377706682 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f0071050 msgr2=0x7fc6d40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6ef400640 1 --2- 172.21.15.133:0/3377706682 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f0071050 0x7fc6d40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6ef400640 1 -- 172.21.15.133:0/3377706682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6d4004d40 msgr2=0x7fc6d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.415+0000 7fc6ef400640 1 --2- 172.21.15.133:0/3377706682 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6d4004d40 0x7fc6d40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.416+0000 7fc6ef400640 1 -- 172.21.15.133:0/3377706682 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d4003520 con 0x7fc6d4003fb0 2024-03-07T06:52:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.417+0000 7fc6ef400640 1 --2- 172.21.15.133:0/3377706682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d4003fb0 0x7fc6d4004800 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c190 tx=0x7fc6d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb952e436ff105 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.417+0000 7fc6eea00640 1 -- 172.21.15.133:0/3377706682 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8007c60 con 0x7fc6d4003fb0 2024-03-07T06:52:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.417+0000 7fc6eea00640 1 -- 172.21.15.133:0/3377706682 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d8030070 con 0x7fc6d4003fb0 2024-03-07T06:52:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.417+0000 7fc6eea00640 1 -- 172.21.15.133:0/3377706682 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8005d90 con 0x7fc6d4003fb0 2024-03-07T06:52:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.418+0000 7fc6f7200640 1 -- 172.21.15.133:0/3377706682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d4003fb0 msgr2=0x7fc6d4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.418+0000 7fc6f7200640 1 --2- 172.21.15.133:0/3377706682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d4003fb0 0x7fc6d4004800 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c190 tx=0x7fc6d4005830 comp rx=0 tx=0).stop 2024-03-07T06:52:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.418+0000 7fc6f5200640 1 -- 172.21.15.133:0/3377706682 reap_dead start 2024-03-07T06:52:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.418+0000 7fc6f7200640 1 -- 172.21.15.133:0/3377706682 shutdown_connections 2024-03-07T06:52:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.418+0000 7fc6f7200640 1 -- 172.21.15.133:0/3377706682 >> 172.21.15.133:0/3377706682 conn(0x7fc6f00d0e10 msgr2=0x7fc6f00d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.418+0000 7fc6f7200640 1 -- 172.21.15.133:0/3377706682 shutdown_connections 2024-03-07T06:52:02.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.418+0000 7fc6f7200640 1 -- 172.21.15.133:0/3377706682 wait complete. 2024-03-07T06:52:02.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.420+0000 7fc6f7200640 1 Processor -- start 2024-03-07T06:52:02.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.420+0000 7fc6f7200640 1 -- start start 2024-03-07T06:52:02.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6f7200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d4003fb0 0x7fc6f015b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6f7200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6f00b5e20 0x7fc6f01e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6f7200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f00b39c0 0x7fc6f00b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6f7200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0162f80 con 0x7fc6d4003fb0 2024-03-07T06:52:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6f7200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0162e00 con 0x7fc6f00b5e20 2024-03-07T06:52:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6f7200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f015f8e0 con 0x7fc6f00b39c0 2024-03-07T06:52:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6efe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d4003fb0 0x7fc6f015b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6efe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d4003fb0 0x7fc6f015b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47336/0 (socket says 172.21.15.133:47336) 2024-03-07T06:52:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6efe00640 1 -- 172.21.15.133:0/1345818162 learned_addr learned my addr 172.21.15.133:0/1345818162 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6ef400640 1 --2- 172.21.15.133:0/1345818162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6f00b5e20 0x7fc6f01e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6f5200640 1 --2- 172.21.15.133:0/1345818162 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f00b39c0 0x7fc6f00b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6efe00640 1 -- 172.21.15.133:0/1345818162 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f00b39c0 msgr2=0x7fc6f00b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6efe00640 1 --2- 172.21.15.133:0/1345818162 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f00b39c0 0x7fc6f00b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6efe00640 1 -- 172.21.15.133:0/1345818162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6f00b5e20 msgr2=0x7fc6f01e8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6efe00640 1 --2- 172.21.15.133:0/1345818162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6f00b5e20 0x7fc6f01e8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.421+0000 7fc6efe00640 1 -- 172.21.15.133:0/1345818162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f01e9770 con 0x7fc6d4003fb0 2024-03-07T06:52:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.422+0000 7fc6efe00640 1 --2- 172.21.15.133:0/1345818162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d4003fb0 0x7fc6f015b4f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000e4f0 tx=0x7fc6e000e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.422+0000 7fc6e7e00640 1 -- 172.21.15.133:0/1345818162 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00052e0 con 0x7fc6d4003fb0 2024-03-07T06:52:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.422+0000 7fc6e7e00640 1 -- 172.21.15.133:0/1345818162 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e0018070 con 0x7fc6d4003fb0 2024-03-07T06:52:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.422+0000 7fc6e7e00640 1 -- 172.21.15.133:0/1345818162 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0003830 con 0x7fc6d4003fb0 2024-03-07T06:52:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.422+0000 7fc6f7200640 1 -- 172.21.15.133:0/1345818162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f01e9a50 con 0x7fc6d4003fb0 2024-03-07T06:52:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.422+0000 7fc6f7200640 1 -- 172.21.15.133:0/1345818162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f01ea010 con 0x7fc6d4003fb0 2024-03-07T06:52:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.423+0000 7fc6f7200640 1 -- 172.21.15.133:0/1345818162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f0062c90 con 0x7fc6d4003fb0 2024-03-07T06:52:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.424+0000 7fc6e7e00640 1 -- 172.21.15.133:0/1345818162 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc6e00039d0 con 0x7fc6d4003fb0 2024-03-07T06:52:02.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.425+0000 7fc6e7e00640 1 --2- 172.21.15.133:0/1345818162 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc6cc06b4d0 0x7fc6cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:02.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.425+0000 7fc6e7e00640 1 -- 172.21.15.133:0/1345818162 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fc6e0089fc0 con 0x7fc6d4003fb0 2024-03-07T06:52:02.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.426+0000 7fc6ef400640 1 --2- 172.21.15.133:0/1345818162 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc6cc06b4d0 0x7fc6cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:02.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.427+0000 7fc6ef400640 1 --2- 172.21.15.133:0/1345818162 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc6cc06b4d0 0x7fc6cc06d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c160 tx=0x7fc6d8009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:02.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.429+0000 7fc6e7e00640 1 -- 172.21.15.133:0/1345818162 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc6e0058de0 con 0x7fc6d4003fb0 2024-03-07T06:52:02.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.623+0000 7fc6f7200640 1 -- 172.21.15.133:0/1345818162 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6f00cab80 con 0x7fc6d4003fb0 2024-03-07T06:52:02.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.624+0000 7fc6e7e00640 1 -- 172.21.15.133:0/1345818162 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc6e00586e0 con 0x7fc6d4003fb0 2024-03-07T06:52:02.624 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:52:02.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.627+0000 7fc6f7200640 1 -- 172.21.15.133:0/1345818162 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc6cc06b4d0 msgr2=0x7fc6cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:02.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.627+0000 7fc6f7200640 1 --2- 172.21.15.133:0/1345818162 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc6cc06b4d0 0x7fc6cc06d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c160 tx=0x7fc6d8009030 comp rx=0 tx=0).stop 2024-03-07T06:52:02.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.627+0000 7fc6f7200640 1 -- 172.21.15.133:0/1345818162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d4003fb0 msgr2=0x7fc6f015b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:02.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.627+0000 7fc6f7200640 1 --2- 172.21.15.133:0/1345818162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d4003fb0 0x7fc6f015b4f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc6e000e4f0 tx=0x7fc6e000e9b0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.627+0000 7fc6f7200640 1 -- 172.21.15.133:0/1345818162 shutdown_connections 2024-03-07T06:52:02.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.627+0000 7fc6f7200640 1 --2- 172.21.15.133:0/1345818162 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6f00b39c0 0x7fc6f00b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.627+0000 7fc6f7200640 1 --2- 172.21.15.133:0/1345818162 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc6cc06b4d0 0x7fc6cc06d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.627+0000 7fc6f7200640 1 --2- 172.21.15.133:0/1345818162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6f00b5e20 0x7fc6f01e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.627+0000 7fc6f7200640 1 --2- 172.21.15.133:0/1345818162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d4003fb0 0x7fc6f015b4f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.627+0000 7fc6f7200640 1 -- 172.21.15.133:0/1345818162 >> 172.21.15.133:0/1345818162 conn(0x7fc6f00d0e10 msgr2=0x7fc6f00d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:02.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.627+0000 7fc6f7200640 1 -- 172.21.15.133:0/1345818162 shutdown_connections 2024-03-07T06:52:02.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.627+0000 7fc6f7200640 1 -- 172.21.15.133:0/1345818162 wait complete. 2024-03-07T06:52:02.647 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:52:02.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.856+0000 7f772fa00640 1 Processor -- start 2024-03-07T06:52:02.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.856+0000 7f772fa00640 1 -- start start 2024-03-07T06:52:02.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.857+0000 7f772fa00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7728145670 0x7f77280cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:02.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.857+0000 7f772fa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f772815cf90 0x7f77280cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:02.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.858+0000 7f772fa00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f772806e150 con 0x7f77280cda20 2024-03-07T06:52:02.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.858+0000 7f772fa00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77280cde70 con 0x7f7728145670 2024-03-07T06:52:02.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.858+0000 7f772fa00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772806ed60 con 0x7f772815cf90 2024-03-07T06:52:02.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.858+0000 7f772d000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7728145670 0x7f77280cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:02.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.858+0000 7f772d000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7728145670 0x7f77280cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47342/0 (socket says 172.21.15.133:47342) 2024-03-07T06:52:02.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.858+0000 7f772da00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f77280cda20 0x7f772815fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47766/0 (socket says 172.21.15.133:47766) 2024-03-07T06:52:02.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.858+0000 7f772d000640 1 -- 172.21.15.133:0/3528954949 learned_addr learned my addr 172.21.15.133:0/3528954949 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:02.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.858+0000 7f7727e00640 1 --2- 172.21.15.133:0/3528954949 >> v2:172.21.15.161:3300/0 conn(0x7f772815cf90 0x7f77280cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:02.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.859+0000 7f772d000640 1 -- 172.21.15.133:0/3528954949 >> v2:172.21.15.161:3300/0 conn(0x7f772815cf90 msgr2=0x7f77280cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:02.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.859+0000 7f772d000640 1 --2- 172.21.15.133:0/3528954949 >> v2:172.21.15.161:3300/0 conn(0x7f772815cf90 0x7f77280cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.859+0000 7f772d000640 1 -- 172.21.15.133:0/3528954949 >> v1:172.21.15.133:6789/0 conn(0x7f77280cda20 legacy=0x7f772815fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:02.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.859+0000 7f772d000640 1 -- 172.21.15.133:0/3528954949 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772806e430 con 0x7f7728145670 2024-03-07T06:52:02.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.860+0000 7f772d000640 1 --2- 172.21.15.133:0/3528954949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7728145670 0x7f77280cf610 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7714009870 tx=0x7f771402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00c2b4d9dafeeac server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:02.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.860+0000 7f7727400640 1 -- 172.21.15.133:0/3528954949 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714035020 con 0x7f7728145670 2024-03-07T06:52:02.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.860+0000 7f7727400640 1 -- 172.21.15.133:0/3528954949 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7714031880 con 0x7f7728145670 2024-03-07T06:52:02.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.860+0000 7f7727400640 1 -- 172.21.15.133:0/3528954949 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7714031c50 con 0x7f7728145670 2024-03-07T06:52:02.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.861+0000 7f772fa00640 1 -- 172.21.15.133:0/3528954949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7728145670 msgr2=0x7f77280cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:02.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.861+0000 7f772fa00640 1 --2- 172.21.15.133:0/3528954949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7728145670 0x7f77280cf610 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7714009870 tx=0x7f771402b920 comp rx=0 tx=0).stop 2024-03-07T06:52:02.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.861+0000 7f772fa00640 1 -- 172.21.15.133:0/3528954949 shutdown_connections 2024-03-07T06:52:02.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.861+0000 7f772fa00640 1 --2- 172.21.15.133:0/3528954949 >> v2:172.21.15.161:3300/0 conn(0x7f772815cf90 0x7f77280cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.861+0000 7f772fa00640 1 --2- 172.21.15.133:0/3528954949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7728145670 0x7f77280cf610 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.861+0000 7f772fa00640 1 -- 172.21.15.133:0/3528954949 >> 172.21.15.133:0/3528954949 conn(0x7f77280b1d30 msgr2=0x7f77280b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:02.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.861+0000 7f772fa00640 1 -- 172.21.15.133:0/3528954949 shutdown_connections 2024-03-07T06:52:02.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.861+0000 7f772fa00640 1 -- 172.21.15.133:0/3528954949 wait complete. 2024-03-07T06:52:02.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.863+0000 7f772fa00640 1 Processor -- start 2024-03-07T06:52:02.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.863+0000 7f772fa00640 1 -- start start 2024-03-07T06:52:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.864+0000 7f772fa00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77280cda20 0x7f7728137260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.864+0000 7f772fa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7728145670 0x7f77280afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.864+0000 7f772fa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772815cf90 0x7f77280cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.864+0000 7f772fa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772815d420 con 0x7f772815cf90 2024-03-07T06:52:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.864+0000 7f772fa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772806e9f0 con 0x7f7728145670 2024-03-07T06:52:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.864+0000 7f772fa00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7728145a50 con 0x7f77280cda20 2024-03-07T06:52:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.864+0000 7f772da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772815cf90 0x7f77280cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.864+0000 7f772da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772815cf90 0x7f77280cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47348/0 (socket says 172.21.15.133:47348) 2024-03-07T06:52:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.864+0000 7f772da00640 1 -- 172.21.15.133:0/4116344380 learned_addr learned my addr 172.21.15.133:0/4116344380 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.864+0000 7f772d000640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77280cda20 0x7f7728137260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.864+0000 7f7727e00640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7728145670 0x7f77280afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.865+0000 7f772da00640 1 -- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77280cda20 msgr2=0x7f7728137260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.865+0000 7f772da00640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77280cda20 0x7f7728137260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.865+0000 7f772da00640 1 -- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7728145670 msgr2=0x7f77280afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.865+0000 7f772da00640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7728145670 0x7f77280afa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.865+0000 7f772da00640 1 -- 172.21.15.133:0/4116344380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7728137960 con 0x7f772815cf90 2024-03-07T06:52:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.865+0000 7f7727e00640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7728145670 0x7f77280afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:52:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.865+0000 7f772da00640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772815cf90 0x7f77280cc740 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f772000c390 tx=0x7f772000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.865+0000 7f7724c00640 1 -- 172.21.15.133:0/4116344380 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77200074e0 con 0x7f772815cf90 2024-03-07T06:52:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.865+0000 7f7724c00640 1 -- 172.21.15.133:0/4116344380 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7720007680 con 0x7f772815cf90 2024-03-07T06:52:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.865+0000 7f772fa00640 1 -- 172.21.15.133:0/4116344380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7728137c40 con 0x7f772815cf90 2024-03-07T06:52:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.865+0000 7f7724c00640 1 -- 172.21.15.133:0/4116344380 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77200049e0 con 0x7f772815cf90 2024-03-07T06:52:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.865+0000 7f772fa00640 1 -- 172.21.15.133:0/4116344380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77280affb0 con 0x7f772815cf90 2024-03-07T06:52:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.866+0000 7f772fa00640 1 -- 172.21.15.133:0/4116344380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7728138160 con 0x7f772815cf90 2024-03-07T06:52:02.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.867+0000 7f7724c00640 1 -- 172.21.15.133:0/4116344380 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f772001a020 con 0x7f772815cf90 2024-03-07T06:52:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.868+0000 7f7724c00640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f770806b4d0 0x7f770806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.868+0000 7f7724c00640 1 -- 172.21.15.133:0/4116344380 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f7720089a40 con 0x7f772815cf90 2024-03-07T06:52:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.869+0000 7f772d000640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f770806b4d0 0x7f770806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.870+0000 7f772d000640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f770806b4d0 0x7f770806d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7714036000 tx=0x7f771403c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:02.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:02.873+0000 7f7724c00640 1 -- 172.21.15.133:0/4116344380 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7728138160 con 0x7f772815cf90 2024-03-07T06:52:03.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.067+0000 7f772fa00640 1 -- 172.21.15.133:0/4116344380 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7728062c90 con 0x7f772815cf90 2024-03-07T06:52:03.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.068+0000 7f7724c00640 1 -- 172.21.15.133:0/4116344380 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7720010300 con 0x7f772815cf90 2024-03-07T06:52:03.068 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:52:03.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.070+0000 7f772fa00640 1 -- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f770806b4d0 msgr2=0x7f770806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:03.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.070+0000 7f772fa00640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f770806b4d0 0x7f770806d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7714036000 tx=0x7f771403c040 comp rx=0 tx=0).stop 2024-03-07T06:52:03.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.070+0000 7f772fa00640 1 -- 172.21.15.133:0/4116344380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772815cf90 msgr2=0x7f77280cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:03.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.070+0000 7f772fa00640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772815cf90 0x7f77280cc740 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f772000c390 tx=0x7f772000c850 comp rx=0 tx=0).stop 2024-03-07T06:52:03.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.070+0000 7f772fa00640 1 -- 172.21.15.133:0/4116344380 shutdown_connections 2024-03-07T06:52:03.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.070+0000 7f772fa00640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f772815cf90 0x7f77280cc740 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:03.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.070+0000 7f772fa00640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f770806b4d0 0x7f770806d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:03.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.071+0000 7f772fa00640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7728145670 0x7f77280afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:03.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.071+0000 7f772fa00640 1 --2- 172.21.15.133:0/4116344380 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77280cda20 0x7f7728137260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:03.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.071+0000 7f772fa00640 1 -- 172.21.15.133:0/4116344380 >> 172.21.15.133:0/4116344380 conn(0x7f77280b1d30 msgr2=0x7f772815e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:03.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.071+0000 7f772fa00640 1 -- 172.21.15.133:0/4116344380 shutdown_connections 2024-03-07T06:52:03.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.071+0000 7f772fa00640 1 -- 172.21.15.133:0/4116344380 wait complete. 2024-03-07T06:52:03.090 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-07T06:52:03.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.294+0000 7fcb7b200640 1 Processor -- start 2024-03-07T06:52:03.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.294+0000 7fcb7b200640 1 -- start start 2024-03-07T06:52:03.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.296+0000 7fcb7b200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb7c140cb0 0x7fcb7c162290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:03.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.296+0000 7fcb7b200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcb7c0b70d0 0x7fcb7c0b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:03.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.296+0000 7fcb7b200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7c143810 con 0x7fcb7c0b9300 2024-03-07T06:52:03.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.296+0000 7fcb7b200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb7c0b7a70 con 0x7fcb7c0b70d0 2024-03-07T06:52:03.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.296+0000 7fcb7b200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c1435c0 con 0x7fcb7c140cb0 2024-03-07T06:52:03.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.297+0000 7fcb73e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcb7c0b70d0 0x7fcb7c0b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:03.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.297+0000 7fcb73e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fcb7c0b70d0 0x7fcb7c0b7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47356/0 (socket says 172.21.15.133:47356) 2024-03-07T06:52:03.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.297+0000 7fcb73e00640 1 -- 172.21.15.133:0/1683194555 learned_addr learned my addr 172.21.15.133:0/1683194555 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:03.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.297+0000 7fcb79e00640 1 --2- 172.21.15.133:0/1683194555 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb7c140cb0 0x7fcb7c162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:03.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.297+0000 7fcb73e00640 1 -- 172.21.15.133:0/1683194555 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb7c140cb0 msgr2=0x7fcb7c162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:03.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.297+0000 7fcb73e00640 1 --2- 172.21.15.133:0/1683194555 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb7c140cb0 0x7fcb7c162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:03.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.297+0000 7fcb73e00640 1 -- 172.21.15.133:0/1683194555 >> v1:172.21.15.133:6789/0 conn(0x7fcb7c0b9300 legacy=0x7fcb7c0b9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:03.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.297+0000 7fcb73e00640 1 -- 172.21.15.133:0/1683194555 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c143af0 con 0x7fcb7c0b70d0 2024-03-07T06:52:03.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.298+0000 7fcb73e00640 1 --2- 172.21.15.133:0/1683194555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb7c0b70d0 0x7fcb7c0b7530 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009870 tx=0x7fcb6402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb05f7b5fd7f7fd2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:03.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.299+0000 7fcb79400640 1 -- 172.21.15.133:0/1683194555 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64034070 con 0x7fcb7c0b70d0 2024-03-07T06:52:03.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.299+0000 7fcb79400640 1 -- 172.21.15.133:0/1683194555 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb64030840 con 0x7fcb7c0b70d0 2024-03-07T06:52:03.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.299+0000 7fcb79400640 1 -- 172.21.15.133:0/1683194555 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64030c10 con 0x7fcb7c0b70d0 2024-03-07T06:52:03.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.299+0000 7fcb7b200640 1 -- 172.21.15.133:0/1683194555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb7c0b70d0 msgr2=0x7fcb7c0b7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:03.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.299+0000 7fcb7b200640 1 --2- 172.21.15.133:0/1683194555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb7c0b70d0 0x7fcb7c0b7530 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009870 tx=0x7fcb6402b980 comp rx=0 tx=0).stop 2024-03-07T06:52:03.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.299+0000 7fcb7b200640 1 -- 172.21.15.133:0/1683194555 shutdown_connections 2024-03-07T06:52:03.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.299+0000 7fcb7b200640 1 --2- 172.21.15.133:0/1683194555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb7c0b70d0 0x7fcb7c0b7530 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:03.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.299+0000 7fcb7b200640 1 --2- 172.21.15.133:0/1683194555 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb7c140cb0 0x7fcb7c162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:03.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.299+0000 7fcb7b200640 1 -- 172.21.15.133:0/1683194555 >> 172.21.15.133:0/1683194555 conn(0x7fcb7c0674e0 msgr2=0x7fcb7c0d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:03.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.300+0000 7fcb7b200640 1 -- 172.21.15.133:0/1683194555 shutdown_connections 2024-03-07T06:52:03.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.300+0000 7fcb7b200640 1 -- 172.21.15.133:0/1683194555 wait complete. 2024-03-07T06:52:03.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.302+0000 7fcb7b200640 1 Processor -- start 2024-03-07T06:52:03.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.302+0000 7fcb7b200640 1 -- start start 2024-03-07T06:52:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.303+0000 7fcb7b200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb7c0b70d0 0x7fcb7c156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.303+0000 7fcb7b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb7c0b9300 0x7fcb7c157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.303+0000 7fcb7b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb7c140cb0 0x7fcb7c154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.303+0000 7fcb7b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c1440b0 con 0x7fcb7c0b9300 2024-03-07T06:52:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.303+0000 7fcb7b200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c144420 con 0x7fcb7c140cb0 2024-03-07T06:52:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.303+0000 7fcb7b200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c1627d0 con 0x7fcb7c0b70d0 2024-03-07T06:52:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.304+0000 7fcb73e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb7c0b9300 0x7fcb7c157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.304+0000 7fcb73e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb7c0b9300 0x7fcb7c157c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47364/0 (socket says 172.21.15.133:47364) 2024-03-07T06:52:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.304+0000 7fcb73e00640 1 -- 172.21.15.133:0/3679972537 learned_addr learned my addr 172.21.15.133:0/3679972537 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:03.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.304+0000 7fcb79e00640 1 --2- 172.21.15.133:0/3679972537 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb7c0b70d0 0x7fcb7c156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:03.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.304+0000 7fcb7a800640 1 --2- 172.21.15.133:0/3679972537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb7c140cb0 0x7fcb7c154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:03.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.304+0000 7fcb73e00640 1 -- 172.21.15.133:0/3679972537 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb7c0b70d0 msgr2=0x7fcb7c156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:03.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.304+0000 7fcb73e00640 1 --2- 172.21.15.133:0/3679972537 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb7c0b70d0 0x7fcb7c156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:03.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.304+0000 7fcb73e00640 1 -- 172.21.15.133:0/3679972537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb7c140cb0 msgr2=0x7fcb7c154000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:03.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.304+0000 7fcb73e00640 1 --2- 172.21.15.133:0/3679972537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb7c140cb0 0x7fcb7c154000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:03.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.304+0000 7fcb73e00640 1 -- 172.21.15.133:0/3679972537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c156bf0 con 0x7fcb7c0b9300 2024-03-07T06:52:03.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.304+0000 7fcb73e00640 1 --2- 172.21.15.133:0/3679972537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb7c0b9300 0x7fcb7c157c90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcb64000c00 tx=0x7fcb64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:03.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.305+0000 7fcb72000640 1 -- 172.21.15.133:0/3679972537 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64048070 con 0x7fcb7c0b9300 2024-03-07T06:52:03.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.305+0000 7fcb72000640 1 -- 172.21.15.133:0/3679972537 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb64030640 con 0x7fcb7c0b9300 2024-03-07T06:52:03.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.305+0000 7fcb7b200640 1 -- 172.21.15.133:0/3679972537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7c156ed0 con 0x7fcb7c0b9300 2024-03-07T06:52:03.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.305+0000 7fcb72000640 1 -- 172.21.15.133:0/3679972537 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64034040 con 0x7fcb7c0b9300 2024-03-07T06:52:03.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.305+0000 7fcb7b200640 1 -- 172.21.15.133:0/3679972537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7c157490 con 0x7fcb7c0b9300 2024-03-07T06:52:03.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.306+0000 7fcb7b200640 1 -- 172.21.15.133:0/3679972537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7c062c90 con 0x7fcb7c0b9300 2024-03-07T06:52:03.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.307+0000 7fcb72000640 1 -- 172.21.15.133:0/3679972537 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcb6404b070 con 0x7fcb7c0b9300 2024-03-07T06:52:03.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.308+0000 7fcb72000640 1 --2- 172.21.15.133:0/3679972537 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fcb5806b4d0 0x7fcb5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:03.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.308+0000 7fcb72000640 1 -- 172.21.15.133:0/3679972537 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fcb640ac220 con 0x7fcb7c0b9300 2024-03-07T06:52:03.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.309+0000 7fcb79e00640 1 --2- 172.21.15.133:0/3679972537 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fcb5806b4d0 0x7fcb5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.310+0000 7fcb79e00640 1 --2- 172.21.15.133:0/3679972537 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fcb5806b4d0 0x7fcb5806d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009a30 tx=0x7fcb6c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:03.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.314+0000 7fcb72000640 1 -- 172.21.15.133:0/3679972537 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcb6407aef0 con 0x7fcb7c0b9300 2024-03-07T06:52:03.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.525+0000 7fcb7b200640 1 -- 172.21.15.133:0/3679972537 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcb7c13d300 con 0x7fcb7c0b9300 2024-03-07T06:52:03.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.526+0000 7fcb72000640 1 -- 172.21.15.133:0/3679972537 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcb64030810 con 0x7fcb7c0b9300 2024-03-07T06:52:03.525 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-03-07T06:52:03.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.528+0000 7fcb7b200640 1 -- 172.21.15.133:0/3679972537 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fcb5806b4d0 msgr2=0x7fcb5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:03.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.528+0000 7fcb7b200640 1 --2- 172.21.15.133:0/3679972537 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fcb5806b4d0 0x7fcb5806d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c009a30 tx=0x7fcb6c008040 comp rx=0 tx=0).stop 2024-03-07T06:52:03.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.528+0000 7fcb7b200640 1 -- 172.21.15.133:0/3679972537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb7c0b9300 msgr2=0x7fcb7c157c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:03.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.528+0000 7fcb7b200640 1 --2- 172.21.15.133:0/3679972537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb7c0b9300 0x7fcb7c157c90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcb64000c00 tx=0x7fcb64004730 comp rx=0 tx=0).stop 2024-03-07T06:52:03.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.529+0000 7fcb7b200640 1 -- 172.21.15.133:0/3679972537 shutdown_connections 2024-03-07T06:52:03.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.529+0000 7fcb7b200640 1 --2- 172.21.15.133:0/3679972537 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fcb5806b4d0 0x7fcb5806d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:03.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.529+0000 7fcb7b200640 1 --2- 172.21.15.133:0/3679972537 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb7c140cb0 0x7fcb7c154000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:03.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.529+0000 7fcb7b200640 1 --2- 172.21.15.133:0/3679972537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb7c0b9300 0x7fcb7c157c90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:03.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.529+0000 7fcb7b200640 1 --2- 172.21.15.133:0/3679972537 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb7c0b70d0 0x7fcb7c156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:03.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.529+0000 7fcb7b200640 1 -- 172.21.15.133:0/3679972537 >> 172.21.15.133:0/3679972537 conn(0x7fcb7c0674e0 msgr2=0x7fcb7c0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:03.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.529+0000 7fcb7b200640 1 -- 172.21.15.133:0/3679972537 shutdown_connections 2024-03-07T06:52:03.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:03.529+0000 7fcb7b200640 1 -- 172.21.15.133:0/3679972537 wait complete. 2024-03-07T06:52:03.548 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-07T06:52:03.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:03.548 DEBUG:teuthology.orchestra.run.smithi133:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-07T06:52:03.649 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:52:03.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:03.650 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-07T06:52:03.714 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:52:03.714 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:52:03.771 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:52:03.771 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:52:03.820 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:52:03.820 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:52:03.915 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:52:04.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.131+0000 7f441de00640 1 Processor -- start 2024-03-07T06:52:04.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.131+0000 7f441de00640 1 -- start start 2024-03-07T06:52:04.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.133+0000 7f441de00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f441815fbb0 0x7f44180cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:04.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.133+0000 7f441de00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4418072950 0x7f44180cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:04.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.133+0000 7f441de00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44180731b0 con 0x7f441815ff90 2024-03-07T06:52:04.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.133+0000 7f441de00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4418072f60 con 0x7f441815fbb0 2024-03-07T06:52:04.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.133+0000 7f441de00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4418073490 con 0x7f4418072950 2024-03-07T06:52:04.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.133+0000 7f4416a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f441815fbb0 0x7f44180cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:04.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.133+0000 7f4416a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f441815fbb0 0x7f44180cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47374/0 (socket says 172.21.15.133:47374) 2024-03-07T06:52:04.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.133+0000 7f4417400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f441815ff90 0x7f44180acd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47790/0 (socket says 172.21.15.133:47790) 2024-03-07T06:52:04.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.133+0000 7f4416a00640 1 -- 172.21.15.133:0/3069732737 learned_addr learned my addr 172.21.15.133:0/3069732737 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:04.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.133+0000 7f4416000640 1 --2- 172.21.15.133:0/3069732737 >> v2:172.21.15.161:3300/0 conn(0x7f4418072950 0x7f44180cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:04.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.134+0000 7f4416a00640 1 -- 172.21.15.133:0/3069732737 >> v2:172.21.15.161:3300/0 conn(0x7f4418072950 msgr2=0x7f44180cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:04.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.134+0000 7f4416a00640 1 --2- 172.21.15.133:0/3069732737 >> v2:172.21.15.161:3300/0 conn(0x7f4418072950 0x7f44180cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.134+0000 7f4416a00640 1 -- 172.21.15.133:0/3069732737 >> v1:172.21.15.133:6789/0 conn(0x7f441815ff90 legacy=0x7f44180acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:04.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.134+0000 7f4416a00640 1 -- 172.21.15.133:0/3069732737 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4418072150 con 0x7f441815fbb0 2024-03-07T06:52:04.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.135+0000 7f4416a00640 1 --2- 172.21.15.133:0/3069732737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f441815fbb0 0x7f44180cf7d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f44180cdd10 tx=0x7f440802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6293cb8c9fe36979 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:04.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.135+0000 7f4415600640 1 -- 172.21.15.133:0/3069732737 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408035020 con 0x7f441815fbb0 2024-03-07T06:52:04.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.135+0000 7f4415600640 1 -- 172.21.15.133:0/3069732737 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4408031870 con 0x7f441815fbb0 2024-03-07T06:52:04.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.135+0000 7f4415600640 1 -- 172.21.15.133:0/3069732737 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408031c40 con 0x7f441815fbb0 2024-03-07T06:52:04.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.136+0000 7f441de00640 1 -- 172.21.15.133:0/3069732737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f441815fbb0 msgr2=0x7f44180cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:04.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.136+0000 7f441de00640 1 --2- 172.21.15.133:0/3069732737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f441815fbb0 0x7f44180cf7d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f44180cdd10 tx=0x7f440802b910 comp rx=0 tx=0).stop 2024-03-07T06:52:04.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.136+0000 7f441de00640 1 -- 172.21.15.133:0/3069732737 shutdown_connections 2024-03-07T06:52:04.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.136+0000 7f441de00640 1 --2- 172.21.15.133:0/3069732737 >> v2:172.21.15.161:3300/0 conn(0x7f4418072950 0x7f44180cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.136+0000 7f441de00640 1 --2- 172.21.15.133:0/3069732737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f441815fbb0 0x7f44180cf7d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.136+0000 7f441de00640 1 -- 172.21.15.133:0/3069732737 >> 172.21.15.133:0/3069732737 conn(0x7f44180b3120 msgr2=0x7f44180b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:04.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.136+0000 7f441de00640 1 -- 172.21.15.133:0/3069732737 shutdown_connections 2024-03-07T06:52:04.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.136+0000 7f441de00640 1 -- 172.21.15.133:0/3069732737 wait complete. 2024-03-07T06:52:04.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.137+0000 7f441de00640 1 Processor -- start 2024-03-07T06:52:04.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.138+0000 7f441de00640 1 -- start start 2024-03-07T06:52:04.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f441de00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f441815fbb0 0x7f441815c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:04.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f441de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f441815ff90 0x7f441815d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:04.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f441de00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f441814d920 0x7f44180ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:04.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f441de00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44180b1a00 con 0x7f441815ff90 2024-03-07T06:52:04.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f441de00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4418160980 con 0x7f441815fbb0 2024-03-07T06:52:04.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f441de00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44181603b0 con 0x7f441814d920 2024-03-07T06:52:04.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f4416000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f441815ff90 0x7f441815d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:04.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f4416000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f441815ff90 0x7f441815d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47388/0 (socket says 172.21.15.133:47388) 2024-03-07T06:52:04.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f4416000640 1 -- 172.21.15.133:0/3626205663 learned_addr learned my addr 172.21.15.133:0/3626205663 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:04.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f4416a00640 1 --2- 172.21.15.133:0/3626205663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f441815fbb0 0x7f441815c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:04.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f4416000640 1 -- 172.21.15.133:0/3626205663 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f441814d920 msgr2=0x7f44180ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:04.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f4416000640 1 --2- 172.21.15.133:0/3626205663 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f441814d920 0x7f44180ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f4416000640 1 -- 172.21.15.133:0/3626205663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f441815fbb0 msgr2=0x7f441815c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:04.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f4416000640 1 --2- 172.21.15.133:0/3626205663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f441815fbb0 0x7f441815c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.139+0000 7f4416000640 1 -- 172.21.15.133:0/3626205663 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441815c900 con 0x7f441815ff90 2024-03-07T06:52:04.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.140+0000 7f4416000640 1 --2- 172.21.15.133:0/3626205663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f441815ff90 0x7f441815d9a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f440400d600 tx=0x7f440400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:04.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.140+0000 7f440f400640 1 -- 172.21.15.133:0/3626205663 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44040125f0 con 0x7f441815ff90 2024-03-07T06:52:04.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.140+0000 7f440f400640 1 -- 172.21.15.133:0/3626205663 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4404019070 con 0x7f441815ff90 2024-03-07T06:52:04.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.140+0000 7f441de00640 1 -- 172.21.15.133:0/3626205663 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441815cbe0 con 0x7f441815ff90 2024-03-07T06:52:04.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.140+0000 7f440f400640 1 -- 172.21.15.133:0/3626205663 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44040129b0 con 0x7f441815ff90 2024-03-07T06:52:04.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.140+0000 7f441de00640 1 -- 172.21.15.133:0/3626205663 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441815d480 con 0x7f441815ff90 2024-03-07T06:52:04.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.141+0000 7f441de00640 1 -- 172.21.15.133:0/3626205663 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4418062c90 con 0x7f441815ff90 2024-03-07T06:52:04.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.142+0000 7f440f400640 1 -- 172.21.15.133:0/3626205663 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4404012b50 con 0x7f441815ff90 2024-03-07T06:52:04.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.143+0000 7f440f400640 1 --2- 172.21.15.133:0/3626205663 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f43f806b500 0x7f43f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:04.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.144+0000 7f440f400640 1 -- 172.21.15.133:0/3626205663 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f440408a5e0 con 0x7f441815ff90 2024-03-07T06:52:04.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.144+0000 7f4416a00640 1 --2- 172.21.15.133:0/3626205663 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f43f806b500 0x7f43f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:04.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.145+0000 7f4416a00640 1 --2- 172.21.15.133:0/3626205663 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f43f806b500 0x7f43f806d9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4408036000 tx=0x7f4408008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:04.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.148+0000 7f440f400640 1 -- 172.21.15.133:0/3626205663 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f44040593f0 con 0x7f441815ff90 2024-03-07T06:52:04.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.344+0000 7f441de00640 1 -- 172.21.15.133:0/3626205663 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44180b2db0 con 0x7f441815ff90 2024-03-07T06:52:04.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.346+0000 7f440f400640 1 -- 172.21.15.133:0/3626205663 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16901 (secure 0 0 0) 0x7f4404058cf0 con 0x7f441815ff90 2024-03-07T06:52:04.346 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:04.346 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:56.413096+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:52:04.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.349+0000 7f441de00640 1 -- 172.21.15.133:0/3626205663 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f43f806b500 msgr2=0x7f43f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:04.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.349+0000 7f441de00640 1 --2- 172.21.15.133:0/3626205663 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f43f806b500 0x7f43f806d9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4408036000 tx=0x7f4408008030 comp rx=0 tx=0).stop 2024-03-07T06:52:04.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.349+0000 7f441de00640 1 -- 172.21.15.133:0/3626205663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f441815ff90 msgr2=0x7f441815d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:04.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.349+0000 7f441de00640 1 --2- 172.21.15.133:0/3626205663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f441815ff90 0x7f441815d9a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f440400d600 tx=0x7f440400dac0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.349+0000 7f441de00640 1 -- 172.21.15.133:0/3626205663 shutdown_connections 2024-03-07T06:52:04.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.349+0000 7f441de00640 1 --2- 172.21.15.133:0/3626205663 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f441814d920 0x7f44180ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.349+0000 7f441de00640 1 --2- 172.21.15.133:0/3626205663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f441815ff90 0x7f441815d9a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.349+0000 7f441de00640 1 --2- 172.21.15.133:0/3626205663 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f43f806b500 0x7f43f806d9c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.349+0000 7f441de00640 1 --2- 172.21.15.133:0/3626205663 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f441815fbb0 0x7f441815c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.349+0000 7f441de00640 1 -- 172.21.15.133:0/3626205663 >> 172.21.15.133:0/3626205663 conn(0x7f44180b3120 msgr2=0x7f441806d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:04.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.349+0000 7f441de00640 1 -- 172.21.15.133:0/3626205663 shutdown_connections 2024-03-07T06:52:04.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.349+0000 7f441de00640 1 -- 172.21.15.133:0/3626205663 wait complete. 2024-03-07T06:52:04.369 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:51:12.858333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:51:13.615348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:52:04.369 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:52:04.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.582+0000 7f57e1c00640 1 Processor -- start 2024-03-07T06:52:04.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.582+0000 7f57e1c00640 1 -- start start 2024-03-07T06:52:04.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.584+0000 7f57e1c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f57dc162350 0x7f57dc0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:04.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.584+0000 7f57e1c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f57dc0b4270 0x7f57dc06df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:04.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.584+0000 7f57e1c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57dc072310 con 0x7f57dc06e4b0 2024-03-07T06:52:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.584+0000 7f57e1c00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57dc06e920 con 0x7f57dc162350 2024-03-07T06:52:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.584+0000 7f57e1c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57dc0725f0 con 0x7f57dc0b4270 2024-03-07T06:52:04.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.584+0000 7f57daa00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f57dc162350 0x7f57dc0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.584+0000 7f57daa00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f57dc162350 0x7f57dc0719a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47394/0 (socket says 172.21.15.133:47394) 2024-03-07T06:52:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.584+0000 7f57daa00640 1 -- 172.21.15.133:0/1655638055 learned_addr learned my addr 172.21.15.133:0/1655638055 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.584+0000 7f57da000640 1 --2- 172.21.15.133:0/1655638055 >> v2:172.21.15.161:3300/0 conn(0x7f57dc0b4270 0x7f57dc06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.585+0000 7f57daa00640 1 -- 172.21.15.133:0/1655638055 >> v2:172.21.15.161:3300/0 conn(0x7f57dc0b4270 msgr2=0x7f57dc06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.585+0000 7f57daa00640 1 --2- 172.21.15.133:0/1655638055 >> v2:172.21.15.161:3300/0 conn(0x7f57dc0b4270 0x7f57dc06df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.585+0000 7f57daa00640 1 -- 172.21.15.133:0/1655638055 >> v1:172.21.15.161:6789/0 conn(0x7f57dc06e4b0 legacy=0x7f57dc166050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.585+0000 7f57daa00640 1 -- 172.21.15.133:0/1655638055 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc0728d0 con 0x7f57dc162350 2024-03-07T06:52:04.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.585+0000 7f57da000640 1 --2- 172.21.15.133:0/1655638055 >> v2:172.21.15.161:3300/0 conn(0x7f57dc0b4270 0x7f57dc06df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:52:04.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.586+0000 7f57daa00640 1 --2- 172.21.15.133:0/1655638055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc162350 0x7f57dc0719a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f57dc071760 tx=0x7f57cc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7068ea772e5066f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:04.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.587+0000 7f57d9600640 1 -- 172.21.15.133:0/1655638055 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc034070 con 0x7f57dc162350 2024-03-07T06:52:04.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.587+0000 7f57d9600640 1 -- 172.21.15.133:0/1655638055 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57cc030860 con 0x7f57dc162350 2024-03-07T06:52:04.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.587+0000 7f57d9600640 1 -- 172.21.15.133:0/1655638055 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57cc030c30 con 0x7f57dc162350 2024-03-07T06:52:04.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.587+0000 7f57e1c00640 1 -- 172.21.15.133:0/1655638055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc162350 msgr2=0x7f57dc0719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:04.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.587+0000 7f57e1c00640 1 --2- 172.21.15.133:0/1655638055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc162350 0x7f57dc0719a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f57dc071760 tx=0x7f57cc02b9a0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.587+0000 7f57e1c00640 1 -- 172.21.15.133:0/1655638055 shutdown_connections 2024-03-07T06:52:04.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.587+0000 7f57e1c00640 1 --2- 172.21.15.133:0/1655638055 >> v2:172.21.15.161:3300/0 conn(0x7f57dc0b4270 0x7f57dc06df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.587+0000 7f57e1c00640 1 --2- 172.21.15.133:0/1655638055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc162350 0x7f57dc0719a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.587+0000 7f57e1c00640 1 -- 172.21.15.133:0/1655638055 >> 172.21.15.133:0/1655638055 conn(0x7f57dc0b1d30 msgr2=0x7f57dc0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:04.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.587+0000 7f57e1c00640 1 -- 172.21.15.133:0/1655638055 shutdown_connections 2024-03-07T06:52:04.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.588+0000 7f57e1c00640 1 -- 172.21.15.133:0/1655638055 wait complete. 2024-03-07T06:52:04.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.589+0000 7f57e1c00640 1 Processor -- start 2024-03-07T06:52:04.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.589+0000 7f57e1c00640 1 -- start start 2024-03-07T06:52:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.590+0000 7f57e1c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57dc06e4b0 0x7f57dc149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.590+0000 7f57e1c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57dc0b4270 0x7f57dc0c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.590+0000 7f57e1c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc162350 0x7f57dc0c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.590+0000 7f57e1c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc071ee0 con 0x7f57dc162350 2024-03-07T06:52:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.590+0000 7f57e1c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc072e90 con 0x7f57dc06e4b0 2024-03-07T06:52:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.590+0000 7f57e1c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc072060 con 0x7f57dc0b4270 2024-03-07T06:52:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.590+0000 7f57db400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc162350 0x7f57dc0c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.591+0000 7f57db400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc162350 0x7f57dc0c5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47396/0 (socket says 172.21.15.133:47396) 2024-03-07T06:52:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.591+0000 7f57db400640 1 -- 172.21.15.133:0/1449496915 learned_addr learned my addr 172.21.15.133:0/1449496915 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.591+0000 7f57daa00640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57dc06e4b0 0x7f57dc149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.591+0000 7f57da000640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57dc0b4270 0x7f57dc0c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.591+0000 7f57db400640 1 -- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57dc0b4270 msgr2=0x7f57dc0c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:04.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.591+0000 7f57db400640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57dc0b4270 0x7f57dc0c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.591+0000 7f57db400640 1 -- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57dc06e4b0 msgr2=0x7f57dc149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:04.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.591+0000 7f57db400640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57dc06e4b0 0x7f57dc149e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.591+0000 7f57db400640 1 -- 172.21.15.133:0/1449496915 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57dc14a3d0 con 0x7f57dc162350 2024-03-07T06:52:04.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.591+0000 7f57daa00640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57dc06e4b0 0x7f57dc149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:04.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.591+0000 7f57da000640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57dc0b4270 0x7f57dc0c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:04.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.591+0000 7f57db400640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc162350 0x7f57dc0c5680 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f57d400c7a0 tx=0x7f57d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:04.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.592+0000 7f57d2a00640 1 -- 172.21.15.133:0/1449496915 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4003b70 con 0x7f57dc162350 2024-03-07T06:52:04.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.592+0000 7f57d2a00640 1 -- 172.21.15.133:0/1449496915 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d4003d30 con 0x7f57dc162350 2024-03-07T06:52:04.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.592+0000 7f57d2a00640 1 -- 172.21.15.133:0/1449496915 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d400aa90 con 0x7f57dc162350 2024-03-07T06:52:04.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.592+0000 7f57e1c00640 1 -- 172.21.15.133:0/1449496915 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57dc160d50 con 0x7f57dc162350 2024-03-07T06:52:04.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.592+0000 7f57e1c00640 1 -- 172.21.15.133:0/1449496915 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57dc1604b0 con 0x7f57dc162350 2024-03-07T06:52:04.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.593+0000 7f57e1c00640 1 -- 172.21.15.133:0/1449496915 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57dc062c90 con 0x7f57dc162350 2024-03-07T06:52:04.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.594+0000 7f57d2a00640 1 -- 172.21.15.133:0/1449496915 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f57d400ac30 con 0x7f57dc162350 2024-03-07T06:52:04.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.595+0000 7f57d2a00640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57bc06b4d0 0x7f57bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:04.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.595+0000 7f57d2a00640 1 -- 172.21.15.133:0/1449496915 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f57d408a1d0 con 0x7f57dc162350 2024-03-07T06:52:04.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.595+0000 7f57daa00640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57bc06b4d0 0x7f57bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:04.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.596+0000 7f57daa00640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57bc06b4d0 0x7f57bc06d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f57cc036000 tx=0x7f57cc03b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:04.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.599+0000 7f57d2a00640 1 -- 172.21.15.133:0/1449496915 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f57d4058fc0 con 0x7f57dc162350 2024-03-07T06:52:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.794+0000 7f57e1c00640 1 -- 172.21.15.133:0/1449496915 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57dc0cc590 con 0x7f57dc162350 2024-03-07T06:52:04.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.795+0000 7f57d2a00640 1 -- 172.21.15.133:0/1449496915 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57d40588c0 con 0x7f57dc162350 2024-03-07T06:52:04.795 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:52:04.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.797+0000 7f57e1c00640 1 -- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57bc06b4d0 msgr2=0x7f57bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.797+0000 7f57e1c00640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57bc06b4d0 0x7f57bc06d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f57cc036000 tx=0x7f57cc03b040 comp rx=0 tx=0).stop 2024-03-07T06:52:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.797+0000 7f57e1c00640 1 -- 172.21.15.133:0/1449496915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc162350 msgr2=0x7f57dc0c5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.797+0000 7f57e1c00640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc162350 0x7f57dc0c5680 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f57d400c7a0 tx=0x7f57d400cc60 comp rx=0 tx=0).stop 2024-03-07T06:52:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.797+0000 7f57e1c00640 1 -- 172.21.15.133:0/1449496915 shutdown_connections 2024-03-07T06:52:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.798+0000 7f57e1c00640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57dc162350 0x7f57dc0c5680 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.798+0000 7f57e1c00640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57dc0b4270 0x7f57dc0c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.798+0000 7f57e1c00640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57bc06b4d0 0x7f57bc06d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.798+0000 7f57e1c00640 1 --2- 172.21.15.133:0/1449496915 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57dc06e4b0 0x7f57dc149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.798+0000 7f57e1c00640 1 -- 172.21.15.133:0/1449496915 >> 172.21.15.133:0/1449496915 conn(0x7f57dc0b1d30 msgr2=0x7f57dc0cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.798+0000 7f57e1c00640 1 -- 172.21.15.133:0/1449496915 shutdown_connections 2024-03-07T06:52:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:04.798+0000 7f57e1c00640 1 -- 172.21.15.133:0/1449496915 wait complete. 2024-03-07T06:52:04.817 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:52:05.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.031+0000 7f57cd000640 1 Processor -- start 2024-03-07T06:52:05.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.031+0000 7f57cd000640 1 -- start start 2024-03-07T06:52:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.032+0000 7f57cd000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f57c8159880 0x7f57c80cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.032+0000 7f57cd000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f57c80b4f10 0x7f57c80b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.032+0000 7f57cd000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c8160720 con 0x7f57c80ccff0 2024-03-07T06:52:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.032+0000 7f57cd000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c80b5880 con 0x7f57c8159880 2024-03-07T06:52:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.032+0000 7f57cd000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c80cde00 con 0x7f57c80b4f10 2024-03-07T06:52:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.033+0000 7f57c7400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f57c8159880 0x7f57c80cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.033+0000 7f57c7400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f57c8159880 0x7f57c80cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47404/0 (socket says 172.21.15.133:47404) 2024-03-07T06:52:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.033+0000 7f57c7400640 1 -- 172.21.15.133:0/1346168562 learned_addr learned my addr 172.21.15.133:0/1346168562 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.033+0000 7f57c7e00640 1 --2- 172.21.15.133:0/1346168562 >> v2:172.21.15.161:3300/0 conn(0x7f57c80b4f10 0x7f57c80b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.033+0000 7f57c7400640 1 -- 172.21.15.133:0/1346168562 >> v2:172.21.15.161:3300/0 conn(0x7f57c80b4f10 msgr2=0x7f57c80b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.033+0000 7f57c7400640 1 --2- 172.21.15.133:0/1346168562 >> v2:172.21.15.161:3300/0 conn(0x7f57c80b4f10 0x7f57c80b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.033+0000 7f57c7400640 1 -- 172.21.15.133:0/1346168562 >> v1:172.21.15.161:6789/0 conn(0x7f57c80ccff0 legacy=0x7f57c8160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:05.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.033+0000 7f57c7400640 1 -- 172.21.15.133:0/1346168562 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c80cd560 con 0x7f57c8159880 2024-03-07T06:52:05.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.034+0000 7f57c7400640 1 --2- 172.21.15.133:0/1346168562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c8159880 0x7f57c80cf2b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f57c80cfb00 tx=0x7f57b802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729986663028c947 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:05.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.035+0000 7f57c6a00640 1 -- 172.21.15.133:0/1346168562 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8035020 con 0x7f57c8159880 2024-03-07T06:52:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.035+0000 7f57c6a00640 1 -- 172.21.15.133:0/1346168562 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b8031870 con 0x7f57c8159880 2024-03-07T06:52:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.035+0000 7f57c6a00640 1 -- 172.21.15.133:0/1346168562 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8031c40 con 0x7f57c8159880 2024-03-07T06:52:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.035+0000 7f57cd000640 1 -- 172.21.15.133:0/1346168562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c8159880 msgr2=0x7f57c80cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.035+0000 7f57cd000640 1 --2- 172.21.15.133:0/1346168562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c8159880 0x7f57c80cf2b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f57c80cfb00 tx=0x7f57b802b910 comp rx=0 tx=0).stop 2024-03-07T06:52:05.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.035+0000 7f57cd000640 1 -- 172.21.15.133:0/1346168562 shutdown_connections 2024-03-07T06:52:05.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.035+0000 7f57cd000640 1 --2- 172.21.15.133:0/1346168562 >> v2:172.21.15.161:3300/0 conn(0x7f57c80b4f10 0x7f57c80b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.035+0000 7f57cd000640 1 --2- 172.21.15.133:0/1346168562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c8159880 0x7f57c80cf2b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.035+0000 7f57cd000640 1 -- 172.21.15.133:0/1346168562 >> 172.21.15.133:0/1346168562 conn(0x7f57c8071860 msgr2=0x7f57c8071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:05.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.036+0000 7f57cd000640 1 -- 172.21.15.133:0/1346168562 shutdown_connections 2024-03-07T06:52:05.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.036+0000 7f57cd000640 1 -- 172.21.15.133:0/1346168562 wait complete. 2024-03-07T06:52:05.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.037+0000 7f57cd000640 1 Processor -- start 2024-03-07T06:52:05.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.037+0000 7f57cd000640 1 -- start start 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.038+0000 7f57cd000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c80b4f10 0x7f57c81f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.038+0000 7f57cd000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c80ccff0 0x7f57c814b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.038+0000 7f57cd000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c8159880 0x7f57c81f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.038+0000 7f57cd000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c815faf0 con 0x7f57c80b4f10 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.038+0000 7f57cd000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c80cdb20 con 0x7f57c8159880 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.038+0000 7f57cd000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c8159c60 con 0x7f57c80ccff0 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.038+0000 7f57c7400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c80b4f10 0x7f57c81f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.038+0000 7f57c7400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c80b4f10 0x7f57c81f45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47418/0 (socket says 172.21.15.133:47418) 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.038+0000 7f57c7400640 1 -- 172.21.15.133:0/3870448371 learned_addr learned my addr 172.21.15.133:0/3870448371 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.039+0000 7f57c7400640 1 -- 172.21.15.133:0/3870448371 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c80ccff0 msgr2=0x7f57c814b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.039+0000 7f57c7400640 1 --2- 172.21.15.133:0/3870448371 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c80ccff0 0x7f57c814b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.039+0000 7f57c7400640 1 -- 172.21.15.133:0/3870448371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c8159880 msgr2=0x7f57c81f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.039+0000 7f57c7400640 1 --2- 172.21.15.133:0/3870448371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c8159880 0x7f57c81f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.039+0000 7f57c7400640 1 -- 172.21.15.133:0/3870448371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c81f7e00 con 0x7f57c80b4f10 2024-03-07T06:52:05.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.039+0000 7f57c7400640 1 --2- 172.21.15.133:0/3870448371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c80b4f10 0x7f57c81f45b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f57b8000c00 tx=0x7f57b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:05.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.039+0000 7f57bf400640 1 -- 172.21.15.133:0/3870448371 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8040070 con 0x7f57c80b4f10 2024-03-07T06:52:05.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.039+0000 7f57bf400640 1 -- 172.21.15.133:0/3870448371 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b8030d00 con 0x7f57c80b4f10 2024-03-07T06:52:05.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.039+0000 7f57cd000640 1 -- 172.21.15.133:0/3870448371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c8156fe0 con 0x7f57c80b4f10 2024-03-07T06:52:05.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.039+0000 7f57bf400640 1 -- 172.21.15.133:0/3870448371 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8035040 con 0x7f57c80b4f10 2024-03-07T06:52:05.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.040+0000 7f57cd000640 1 -- 172.21.15.133:0/3870448371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c81f3ad0 con 0x7f57c80b4f10 2024-03-07T06:52:05.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.041+0000 7f57cd000640 1 -- 172.21.15.133:0/3870448371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c8067c80 con 0x7f57c80b4f10 2024-03-07T06:52:05.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.042+0000 7f57bf400640 1 -- 172.21.15.133:0/3870448371 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f57b804b060 con 0x7f57c80b4f10 2024-03-07T06:52:05.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.042+0000 7f57bf400640 1 --2- 172.21.15.133:0/3870448371 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57a406b500 0x7f57a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.043+0000 7f57bf400640 1 -- 172.21.15.133:0/3870448371 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f57b80ac060 con 0x7f57c80b4f10 2024-03-07T06:52:05.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.043+0000 7f57bfe00640 1 --2- 172.21.15.133:0/3870448371 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57a406b500 0x7f57a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.044+0000 7f57bfe00640 1 --2- 172.21.15.133:0/3870448371 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57a406b500 0x7f57a406d9c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f57b0006fd0 tx=0x7f57b0006bc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:05.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.047+0000 7f57bf400640 1 -- 172.21.15.133:0/3870448371 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f57b807a800 con 0x7f57c80b4f10 2024-03-07T06:52:05.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.240+0000 7f57cd000640 1 -- 172.21.15.133:0/3870448371 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57c814efe0 con 0x7f57c80b4f10 2024-03-07T06:52:05.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.241+0000 7f57bf400640 1 -- 172.21.15.133:0/3870448371 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57b807e8a0 con 0x7f57c80b4f10 2024-03-07T06:52:05.244 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:52:05.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.244+0000 7f57cd000640 1 -- 172.21.15.133:0/3870448371 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57a406b500 msgr2=0x7f57a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.244+0000 7f57cd000640 1 --2- 172.21.15.133:0/3870448371 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57a406b500 0x7f57a406d9c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f57b0006fd0 tx=0x7f57b0006bc0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.244+0000 7f57cd000640 1 -- 172.21.15.133:0/3870448371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c80b4f10 msgr2=0x7f57c81f45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.244+0000 7f57cd000640 1 --2- 172.21.15.133:0/3870448371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c80b4f10 0x7f57c81f45b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f57b8000c00 tx=0x7f57b80046c0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.244+0000 7f57cd000640 1 -- 172.21.15.133:0/3870448371 shutdown_connections 2024-03-07T06:52:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.244+0000 7f57cd000640 1 --2- 172.21.15.133:0/3870448371 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57a406b500 0x7f57a406d9c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.244+0000 7f57cd000640 1 --2- 172.21.15.133:0/3870448371 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57c8159880 0x7f57c81f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.244+0000 7f57cd000640 1 --2- 172.21.15.133:0/3870448371 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57c80ccff0 0x7f57c814b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.244+0000 7f57cd000640 1 --2- 172.21.15.133:0/3870448371 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57c80b4f10 0x7f57c81f45b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.244+0000 7f57cd000640 1 -- 172.21.15.133:0/3870448371 >> 172.21.15.133:0/3870448371 conn(0x7f57c8071860 msgr2=0x7f57c8155bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.244+0000 7f57cd000640 1 -- 172.21.15.133:0/3870448371 shutdown_connections 2024-03-07T06:52:05.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.244+0000 7f57cd000640 1 -- 172.21.15.133:0/3870448371 wait complete. 2024-03-07T06:52:05.263 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:52:05.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.476+0000 7f244a000640 1 Processor -- start 2024-03-07T06:52:05.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.476+0000 7f244a000640 1 -- start start 2024-03-07T06:52:05.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.478+0000 7f244a000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f24440cf610 0x7f24440cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.478+0000 7f244a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f24440cda20 0x7f24440719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.478+0000 7f244a000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244406e150 con 0x7f2444145670 2024-03-07T06:52:05.478 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.478+0000 7f244a000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24440cde70 con 0x7f24440cf610 2024-03-07T06:52:05.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.478+0000 7f244a000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244406ed60 con 0x7f24440cda20 2024-03-07T06:52:05.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.478+0000 7f2443400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2444145670 0x7f244415cf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47804/0 (socket says 172.21.15.133:47804) 2024-03-07T06:52:05.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.478+0000 7f2443400640 1 -- 172.21.15.133:0/42729720 learned_addr learned my addr 172.21.15.133:0/42729720 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:05.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.478+0000 7f2442a00640 1 --2- 172.21.15.133:0/42729720 >> v2:172.21.15.133:3300/0 conn(0x7f24440cf610 0x7f24440cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.478+0000 7f2443e00640 1 --2- 172.21.15.133:0/42729720 >> v2:172.21.15.161:3300/0 conn(0x7f24440cda20 0x7f24440719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.478+0000 7f2442000640 1 -- 172.21.15.133:0/42729720 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3382186068 0 0) 0x7f244406e150 con 0x7f2444145670 2024-03-07T06:52:05.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.478+0000 7f2442000640 1 -- 172.21.15.133:0/42729720 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2428003520 con 0x7f2444145670 2024-03-07T06:52:05.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.478+0000 7f2442a00640 1 -- 172.21.15.133:0/42729720 >> v2:172.21.15.161:3300/0 conn(0x7f24440cda20 msgr2=0x7f24440719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:05.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.478+0000 7f2442a00640 1 --2- 172.21.15.133:0/42729720 >> v2:172.21.15.161:3300/0 conn(0x7f24440cda20 0x7f24440719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.478+0000 7f2442a00640 1 -- 172.21.15.133:0/42729720 >> v1:172.21.15.133:6789/0 conn(0x7f2444145670 legacy=0x7f244415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.479+0000 7f2442a00640 1 -- 172.21.15.133:0/42729720 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244406e430 con 0x7f24440cf610 2024-03-07T06:52:05.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.480+0000 7f2442a00640 1 --2- 172.21.15.133:0/42729720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24440cf610 0x7f24440cd4e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f24300097a0 tx=0x7f243002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a309dc9aacbf8896 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:05.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.481+0000 7f2442000640 1 -- 172.21.15.133:0/42729720 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24300315a0 con 0x7f24440cf610 2024-03-07T06:52:05.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.481+0000 7f2442000640 1 -- 172.21.15.133:0/42729720 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2430037070 con 0x7f24440cf610 2024-03-07T06:52:05.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.481+0000 7f2442000640 1 -- 172.21.15.133:0/42729720 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430031960 con 0x7f24440cf610 2024-03-07T06:52:05.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.482+0000 7f244a000640 1 -- 172.21.15.133:0/42729720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24440cf610 msgr2=0x7f24440cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.482+0000 7f244a000640 1 --2- 172.21.15.133:0/42729720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24440cf610 0x7f24440cd4e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f24300097a0 tx=0x7f243002b840 comp rx=0 tx=0).stop 2024-03-07T06:52:05.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.482+0000 7f244a000640 1 -- 172.21.15.133:0/42729720 shutdown_connections 2024-03-07T06:52:05.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.482+0000 7f244a000640 1 --2- 172.21.15.133:0/42729720 >> v2:172.21.15.161:3300/0 conn(0x7f24440cda20 0x7f24440719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.482+0000 7f244a000640 1 --2- 172.21.15.133:0/42729720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24440cf610 0x7f24440cd4e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.482+0000 7f244a000640 1 -- 172.21.15.133:0/42729720 >> 172.21.15.133:0/42729720 conn(0x7f24440b1d30 msgr2=0x7f24440b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:05.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.482+0000 7f244a000640 1 -- 172.21.15.133:0/42729720 shutdown_connections 2024-03-07T06:52:05.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.482+0000 7f244a000640 1 -- 172.21.15.133:0/42729720 wait complete. 2024-03-07T06:52:05.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.484+0000 7f244a000640 1 Processor -- start 2024-03-07T06:52:05.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.484+0000 7f244a000640 1 -- start start 2024-03-07T06:52:05.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.485+0000 7f244a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24440cda20 0x7f2444071c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.485+0000 7f244a000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24440cf610 0x7f24440aa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.485+0000 7f244a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2444145670 0x7f244415b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.485+0000 7f244a000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2444145a50 con 0x7f2444145670 2024-03-07T06:52:05.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.485+0000 7f244a000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244406e9f0 con 0x7f24440cda20 2024-03-07T06:52:05.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.485+0000 7f244a000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2444145bd0 con 0x7f24440cf610 2024-03-07T06:52:05.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.486+0000 7f2443e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2444145670 0x7f244415b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.486+0000 7f2443e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2444145670 0x7f244415b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47428/0 (socket says 172.21.15.133:47428) 2024-03-07T06:52:05.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.486+0000 7f2443e00640 1 -- 172.21.15.133:0/626743828 learned_addr learned my addr 172.21.15.133:0/626743828 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:05.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.486+0000 7f2442a00640 1 --2- 172.21.15.133:0/626743828 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24440cf610 0x7f24440aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.486+0000 7f2443e00640 1 -- 172.21.15.133:0/626743828 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24440cf610 msgr2=0x7f24440aa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.486+0000 7f2443e00640 1 --2- 172.21.15.133:0/626743828 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24440cf610 0x7f24440aa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.486+0000 7f2443e00640 1 -- 172.21.15.133:0/626743828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24440cda20 msgr2=0x7f2444071c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.486+0000 7f2443e00640 1 --2- 172.21.15.133:0/626743828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24440cda20 0x7f2444071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.486+0000 7f2443e00640 1 -- 172.21.15.133:0/626743828 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24440ab0e0 con 0x7f2444145670 2024-03-07T06:52:05.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.486+0000 7f2443e00640 1 --2- 172.21.15.133:0/626743828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2444145670 0x7f244415b3f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f243c007e60 tx=0x7f243c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:05.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.489+0000 7f243b400640 1 -- 172.21.15.133:0/626743828 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c014420 con 0x7f2444145670 2024-03-07T06:52:05.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.489+0000 7f243b400640 1 -- 172.21.15.133:0/626743828 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c019070 con 0x7f2444145670 2024-03-07T06:52:05.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.489+0000 7f243b400640 1 -- 172.21.15.133:0/626743828 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c014820 con 0x7f2444145670 2024-03-07T06:52:05.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.489+0000 7f244a000640 1 -- 172.21.15.133:0/626743828 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24440ab670 con 0x7f2444145670 2024-03-07T06:52:05.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.489+0000 7f244a000640 1 -- 172.21.15.133:0/626743828 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24440abc30 con 0x7f2444145670 2024-03-07T06:52:05.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.490+0000 7f244a000640 1 -- 172.21.15.133:0/626743828 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2444062c90 con 0x7f2444145670 2024-03-07T06:52:05.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.491+0000 7f243b400640 1 -- 172.21.15.133:0/626743828 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f243c0149c0 con 0x7f2444145670 2024-03-07T06:52:05.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.492+0000 7f243b400640 1 --2- 172.21.15.133:0/626743828 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f242406b4d0 0x7f242406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.492+0000 7f243b400640 1 -- 172.21.15.133:0/626743828 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f243c08ab80 con 0x7f2444145670 2024-03-07T06:52:05.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.493+0000 7f2443400640 1 --2- 172.21.15.133:0/626743828 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f242406b4d0 0x7f242406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.494+0000 7f2443400640 1 --2- 172.21.15.133:0/626743828 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f242406b4d0 0x7f242406d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2434000f60 tx=0x7f24340078f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:05.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.497+0000 7f243b400640 1 -- 172.21.15.133:0/626743828 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f243c0598e0 con 0x7f2444145670 2024-03-07T06:52:05.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.693+0000 7f244a000640 1 -- 172.21.15.133:0/626743828 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f244415dde0 con 0x7f2444145670 2024-03-07T06:52:05.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.694+0000 7f243b400640 1 -- 172.21.15.133:0/626743828 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f243c0591e0 con 0x7f2444145670 2024-03-07T06:52:05.693 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:52:05.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.696+0000 7f244a000640 1 -- 172.21.15.133:0/626743828 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f242406b4d0 msgr2=0x7f242406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.696+0000 7f244a000640 1 --2- 172.21.15.133:0/626743828 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f242406b4d0 0x7f242406d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2434000f60 tx=0x7f24340078f0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.696+0000 7f244a000640 1 -- 172.21.15.133:0/626743828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2444145670 msgr2=0x7f244415b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.696+0000 7f244a000640 1 --2- 172.21.15.133:0/626743828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2444145670 0x7f244415b3f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f243c007e60 tx=0x7f243c00e4b0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.696+0000 7f244a000640 1 -- 172.21.15.133:0/626743828 shutdown_connections 2024-03-07T06:52:05.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.696+0000 7f244a000640 1 --2- 172.21.15.133:0/626743828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2444145670 0x7f244415b3f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.696+0000 7f244a000640 1 --2- 172.21.15.133:0/626743828 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24440cf610 0x7f24440aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.696+0000 7f244a000640 1 --2- 172.21.15.133:0/626743828 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f242406b4d0 0x7f242406d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.696+0000 7f244a000640 1 --2- 172.21.15.133:0/626743828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24440cda20 0x7f2444071c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.696+0000 7f244a000640 1 -- 172.21.15.133:0/626743828 >> 172.21.15.133:0/626743828 conn(0x7f24440b1d30 msgr2=0x7f244415e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:05.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.696+0000 7f244a000640 1 -- 172.21.15.133:0/626743828 shutdown_connections 2024-03-07T06:52:05.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.696+0000 7f244a000640 1 -- 172.21.15.133:0/626743828 wait complete. 2024-03-07T06:52:05.716 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-07T06:52:05.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.922+0000 7f6680c00640 1 Processor -- start 2024-03-07T06:52:05.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.922+0000 7f6680c00640 1 -- start start 2024-03-07T06:52:05.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.924+0000 7f6680c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f667c15be00 0x7f667c0d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.924+0000 7f6680c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f667c160240 0x7f667c0cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.924+0000 7f6680c00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667c0b7910 con 0x7f667c0d0800 2024-03-07T06:52:05.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.924+0000 7f6680c00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f667c0b7bf0 con 0x7f667c160240 2024-03-07T06:52:05.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.924+0000 7f6680c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667c0ceb20 con 0x7f667c15be00 2024-03-07T06:52:05.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.924+0000 7f667a800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f667c160240 0x7f667c0cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.924+0000 7f667a800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f667c160240 0x7f667c0cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47440/0 (socket says 172.21.15.133:47440) 2024-03-07T06:52:05.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.924+0000 7f667a800640 1 -- 172.21.15.133:0/2825896412 learned_addr learned my addr 172.21.15.133:0/2825896412 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:05.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.925+0000 7f6679e00640 1 --2- 172.21.15.133:0/2825896412 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f667c15be00 0x7f667c0d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.925+0000 7f667a800640 1 -- 172.21.15.133:0/2825896412 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f667c15be00 msgr2=0x7f667c0d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:05.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.925+0000 7f667a800640 1 --2- 172.21.15.133:0/2825896412 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f667c15be00 0x7f667c0d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.925+0000 7f667a800640 1 -- 172.21.15.133:0/2825896412 >> v1:172.21.15.133:6789/0 conn(0x7f667c0d0800 legacy=0x7f667c15fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.925+0000 7f667a800640 1 -- 172.21.15.133:0/2825896412 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c0cee00 con 0x7f667c160240 2024-03-07T06:52:05.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.925+0000 7f6679e00640 1 --2- 172.21.15.133:0/2825896412 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f667c15be00 0x7f667c0d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.926+0000 7f667a800640 1 --2- 172.21.15.133:0/2825896412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667c160240 0x7f667c0cf320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6674009870 tx=0x7f667402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dee2c22c7717b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.926+0000 7f6678a00640 1 -- 172.21.15.133:0/2825896412 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674034070 con 0x7f667c160240 2024-03-07T06:52:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.926+0000 7f6678a00640 1 -- 172.21.15.133:0/2825896412 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6674030840 con 0x7f667c160240 2024-03-07T06:52:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.926+0000 7f6678a00640 1 -- 172.21.15.133:0/2825896412 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674030c10 con 0x7f667c160240 2024-03-07T06:52:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.927+0000 7f6680c00640 1 -- 172.21.15.133:0/2825896412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667c160240 msgr2=0x7f667c0cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.927+0000 7f6680c00640 1 --2- 172.21.15.133:0/2825896412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667c160240 0x7f667c0cf320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6674009870 tx=0x7f667402b980 comp rx=0 tx=0).stop 2024-03-07T06:52:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.927+0000 7f6680c00640 1 -- 172.21.15.133:0/2825896412 shutdown_connections 2024-03-07T06:52:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.927+0000 7f6680c00640 1 --2- 172.21.15.133:0/2825896412 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667c160240 0x7f667c0cf320 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.927+0000 7f6680c00640 1 --2- 172.21.15.133:0/2825896412 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f667c15be00 0x7f667c0d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.927+0000 7f6680c00640 1 -- 172.21.15.133:0/2825896412 >> 172.21.15.133:0/2825896412 conn(0x7f667c066e50 msgr2=0x7f667c067260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.927+0000 7f6680c00640 1 -- 172.21.15.133:0/2825896412 shutdown_connections 2024-03-07T06:52:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.927+0000 7f6680c00640 1 -- 172.21.15.133:0/2825896412 wait complete. 2024-03-07T06:52:05.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.929+0000 7f6680c00640 1 Processor -- start 2024-03-07T06:52:05.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.929+0000 7f6680c00640 1 -- start start 2024-03-07T06:52:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.930+0000 7f6680c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f667c0d0800 0x7f667c153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.930+0000 7f6680c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f667c15be00 0x7f667c147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.930+0000 7f6680c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667c160240 0x7f667c144720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.930+0000 7f6680c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c0cf9e0 con 0x7f667c160240 2024-03-07T06:52:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.930+0000 7f6680c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c0cf860 con 0x7f667c15be00 2024-03-07T06:52:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.930+0000 7f6680c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667c0cfb60 con 0x7f667c0d0800 2024-03-07T06:52:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.930+0000 7f667a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667c160240 0x7f667c144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.930+0000 7f667a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667c160240 0x7f667c144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47444/0 (socket says 172.21.15.133:47444) 2024-03-07T06:52:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.930+0000 7f667a800640 1 -- 172.21.15.133:0/2889259791 learned_addr learned my addr 172.21.15.133:0/2889259791 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.931+0000 7f6679e00640 1 --2- 172.21.15.133:0/2889259791 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f667c0d0800 0x7f667c153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.931+0000 7f667a800640 1 -- 172.21.15.133:0/2889259791 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f667c0d0800 msgr2=0x7f667c153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.931+0000 7f667a800640 1 --2- 172.21.15.133:0/2889259791 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f667c0d0800 0x7f667c153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.931+0000 7f6679400640 1 --2- 172.21.15.133:0/2889259791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f667c15be00 0x7f667c147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.931+0000 7f667a800640 1 -- 172.21.15.133:0/2889259791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f667c15be00 msgr2=0x7f667c147ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:05.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.931+0000 7f667a800640 1 --2- 172.21.15.133:0/2889259791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f667c15be00 0x7f667c147ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:05.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.931+0000 7f667a800640 1 -- 172.21.15.133:0/2889259791 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c153f30 con 0x7f667c160240 2024-03-07T06:52:05.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.931+0000 7f667a800640 1 --2- 172.21.15.133:0/2889259791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667c160240 0x7f667c144720 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f667402b9c0 tx=0x7f66740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:05.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.931+0000 7f6672000640 1 -- 172.21.15.133:0/2889259791 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674034040 con 0x7f667c160240 2024-03-07T06:52:05.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.931+0000 7f6672000640 1 -- 172.21.15.133:0/2889259791 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6674039d90 con 0x7f667c160240 2024-03-07T06:52:05.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.931+0000 7f6672000640 1 -- 172.21.15.133:0/2889259791 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674043610 con 0x7f667c160240 2024-03-07T06:52:05.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.931+0000 7f6680c00640 1 -- 172.21.15.133:0/2889259791 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667c155390 con 0x7f667c160240 2024-03-07T06:52:05.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.932+0000 7f6680c00640 1 -- 172.21.15.133:0/2889259791 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667c155630 con 0x7f667c160240 2024-03-07T06:52:05.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.933+0000 7f6680c00640 1 -- 172.21.15.133:0/2889259791 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667c062c90 con 0x7f667c160240 2024-03-07T06:52:05.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.933+0000 7f6672000640 1 -- 172.21.15.133:0/2889259791 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6674037070 con 0x7f667c160240 2024-03-07T06:52:05.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.934+0000 7f6672000640 1 --2- 172.21.15.133:0/2889259791 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f665806b4d0 0x7f665806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:05.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.934+0000 7f6672000640 1 -- 172.21.15.133:0/2889259791 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f66740ac520 con 0x7f667c160240 2024-03-07T06:52:05.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.935+0000 7f6679e00640 1 --2- 172.21.15.133:0/2889259791 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f665806b4d0 0x7f665806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:05.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.936+0000 7f6679e00640 1 --2- 172.21.15.133:0/2889259791 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f665806b4d0 0x7f665806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f666400a990 tx=0x7f6664008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:05.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:05.939+0000 7f6672000640 1 -- 172.21.15.133:0/2889259791 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f667c153f30 con 0x7f667c160240 2024-03-07T06:52:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.148+0000 7f6680c00640 1 -- 172.21.15.133:0/2889259791 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f667c15f8a0 con 0x7f667c160240 2024-03-07T06:52:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.149+0000 7f6672000640 1 -- 172.21.15.133:0/2889259791 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f667403e070 con 0x7f667c160240 2024-03-07T06:52:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-03-07T06:52:06.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.151+0000 7f6680c00640 1 -- 172.21.15.133:0/2889259791 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f665806b4d0 msgr2=0x7f665806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:06.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.151+0000 7f6680c00640 1 --2- 172.21.15.133:0/2889259791 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f665806b4d0 0x7f665806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f666400a990 tx=0x7f6664008040 comp rx=0 tx=0).stop 2024-03-07T06:52:06.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.151+0000 7f6680c00640 1 -- 172.21.15.133:0/2889259791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667c160240 msgr2=0x7f667c144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:06.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.151+0000 7f6680c00640 1 --2- 172.21.15.133:0/2889259791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667c160240 0x7f667c144720 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f667402b9c0 tx=0x7f66740046c0 comp rx=0 tx=0).stop 2024-03-07T06:52:06.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.151+0000 7f6680c00640 1 -- 172.21.15.133:0/2889259791 shutdown_connections 2024-03-07T06:52:06.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.151+0000 7f6680c00640 1 --2- 172.21.15.133:0/2889259791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f667c160240 0x7f667c144720 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:06.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.151+0000 7f6680c00640 1 --2- 172.21.15.133:0/2889259791 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f665806b4d0 0x7f665806d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:06.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.151+0000 7f6680c00640 1 --2- 172.21.15.133:0/2889259791 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f667c15be00 0x7f667c147ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:06.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.151+0000 7f6680c00640 1 --2- 172.21.15.133:0/2889259791 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f667c0d0800 0x7f667c153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:06.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.151+0000 7f6680c00640 1 -- 172.21.15.133:0/2889259791 >> 172.21.15.133:0/2889259791 conn(0x7f667c066e50 msgr2=0x7f667c0740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.151+0000 7f6680c00640 1 -- 172.21.15.133:0/2889259791 shutdown_connections 2024-03-07T06:52:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.151+0000 7f6680c00640 1 -- 172.21.15.133:0/2889259791 wait complete. 2024-03-07T06:52:06.170 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-07T06:52:06.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:06.170 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-07T06:52:06.276 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-03-07T06:52:06.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:06.276 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-07T06:52:06.337 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:52:06.337 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:52:06.392 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:52:06.392 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:52:06.443 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:52:06.443 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:52:06.492 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:52:06.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.751+0000 7f280bc00640 1 Processor -- start 2024-03-07T06:52:06.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.751+0000 7f280bc00640 1 -- start start 2024-03-07T06:52:06.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.753+0000 7f280bc00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2804071610 0x7f28040cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:06.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.753+0000 7f280bc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28040cdee0 0x7f28040aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:06.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.753+0000 7f280bc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28040cfd10 con 0x7f28040af310 2024-03-07T06:52:06.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.753+0000 7f280bc00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2804073450 con 0x7f2804071610 2024-03-07T06:52:06.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.753+0000 7f280bc00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040adbb0 con 0x7f28040cdee0 2024-03-07T06:52:06.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.753+0000 7f2809200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2804071610 0x7f28040cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:06.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.753+0000 7f2809200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2804071610 0x7f28040cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47456/0 (socket says 172.21.15.133:47456) 2024-03-07T06:52:06.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.753+0000 7f2809200640 1 -- 172.21.15.133:0/438426796 learned_addr learned my addr 172.21.15.133:0/438426796 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:06.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.753+0000 7f2809200640 1 -- 172.21.15.133:0/438426796 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28040cdee0 msgr2=0x7f28040aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:06.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.753+0000 7f2809200640 1 --2- 172.21.15.133:0/438426796 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28040cdee0 0x7f28040aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:06.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.753+0000 7f2809200640 1 -- 172.21.15.133:0/438426796 >> v1:172.21.15.161:6789/0 conn(0x7f28040af310 legacy=0x7f28040ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:06.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.753+0000 7f2809200640 1 -- 172.21.15.133:0/438426796 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2804072e90 con 0x7f2804071610 2024-03-07T06:52:06.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.754+0000 7f2809200640 1 --2- 172.21.15.133:0/438426796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2804071610 0x7f28040cf7d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009870 tx=0x7f27f802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bceaa89ab01f602 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:06.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.754+0000 7f2803400640 1 -- 172.21.15.133:0/438426796 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f80317c0 con 0x7f2804071610 2024-03-07T06:52:06.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.754+0000 7f2803400640 1 -- 172.21.15.133:0/438426796 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f8037070 con 0x7f2804071610 2024-03-07T06:52:06.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.754+0000 7f2803400640 1 -- 172.21.15.133:0/438426796 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8031b80 con 0x7f2804071610 2024-03-07T06:52:06.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.754+0000 7f280bc00640 1 -- 172.21.15.133:0/438426796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2804071610 msgr2=0x7f28040cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:06.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.755+0000 7f280bc00640 1 --2- 172.21.15.133:0/438426796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2804071610 0x7f28040cf7d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009870 tx=0x7f27f802b860 comp rx=0 tx=0).stop 2024-03-07T06:52:06.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.755+0000 7f280bc00640 1 -- 172.21.15.133:0/438426796 shutdown_connections 2024-03-07T06:52:06.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.755+0000 7f280bc00640 1 --2- 172.21.15.133:0/438426796 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28040cdee0 0x7f28040aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:06.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.755+0000 7f280bc00640 1 --2- 172.21.15.133:0/438426796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2804071610 0x7f28040cf7d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:06.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.755+0000 7f280bc00640 1 -- 172.21.15.133:0/438426796 >> 172.21.15.133:0/438426796 conn(0x7f28040b3120 msgr2=0x7f28040b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:06.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.755+0000 7f280bc00640 1 -- 172.21.15.133:0/438426796 shutdown_connections 2024-03-07T06:52:06.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.755+0000 7f280bc00640 1 -- 172.21.15.133:0/438426796 wait complete. 2024-03-07T06:52:06.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.756+0000 7f280bc00640 1 Processor -- start 2024-03-07T06:52:06.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.757+0000 7f280bc00640 1 -- start start 2024-03-07T06:52:06.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.757+0000 7f280bc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2804071610 0x7f28041546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:06.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.757+0000 7f280bc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28040af310 0x7f2804154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:06.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.757+0000 7f280bc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28040cdee0 0x7f280415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:06.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.757+0000 7f280bc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040ae750 con 0x7f28040af310 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.757+0000 7f280bc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040ad840 con 0x7f28040cdee0 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.757+0000 7f280bc00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040ae8d0 con 0x7f2804071610 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.758+0000 7f2803e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28040af310 0x7f2804154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.758+0000 7f2803e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28040af310 0x7f2804154c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47468/0 (socket says 172.21.15.133:47468) 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.758+0000 7f2803e00640 1 -- 172.21.15.133:0/3121683935 learned_addr learned my addr 172.21.15.133:0/3121683935 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.758+0000 7f2809c00640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28040cdee0 0x7f280415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.758+0000 7f2809200640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2804071610 0x7f28041546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.758+0000 7f2803e00640 1 -- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2804071610 msgr2=0x7f28041546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.758+0000 7f2803e00640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2804071610 0x7f28041546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.758+0000 7f2803e00640 1 -- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28040cdee0 msgr2=0x7f280415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.758+0000 7f2803e00640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28040cdee0 0x7f280415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.758+0000 7f2803e00640 1 -- 172.21.15.133:0/3121683935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280415d520 con 0x7f28040af310 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.758+0000 7f2809c00640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28040cdee0 0x7f280415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.758+0000 7f2803e00640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28040af310 0x7f2804154c20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f27f000b4d0 tx=0x7f27f000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:06.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.759+0000 7f2800c00640 1 -- 172.21.15.133:0/3121683935 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0014070 con 0x7f28040af310 2024-03-07T06:52:06.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.759+0000 7f2800c00640 1 -- 172.21.15.133:0/3121683935 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f0010870 con 0x7f28040af310 2024-03-07T06:52:06.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.759+0000 7f2800c00640 1 -- 172.21.15.133:0/3121683935 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0010c60 con 0x7f28040af310 2024-03-07T06:52:06.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.759+0000 7f280bc00640 1 -- 172.21.15.133:0/3121683935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280415d800 con 0x7f28040af310 2024-03-07T06:52:06.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.759+0000 7f280bc00640 1 -- 172.21.15.133:0/3121683935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280415ddc0 con 0x7f28040af310 2024-03-07T06:52:06.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.760+0000 7f280bc00640 1 -- 172.21.15.133:0/3121683935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2804072b00 con 0x7f28040af310 2024-03-07T06:52:06.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.761+0000 7f2800c00640 1 -- 172.21.15.133:0/3121683935 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f27f0017070 con 0x7f28040af310 2024-03-07T06:52:06.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.762+0000 7f2800c00640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f27e406b4d0 0x7f27e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:06.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.762+0000 7f2800c00640 1 -- 172.21.15.133:0/3121683935 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f27f008a160 con 0x7f28040af310 2024-03-07T06:52:06.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.762+0000 7f2809200640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f27e406b4d0 0x7f27e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:06.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.765+0000 7f2809200640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f27e406b4d0 0x7f27e406d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27f8036000 tx=0x7f27f8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:06.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.767+0000 7f2800c00640 1 -- 172.21.15.133:0/3121683935 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2804072b00 con 0x7f28040af310 2024-03-07T06:52:06.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.961+0000 7f280bc00640 1 -- 172.21.15.133:0/3121683935 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f280414ef10 con 0x7f28040af310 2024-03-07T06:52:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.964+0000 7f2800c00640 1 -- 172.21.15.133:0/3121683935 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16901 (secure 0 0 0) 0x7f27f0058e30 con 0x7f28040af310 2024-03-07T06:52:06.964 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:06.964 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:56.413096+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:52:06.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.966+0000 7f280bc00640 1 -- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f27e406b4d0 msgr2=0x7f27e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:06.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.966+0000 7f280bc00640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f27e406b4d0 0x7f27e406d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27f8036000 tx=0x7f27f8008030 comp rx=0 tx=0).stop 2024-03-07T06:52:06.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.966+0000 7f280bc00640 1 -- 172.21.15.133:0/3121683935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28040af310 msgr2=0x7f2804154c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:06.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.966+0000 7f280bc00640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28040af310 0x7f2804154c20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f27f000b4d0 tx=0x7f27f000b990 comp rx=0 tx=0).stop 2024-03-07T06:52:06.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.966+0000 7f280bc00640 1 -- 172.21.15.133:0/3121683935 shutdown_connections 2024-03-07T06:52:06.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.966+0000 7f280bc00640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f27e406b4d0 0x7f27e406d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:06.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.966+0000 7f280bc00640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28040cdee0 0x7f280415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:06.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.966+0000 7f280bc00640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28040af310 0x7f2804154c20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:06.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.966+0000 7f280bc00640 1 --2- 172.21.15.133:0/3121683935 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2804071610 0x7f28041546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:06.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.966+0000 7f280bc00640 1 -- 172.21.15.133:0/3121683935 >> 172.21.15.133:0/3121683935 conn(0x7f28040b3120 msgr2=0x7f28040b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:06.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.967+0000 7f280bc00640 1 -- 172.21.15.133:0/3121683935 shutdown_connections 2024-03-07T06:52:06.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:06.967+0000 7f280bc00640 1 -- 172.21.15.133:0/3121683935 wait complete. 2024-03-07T06:52:06.986 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:51:12.858333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:51:13.615348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:52:06.986 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:52:07.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.197+0000 7fab3f200640 1 Processor -- start 2024-03-07T06:52:07.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.197+0000 7fab3f200640 1 -- start start 2024-03-07T06:52:07.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.199+0000 7fab3f200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab38145670 0x7fab380cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.199+0000 7fab3f200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fab3815cf90 0x7fab380cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.199+0000 7fab3f200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab3806e150 con 0x7fab380cda20 2024-03-07T06:52:07.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.199+0000 7fab3f200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab3806ed60 con 0x7fab3815cf90 2024-03-07T06:52:07.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.199+0000 7fab3f200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3806e430 con 0x7fab38145670 2024-03-07T06:52:07.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.200+0000 7fab2f400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fab3815cf90 0x7fab380cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.200+0000 7fab37e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab38145670 0x7fab380cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.200+0000 7fab2f400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fab3815cf90 0x7fab380cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:44536/0 (socket says 172.21.15.133:44536) 2024-03-07T06:52:07.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.200+0000 7fab2f400640 1 -- 172.21.15.133:0/4006771999 learned_addr learned my addr 172.21.15.133:0/4006771999 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:07.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.201+0000 7fab2f400640 1 -- 172.21.15.133:0/4006771999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab38145670 msgr2=0x7fab380cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:07.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.201+0000 7fab2f400640 1 --2- 172.21.15.133:0/4006771999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab38145670 0x7fab380cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.201+0000 7fab2f400640 1 -- 172.21.15.133:0/4006771999 >> v1:172.21.15.161:6789/0 conn(0x7fab380cda20 legacy=0x7fab3815fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:07.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.201+0000 7fab2f400640 1 -- 172.21.15.133:0/4006771999 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3806e710 con 0x7fab3815cf90 2024-03-07T06:52:07.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.202+0000 7fab2f400640 1 --2- 172.21.15.133:0/4006771999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab3815cf90 0x7fab380cd4e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fab20009870 tx=0x7fab2002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8efd328f66126a32 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:07.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.203+0000 7fab37400640 1 -- 172.21.15.133:0/4006771999 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20035020 con 0x7fab3815cf90 2024-03-07T06:52:07.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.203+0000 7fab37400640 1 -- 172.21.15.133:0/4006771999 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab20031640 con 0x7fab3815cf90 2024-03-07T06:52:07.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.203+0000 7fab37400640 1 -- 172.21.15.133:0/4006771999 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2002d040 con 0x7fab3815cf90 2024-03-07T06:52:07.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.204+0000 7fab3f200640 1 -- 172.21.15.133:0/4006771999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab3815cf90 msgr2=0x7fab380cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:07.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.204+0000 7fab3f200640 1 --2- 172.21.15.133:0/4006771999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab3815cf90 0x7fab380cd4e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fab20009870 tx=0x7fab2002b8f0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.204+0000 7fab3f200640 1 -- 172.21.15.133:0/4006771999 shutdown_connections 2024-03-07T06:52:07.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.204+0000 7fab3f200640 1 --2- 172.21.15.133:0/4006771999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab3815cf90 0x7fab380cd4e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.204+0000 7fab3f200640 1 --2- 172.21.15.133:0/4006771999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab38145670 0x7fab380cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.204+0000 7fab3f200640 1 -- 172.21.15.133:0/4006771999 >> 172.21.15.133:0/4006771999 conn(0x7fab380b1d30 msgr2=0x7fab380b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:07.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.204+0000 7fab3f200640 1 -- 172.21.15.133:0/4006771999 shutdown_connections 2024-03-07T06:52:07.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.204+0000 7fab3f200640 1 -- 172.21.15.133:0/4006771999 wait complete. 2024-03-07T06:52:07.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.206+0000 7fab3f200640 1 Processor -- start 2024-03-07T06:52:07.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.207+0000 7fab3f200640 1 -- start start 2024-03-07T06:52:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.208+0000 7fab3f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab380cda20 0x7fab381f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.208+0000 7fab3f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab38145670 0x7fab380cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.208+0000 7fab3f200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3815cf90 0x7fab381fb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.208+0000 7fab3f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab38145a50 con 0x7fab380cda20 2024-03-07T06:52:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.208+0000 7fab3f200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3815d3f0 con 0x7fab38145670 2024-03-07T06:52:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.208+0000 7fab3f200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab38145bd0 con 0x7fab3815cf90 2024-03-07T06:52:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.208+0000 7fab37e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab380cda20 0x7fab381f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.208+0000 7fab37e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab380cda20 0x7fab381f88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47474/0 (socket says 172.21.15.133:47474) 2024-03-07T06:52:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.208+0000 7fab37e00640 1 -- 172.21.15.133:0/2671493428 learned_addr learned my addr 172.21.15.133:0/2671493428 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.209+0000 7fab2f400640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab38145670 0x7fab380cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.209+0000 7fab3d200640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3815cf90 0x7fab381fb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.209+0000 7fab37e00640 1 -- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3815cf90 msgr2=0x7fab381fb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:07.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.209+0000 7fab37e00640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3815cf90 0x7fab381fb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.209+0000 7fab37e00640 1 -- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab38145670 msgr2=0x7fab380cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:07.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.209+0000 7fab37e00640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab38145670 0x7fab380cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.209+0000 7fab37e00640 1 -- 172.21.15.133:0/2671493428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab381fbec0 con 0x7fab380cda20 2024-03-07T06:52:07.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.209+0000 7fab2f400640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab38145670 0x7fab380cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:07.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.209+0000 7fab3d200640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3815cf90 0x7fab381fb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:07.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.209+0000 7fab37e00640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab380cda20 0x7fab381f88b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fab2800efc0 tx=0x7fab2800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:07.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.210+0000 7fab34c00640 1 -- 172.21.15.133:0/2671493428 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28018020 con 0x7fab380cda20 2024-03-07T06:52:07.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.210+0000 7fab34c00640 1 -- 172.21.15.133:0/2671493428 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab28014480 con 0x7fab380cda20 2024-03-07T06:52:07.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.210+0000 7fab34c00640 1 -- 172.21.15.133:0/2671493428 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab28014850 con 0x7fab380cda20 2024-03-07T06:52:07.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.210+0000 7fab3f200640 1 -- 172.21.15.133:0/2671493428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab381fc1a0 con 0x7fab380cda20 2024-03-07T06:52:07.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.210+0000 7fab3f200640 1 -- 172.21.15.133:0/2671493428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3813f3b0 con 0x7fab380cda20 2024-03-07T06:52:07.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.211+0000 7fab3f200640 1 -- 172.21.15.133:0/2671493428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab38062c90 con 0x7fab380cda20 2024-03-07T06:52:07.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.212+0000 7fab34c00640 1 -- 172.21.15.133:0/2671493428 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fab28005d10 con 0x7fab380cda20 2024-03-07T06:52:07.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.214+0000 7fab34c00640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fab1406b4d0 0x7fab1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.214+0000 7fab34c00640 1 -- 172.21.15.133:0/2671493428 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fab2808ad90 con 0x7fab380cda20 2024-03-07T06:52:07.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.214+0000 7fab2f400640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fab1406b4d0 0x7fab1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.215+0000 7fab2f400640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fab1406b4d0 0x7fab1406d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fab20036000 tx=0x7fab2003c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:07.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.219+0000 7fab34c00640 1 -- 172.21.15.133:0/2671493428 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fab28022070 con 0x7fab380cda20 2024-03-07T06:52:07.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.415+0000 7fab3f200640 1 -- 172.21.15.133:0/2671493428 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab380cf610 con 0x7fab380cda20 2024-03-07T06:52:07.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.416+0000 7fab34c00640 1 -- 172.21.15.133:0/2671493428 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab28010040 con 0x7fab380cda20 2024-03-07T06:52:07.416 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:52:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.418+0000 7fab3f200640 1 -- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fab1406b4d0 msgr2=0x7fab1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.418+0000 7fab3f200640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fab1406b4d0 0x7fab1406d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fab20036000 tx=0x7fab2003c040 comp rx=0 tx=0).stop 2024-03-07T06:52:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.418+0000 7fab3f200640 1 -- 172.21.15.133:0/2671493428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab380cda20 msgr2=0x7fab381f88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.418+0000 7fab3f200640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab380cda20 0x7fab381f88b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fab2800efc0 tx=0x7fab2800c490 comp rx=0 tx=0).stop 2024-03-07T06:52:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.419+0000 7fab3f200640 1 -- 172.21.15.133:0/2671493428 shutdown_connections 2024-03-07T06:52:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.419+0000 7fab3f200640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fab3815cf90 0x7fab381fb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.419+0000 7fab3f200640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fab1406b4d0 0x7fab1406d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.419+0000 7fab3f200640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fab38145670 0x7fab380cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.419+0000 7fab3f200640 1 --2- 172.21.15.133:0/2671493428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab380cda20 0x7fab381f88b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.419+0000 7fab3f200640 1 -- 172.21.15.133:0/2671493428 >> 172.21.15.133:0/2671493428 conn(0x7fab380b1d30 msgr2=0x7fab38144950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:07.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.419+0000 7fab3f200640 1 -- 172.21.15.133:0/2671493428 shutdown_connections 2024-03-07T06:52:07.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.419+0000 7fab3f200640 1 -- 172.21.15.133:0/2671493428 wait complete. 2024-03-07T06:52:07.436 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:52:07.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.654+0000 7fb5d3600640 1 Processor -- start 2024-03-07T06:52:07.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.654+0000 7fb5d3600640 1 -- start start 2024-03-07T06:52:07.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5d3600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb5cc162350 0x7fb5cc0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5d3600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5cc06e680 0x7fb5cc0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5d3600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5cc072310 con 0x7fb5cc0b4270 2024-03-07T06:52:07.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5d3600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5cc0725f0 con 0x7fb5cc162350 2024-03-07T06:52:07.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5d3600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc0728d0 con 0x7fb5cc06e680 2024-03-07T06:52:07.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cbe00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb5cc0b4270 0x7fb5cc06df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47834/0 (socket says 172.21.15.133:47834) 2024-03-07T06:52:07.655 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cbe00640 1 -- 172.21.15.133:0/3487441458 learned_addr learned my addr 172.21.15.133:0/3487441458 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5d1600640 1 --2- 172.21.15.133:0/3487441458 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5cc06e680 0x7fb5cc0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5d0c00640 1 --2- 172.21.15.133:0/3487441458 >> v2:172.21.15.161:3300/0 conn(0x7fb5cc162350 0x7fb5cc0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 -- 172.21.15.133:0/3487441458 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390735041 0 0) 0x7fb5cc072310 con 0x7fb5cc0b4270 2024-03-07T06:52:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 -- 172.21.15.133:0/3487441458 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5b4003520 con 0x7fb5cc0b4270 2024-03-07T06:52:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 -- 172.21.15.133:0/3487441458 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1368842390 0 0) 0x7fb5b4003520 con 0x7fb5cc0b4270 2024-03-07T06:52:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 -- 172.21.15.133:0/3487441458 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5cc06e680 msgr2=0x7fb5cc0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 --2- 172.21.15.133:0/3487441458 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5cc06e680 0x7fb5cc0cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 -- 172.21.15.133:0/3487441458 >> v2:172.21.15.161:3300/0 conn(0x7fb5cc162350 msgr2=0x7fb5cc0719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 --2- 172.21.15.133:0/3487441458 >> v2:172.21.15.161:3300/0 conn(0x7fb5cc162350 0x7fb5cc0719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 -- 172.21.15.133:0/3487441458 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc072bb0 con 0x7fb5cc0b4270 2024-03-07T06:52:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 -- 172.21.15.133:0/3487441458 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fb5bc001670 con 0x7fb5cc0b4270 2024-03-07T06:52:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 -- 172.21.15.133:0/3487441458 >> v1:172.21.15.133:6789/0 conn(0x7fb5cc0b4270 legacy=0x7fb5cc06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 --2- 172.21.15.133:0/3487441458 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5cc06e680 0x7fb5b40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 --2- 172.21.15.133:0/3487441458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5b4003f80 0x7fb5b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 --2- 172.21.15.133:0/3487441458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b4004d40 0x7fb5b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 -- 172.21.15.133:0/3487441458 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc0725f0 con 0x7fb5b4004d40 2024-03-07T06:52:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 -- 172.21.15.133:0/3487441458 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc0728d0 con 0x7fb5b4003f80 2024-03-07T06:52:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5cb400640 1 -- 172.21.15.133:0/3487441458 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b400b9c0 con 0x7fb5cc06e680 2024-03-07T06:52:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.656+0000 7fb5d1600640 1 --2- 172.21.15.133:0/3487441458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b4004d40 0x7fb5b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.657+0000 7fb5cbe00640 1 --2- 172.21.15.133:0/3487441458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5b4003f80 0x7fb5b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.657+0000 7fb5d0c00640 1 --2- 172.21.15.133:0/3487441458 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5cc06e680 0x7fb5b40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.657+0000 7fb5d1600640 1 -- 172.21.15.133:0/3487441458 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5cc06e680 msgr2=0x7fb5b40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.657+0000 7fb5d1600640 1 --2- 172.21.15.133:0/3487441458 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5cc06e680 0x7fb5b40039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.657+0000 7fb5d1600640 1 -- 172.21.15.133:0/3487441458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5b4003f80 msgr2=0x7fb5b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.657+0000 7fb5d1600640 1 --2- 172.21.15.133:0/3487441458 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5b4003f80 0x7fb5b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.657+0000 7fb5d1600640 1 -- 172.21.15.133:0/3487441458 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b4003520 con 0x7fb5b4004d40 2024-03-07T06:52:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.658+0000 7fb5d1600640 1 --2- 172.21.15.133:0/3487441458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b4004d40 0x7fb5b40051c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009f90 tx=0x7fb5b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86aeb9cfd7993a2c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.658+0000 7fb5cb400640 1 -- 172.21.15.133:0/3487441458 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4039070 con 0x7fb5b4004d40 2024-03-07T06:52:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.658+0000 7fb5cb400640 1 -- 172.21.15.133:0/3487441458 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c402cb90 con 0x7fb5b4004d40 2024-03-07T06:52:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.658+0000 7fb5cb400640 1 -- 172.21.15.133:0/3487441458 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4009c10 con 0x7fb5b4004d40 2024-03-07T06:52:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.659+0000 7fb5d3600640 1 -- 172.21.15.133:0/3487441458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b4004d40 msgr2=0x7fb5b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.659+0000 7fb5d3600640 1 --2- 172.21.15.133:0/3487441458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b4004d40 0x7fb5b40051c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4009f90 tx=0x7fb5b4005830 comp rx=0 tx=0).stop 2024-03-07T06:52:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.659+0000 7fb5d1600640 1 -- 172.21.15.133:0/3487441458 reap_dead start 2024-03-07T06:52:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.659+0000 7fb5d3600640 1 -- 172.21.15.133:0/3487441458 shutdown_connections 2024-03-07T06:52:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.659+0000 7fb5d3600640 1 -- 172.21.15.133:0/3487441458 >> 172.21.15.133:0/3487441458 conn(0x7fb5cc0b1d30 msgr2=0x7fb5cc0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.659+0000 7fb5d3600640 1 -- 172.21.15.133:0/3487441458 shutdown_connections 2024-03-07T06:52:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.659+0000 7fb5d3600640 1 -- 172.21.15.133:0/3487441458 wait complete. 2024-03-07T06:52:07.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.661+0000 7fb5d3600640 1 Processor -- start 2024-03-07T06:52:07.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.661+0000 7fb5d3600640 1 -- start start 2024-03-07T06:52:07.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.662+0000 7fb5d3600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5b4004d40 0x7fb5cc146950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.662+0000 7fb5d3600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc145ee0 0x7fb5cc15e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.665 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.662+0000 7fb5d3600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5cc164840 0x7fb5cc164cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.662+0000 7fb5d3600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc072060 con 0x7fb5cc164840 2024-03-07T06:52:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.662+0000 7fb5d3600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc071ee0 con 0x7fb5cc145ee0 2024-03-07T06:52:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.662+0000 7fb5d3600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc162730 con 0x7fb5b4004d40 2024-03-07T06:52:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.662+0000 7fb5d1600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5cc164840 0x7fb5cc164cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.662+0000 7fb5d1600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5cc164840 0x7fb5cc164cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47500/0 (socket says 172.21.15.133:47500) 2024-03-07T06:52:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.662+0000 7fb5d1600640 1 -- 172.21.15.133:0/374363535 learned_addr learned my addr 172.21.15.133:0/374363535 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.662+0000 7fb5d0c00640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5b4004d40 0x7fb5cc146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.662+0000 7fb5cbe00640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc145ee0 0x7fb5cc15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.662+0000 7fb5d1600640 1 -- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5b4004d40 msgr2=0x7fb5cc146950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.662+0000 7fb5d1600640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5b4004d40 0x7fb5cc146950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.663+0000 7fb5d1600640 1 -- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc145ee0 msgr2=0x7fb5cc15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.663+0000 7fb5d1600640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc145ee0 0x7fb5cc15e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.663+0000 7fb5d1600640 1 -- 172.21.15.133:0/374363535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc15ec30 con 0x7fb5cc164840 2024-03-07T06:52:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.663+0000 7fb5cbe00640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc145ee0 0x7fb5cc15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.663+0000 7fb5d1600640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5cc164840 0x7fb5cc164cc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4037040 tx=0x7fb5c4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.663+0000 7fb5c8c00640 1 -- 172.21.15.133:0/374363535 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4039030 con 0x7fb5cc164840 2024-03-07T06:52:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.663+0000 7fb5c8c00640 1 -- 172.21.15.133:0/374363535 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c403a040 con 0x7fb5cc164840 2024-03-07T06:52:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.663+0000 7fb5c8c00640 1 -- 172.21.15.133:0/374363535 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c4003990 con 0x7fb5cc164840 2024-03-07T06:52:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.663+0000 7fb5d3600640 1 -- 172.21.15.133:0/374363535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5cc15ef10 con 0x7fb5cc164840 2024-03-07T06:52:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.663+0000 7fb5d3600640 1 -- 172.21.15.133:0/374363535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5cc15fe80 con 0x7fb5cc164840 2024-03-07T06:52:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.665+0000 7fb5d3600640 1 -- 172.21.15.133:0/374363535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5cc062c90 con 0x7fb5cc164840 2024-03-07T06:52:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.665+0000 7fb5c8c00640 1 -- 172.21.15.133:0/374363535 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb5c4045020 con 0x7fb5cc164840 2024-03-07T06:52:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.667+0000 7fb5c8c00640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb5b006b500 0x7fb5b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.667+0000 7fb5c8c00640 1 -- 172.21.15.133:0/374363535 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb5c40ac120 con 0x7fb5cc164840 2024-03-07T06:52:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.667+0000 7fb5d0c00640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb5b006b500 0x7fb5b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.669+0000 7fb5d0c00640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb5b006b500 0x7fb5b006d9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009a90 tx=0x7fb5c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:07.671 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.671+0000 7fb5c8c00640 1 -- 172.21.15.133:0/374363535 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb5c407ae80 con 0x7fb5cc164840 2024-03-07T06:52:07.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.867+0000 7fb5d3600640 1 -- 172.21.15.133:0/374363535 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5cc0b3a60 con 0x7fb5cc164840 2024-03-07T06:52:07.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.867+0000 7fb5c8c00640 1 -- 172.21.15.133:0/374363535 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb5c407a780 con 0x7fb5cc164840 2024-03-07T06:52:07.866 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:52:07.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.869+0000 7fb5d3600640 1 -- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb5b006b500 msgr2=0x7fb5b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:07.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.869+0000 7fb5d3600640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb5b006b500 0x7fb5b006d9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009a90 tx=0x7fb5c0008040 comp rx=0 tx=0).stop 2024-03-07T06:52:07.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.869+0000 7fb5d3600640 1 -- 172.21.15.133:0/374363535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5cc164840 msgr2=0x7fb5cc164cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:07.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.869+0000 7fb5d3600640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5cc164840 0x7fb5cc164cc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4037040 tx=0x7fb5c4009750 comp rx=0 tx=0).stop 2024-03-07T06:52:07.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.869+0000 7fb5d3600640 1 -- 172.21.15.133:0/374363535 shutdown_connections 2024-03-07T06:52:07.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.869+0000 7fb5d3600640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5cc164840 0x7fb5cc164cc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.869+0000 7fb5d3600640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb5b006b500 0x7fb5b006d9c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.869+0000 7fb5d3600640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb5cc145ee0 0x7fb5cc15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.869+0000 7fb5d3600640 1 --2- 172.21.15.133:0/374363535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5b4004d40 0x7fb5cc146950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:07.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.869+0000 7fb5d3600640 1 -- 172.21.15.133:0/374363535 >> 172.21.15.133:0/374363535 conn(0x7fb5cc0b1d30 msgr2=0x7fb5cc06d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:07.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.869+0000 7fb5d3600640 1 -- 172.21.15.133:0/374363535 shutdown_connections 2024-03-07T06:52:07.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:07.869+0000 7fb5d3600640 1 -- 172.21.15.133:0/374363535 wait complete. 2024-03-07T06:52:07.887 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:52:08.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.100+0000 7f9ec5a00640 1 Processor -- start 2024-03-07T06:52:08.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.100+0000 7f9ec5a00640 1 -- start start 2024-03-07T06:52:08.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.102+0000 7f9ec5a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9ec015b370 0x7f9ec00cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:08.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.102+0000 7f9ec5a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9ec006d180 0x7f9ec00cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:08.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.102+0000 7f9ec5a00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec006d740 con 0x7f9ec015b750 2024-03-07T06:52:08.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.102+0000 7f9ec5a00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec006da20 con 0x7f9ec006d180 2024-03-07T06:52:08.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.102+0000 7f9ec5a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec006dd00 con 0x7f9ec015b370 2024-03-07T06:52:08.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.102+0000 7f9ebe000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9ec006d180 0x7f9ec00cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:08.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.102+0000 7f9ebf400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9ec015b750 0x7f9ec015bc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47836/0 (socket says 172.21.15.133:47836) 2024-03-07T06:52:08.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.102+0000 7f9ebf400640 1 -- 172.21.15.133:0/3862242034 learned_addr learned my addr 172.21.15.133:0/3862242034 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:08.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.103+0000 7f9ebea00640 1 --2- 172.21.15.133:0/3862242034 >> v2:172.21.15.161:3300/0 conn(0x7f9ec015b370 0x7f9ec00cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:08.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.103+0000 7f9ebe000640 1 -- 172.21.15.133:0/3862242034 >> v2:172.21.15.161:3300/0 conn(0x7f9ec015b370 msgr2=0x7f9ec00cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:08.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.103+0000 7f9ebe000640 1 --2- 172.21.15.133:0/3862242034 >> v2:172.21.15.161:3300/0 conn(0x7f9ec015b370 0x7f9ec00cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.103+0000 7f9ebe000640 1 -- 172.21.15.133:0/3862242034 >> v1:172.21.15.133:6789/0 conn(0x7f9ec015b750 legacy=0x7f9ec015bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:08.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.103+0000 7f9ebe000640 1 -- 172.21.15.133:0/3862242034 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec006dfe0 con 0x7f9ec006d180 2024-03-07T06:52:08.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.103+0000 7f9ebd600640 1 -- 172.21.15.133:0/3862242034 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756328810 0 0) 0x7f9ec006d740 con 0x7f9ec015b750 2024-03-07T06:52:08.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.104+0000 7f9ebe000640 1 --2- 172.21.15.133:0/3862242034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec006d180 0x7f9ec00cf490 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0073540 tx=0x7f9ea802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfb3bae9a2cefb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:08.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.104+0000 7f9ebd600640 1 -- 172.21.15.133:0/3862242034 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8039070 con 0x7f9ec006d180 2024-03-07T06:52:08.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.104+0000 7f9ebd600640 1 -- 172.21.15.133:0/3862242034 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea802cc80 con 0x7f9ec006d180 2024-03-07T06:52:08.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.104+0000 7f9ebd600640 1 -- 172.21.15.133:0/3862242034 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea8034660 con 0x7f9ec006d180 2024-03-07T06:52:08.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.105+0000 7f9ec5a00640 1 -- 172.21.15.133:0/3862242034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec006d180 msgr2=0x7f9ec00cf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:08.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.105+0000 7f9ec5a00640 1 --2- 172.21.15.133:0/3862242034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec006d180 0x7f9ec00cf490 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0073540 tx=0x7f9ea802bed0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.105+0000 7f9ec5a00640 1 -- 172.21.15.133:0/3862242034 shutdown_connections 2024-03-07T06:52:08.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.105+0000 7f9ec5a00640 1 --2- 172.21.15.133:0/3862242034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec006d180 0x7f9ec00cf490 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.105+0000 7f9ec5a00640 1 --2- 172.21.15.133:0/3862242034 >> v2:172.21.15.161:3300/0 conn(0x7f9ec015b370 0x7f9ec00cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.105+0000 7f9ec5a00640 1 -- 172.21.15.133:0/3862242034 >> 172.21.15.133:0/3862242034 conn(0x7f9ec00b1d30 msgr2=0x7f9ec00b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:08.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.105+0000 7f9ec5a00640 1 -- 172.21.15.133:0/3862242034 shutdown_connections 2024-03-07T06:52:08.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.105+0000 7f9ec5a00640 1 -- 172.21.15.133:0/3862242034 wait complete. 2024-03-07T06:52:08.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.107+0000 7f9ec5a00640 1 Processor -- start 2024-03-07T06:52:08.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.107+0000 7f9ec5a00640 1 -- start start 2024-03-07T06:52:08.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.108+0000 7f9ec5a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec006d180 0x7f9ec01f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:08.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.108+0000 7f9ec5a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ec015b370 0x7f9ec0143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:08.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.108+0000 7f9ec5a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ec015b750 0x7f9ec0144510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:08.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.108+0000 7f9ec5a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec00b4050 con 0x7f9ec006d180 2024-03-07T06:52:08.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.108+0000 7f9ec5a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec00b3ed0 con 0x7f9ec015b370 2024-03-07T06:52:08.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.108+0000 7f9ec5a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec00b41d0 con 0x7f9ec015b750 2024-03-07T06:52:08.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.108+0000 7f9ebea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec006d180 0x7f9ec01f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:08.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.108+0000 7f9ebea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec006d180 0x7f9ec01f8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47518/0 (socket says 172.21.15.133:47518) 2024-03-07T06:52:08.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.108+0000 7f9ebea00640 1 -- 172.21.15.133:0/395377369 learned_addr learned my addr 172.21.15.133:0/395377369 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:08.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.109+0000 7f9ebf400640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ec015b750 0x7f9ec0144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:08.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.109+0000 7f9ebe000640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ec015b370 0x7f9ec0143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:08.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.109+0000 7f9ebea00640 1 -- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ec015b750 msgr2=0x7f9ec0144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:08.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.109+0000 7f9ebea00640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ec015b750 0x7f9ec0144510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.109+0000 7f9ebea00640 1 -- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ec015b370 msgr2=0x7f9ec0143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:08.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.109+0000 7f9ebea00640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ec015b370 0x7f9ec0143fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.109+0000 7f9ebea00640 1 -- 172.21.15.133:0/395377369 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec0144a50 con 0x7f9ec006d180 2024-03-07T06:52:08.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.109+0000 7f9ebf400640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ec015b750 0x7f9ec0144510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:08.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.109+0000 7f9ebe000640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ec015b370 0x7f9ec0143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:08.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.109+0000 7f9ebea00640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec006d180 0x7f9ec01f8d00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9eb000e9b0 tx=0x7f9eb000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:08.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.109+0000 7f9eb6a00640 1 -- 172.21.15.133:0/395377369 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0017070 con 0x7f9ec006d180 2024-03-07T06:52:08.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.109+0000 7f9eb6a00640 1 -- 172.21.15.133:0/395377369 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb000c9a0 con 0x7f9ec006d180 2024-03-07T06:52:08.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.109+0000 7f9ec5a00640 1 -- 172.21.15.133:0/395377369 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec0144ce0 con 0x7f9ec006d180 2024-03-07T06:52:08.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.109+0000 7f9eb6a00640 1 -- 172.21.15.133:0/395377369 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb000cd90 con 0x7f9ec006d180 2024-03-07T06:52:08.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.110+0000 7f9ec5a00640 1 -- 172.21.15.133:0/395377369 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec01525c0 con 0x7f9ec006d180 2024-03-07T06:52:08.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.111+0000 7f9ec5a00640 1 -- 172.21.15.133:0/395377369 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec0062c90 con 0x7f9ec006d180 2024-03-07T06:52:08.110 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.112+0000 7f9eb6a00640 1 -- 172.21.15.133:0/395377369 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9eb001a070 con 0x7f9ec006d180 2024-03-07T06:52:08.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.113+0000 7f9eb6a00640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9e9c06b4d0 0x7f9e9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:08.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.113+0000 7f9eb6a00640 1 -- 172.21.15.133:0/395377369 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9eb008a230 con 0x7f9ec006d180 2024-03-07T06:52:08.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.114+0000 7f9ebe000640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9e9c06b4d0 0x7f9e9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:08.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.115+0000 7f9ebe000640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9e9c06b4d0 0x7f9e9c06d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8036000 tx=0x7f9ea8030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:08.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.117+0000 7f9eb6a00640 1 -- 172.21.15.133:0/395377369 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9eb00590c0 con 0x7f9ec006d180 2024-03-07T06:52:08.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.297+0000 7f9ec5a00640 1 -- 172.21.15.133:0/395377369 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ec0150580 con 0x7f9ec006d180 2024-03-07T06:52:08.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.298+0000 7f9eb6a00640 1 -- 172.21.15.133:0/395377369 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9eb00589c0 con 0x7f9ec006d180 2024-03-07T06:52:08.296 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:52:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.300+0000 7f9ec5a00640 1 -- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9e9c06b4d0 msgr2=0x7f9e9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.300+0000 7f9ec5a00640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9e9c06b4d0 0x7f9e9c06d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8036000 tx=0x7f9ea8030000 comp rx=0 tx=0).stop 2024-03-07T06:52:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.300+0000 7f9ec5a00640 1 -- 172.21.15.133:0/395377369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec006d180 msgr2=0x7f9ec01f8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.300+0000 7f9ec5a00640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec006d180 0x7f9ec01f8d00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9eb000e9b0 tx=0x7f9eb000ee70 comp rx=0 tx=0).stop 2024-03-07T06:52:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.300+0000 7f9ec5a00640 1 -- 172.21.15.133:0/395377369 shutdown_connections 2024-03-07T06:52:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.300+0000 7f9ec5a00640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ec015b750 0x7f9ec0144510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.300+0000 7f9ec5a00640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9e9c06b4d0 0x7f9e9c06d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.300+0000 7f9ec5a00640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ec015b370 0x7f9ec0143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.300+0000 7f9ec5a00640 1 --2- 172.21.15.133:0/395377369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ec006d180 0x7f9ec01f8d00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.300+0000 7f9ec5a00640 1 -- 172.21.15.133:0/395377369 >> 172.21.15.133:0/395377369 conn(0x7f9ec00b1d30 msgr2=0x7f9ec0072680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:08.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.300+0000 7f9ec5a00640 1 -- 172.21.15.133:0/395377369 shutdown_connections 2024-03-07T06:52:08.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.300+0000 7f9ec5a00640 1 -- 172.21.15.133:0/395377369 wait complete. 2024-03-07T06:52:08.317 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-07T06:52:08.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.521+0000 7f61b7000640 1 Processor -- start 2024-03-07T06:52:08.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.521+0000 7f61b7000640 1 -- start start 2024-03-07T06:52:08.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.522+0000 7f61b7000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61b0072950 0x7f61b00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.522+0000 7f61b7000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61b00cdee0 0x7f61b015c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.522+0000 7f61b7000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b00731b0 con 0x7f61b015fbb0 2024-03-07T06:52:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.522+0000 7f61b7000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b0072f60 con 0x7f61b00cdee0 2024-03-07T06:52:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.522+0000 7f61b7000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b0073490 con 0x7f61b0072950 2024-03-07T06:52:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.523+0000 7f61b5000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61b00cdee0 0x7f61b015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.523+0000 7f61b5000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f61b00cdee0 0x7f61b015c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47534/0 (socket says 172.21.15.133:47534) 2024-03-07T06:52:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.523+0000 7f61b5000640 1 -- 172.21.15.133:0/152717074 learned_addr learned my addr 172.21.15.133:0/152717074 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.523+0000 7f61af400640 1 --2- 172.21.15.133:0/152717074 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61b0072950 0x7f61b00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.523+0000 7f61b5000640 1 -- 172.21.15.133:0/152717074 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61b0072950 msgr2=0x7f61b00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.523+0000 7f61b5000640 1 --2- 172.21.15.133:0/152717074 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61b0072950 0x7f61b00cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.523+0000 7f61b5000640 1 -- 172.21.15.133:0/152717074 >> v1:172.21.15.161:6789/0 conn(0x7f61b015fbb0 legacy=0x7f61b015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:08.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.523+0000 7f61b5000640 1 -- 172.21.15.133:0/152717074 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b0072150 con 0x7f61b00cdee0 2024-03-07T06:52:08.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.524+0000 7f61b5000640 1 --2- 172.21.15.133:0/152717074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61b00cdee0 0x7f61b015c550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f61b00730e0 tx=0x7f61a802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adabbb5c94873d92 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:08.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.525+0000 7f61aea00640 1 -- 172.21.15.133:0/152717074 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a80317c0 con 0x7f61b00cdee0 2024-03-07T06:52:08.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.525+0000 7f61aea00640 1 -- 172.21.15.133:0/152717074 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a8037070 con 0x7f61b00cdee0 2024-03-07T06:52:08.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.525+0000 7f61aea00640 1 -- 172.21.15.133:0/152717074 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a8031b80 con 0x7f61b00cdee0 2024-03-07T06:52:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.525+0000 7f61b7000640 1 -- 172.21.15.133:0/152717074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61b00cdee0 msgr2=0x7f61b015c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.525+0000 7f61b7000640 1 --2- 172.21.15.133:0/152717074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61b00cdee0 0x7f61b015c550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f61b00730e0 tx=0x7f61a802b860 comp rx=0 tx=0).stop 2024-03-07T06:52:08.524 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.525+0000 7f61b7000640 1 -- 172.21.15.133:0/152717074 shutdown_connections 2024-03-07T06:52:08.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.525+0000 7f61b7000640 1 --2- 172.21.15.133:0/152717074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61b00cdee0 0x7f61b015c550 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.525+0000 7f61b7000640 1 --2- 172.21.15.133:0/152717074 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61b0072950 0x7f61b00cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.525+0000 7f61b7000640 1 -- 172.21.15.133:0/152717074 >> 172.21.15.133:0/152717074 conn(0x7f61b00b3120 msgr2=0x7f61b00b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:08.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.526+0000 7f61b7000640 1 -- 172.21.15.133:0/152717074 shutdown_connections 2024-03-07T06:52:08.525 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.526+0000 7f61b7000640 1 -- 172.21.15.133:0/152717074 wait complete. 2024-03-07T06:52:08.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.527+0000 7f61b7000640 1 Processor -- start 2024-03-07T06:52:08.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.527+0000 7f61b7000640 1 -- start start 2024-03-07T06:52:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.528+0000 7f61b7000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61b00cdee0 0x7f61b0150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.528+0000 7f61b7000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61b015fbb0 0x7f61b006e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.528+0000 7f61b7000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61b00b02c0 0x7f61b00b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.528+0000 7f61b7000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b0160980 con 0x7f61b00cdee0 2024-03-07T06:52:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.528+0000 7f61b7000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b00ce2f0 con 0x7f61b015fbb0 2024-03-07T06:52:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.528+0000 7f61b7000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b00b1a00 con 0x7f61b00b02c0 2024-03-07T06:52:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.529+0000 7f61afe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61b00cdee0 0x7f61b0150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.529+0000 7f61afe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61b00cdee0 0x7f61b0150c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47544/0 (socket says 172.21.15.133:47544) 2024-03-07T06:52:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.529+0000 7f61afe00640 1 -- 172.21.15.133:0/2322682979 learned_addr learned my addr 172.21.15.133:0/2322682979 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.529+0000 7f61b5000640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61b00b02c0 0x7f61b00b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.529+0000 7f61af400640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61b015fbb0 0x7f61b006e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.529+0000 7f61afe00640 1 -- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61b00b02c0 msgr2=0x7f61b00b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:08.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.529+0000 7f61afe00640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61b00b02c0 0x7f61b00b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.529+0000 7f61afe00640 1 -- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61b015fbb0 msgr2=0x7f61b006e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:08.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.529+0000 7f61afe00640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61b015fbb0 0x7f61b006e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.529+0000 7f61afe00640 1 -- 172.21.15.133:0/2322682979 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b00ca190 con 0x7f61b00cdee0 2024-03-07T06:52:08.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.529+0000 7f61b5000640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61b00b02c0 0x7f61b00b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:08.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.529+0000 7f61af400640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61b015fbb0 0x7f61b006e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:08.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.529+0000 7f61afe00640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61b00cdee0 0x7f61b0150c70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f61a000c7f0 tx=0x7f61a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:08.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.530+0000 7f61a7e00640 1 -- 172.21.15.133:0/2322682979 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a0016020 con 0x7f61b00cdee0 2024-03-07T06:52:08.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.530+0000 7f61a7e00640 1 -- 172.21.15.133:0/2322682979 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a000aa50 con 0x7f61b00cdee0 2024-03-07T06:52:08.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.530+0000 7f61b7000640 1 -- 172.21.15.133:0/2322682979 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b006d2a0 con 0x7f61b00cdee0 2024-03-07T06:52:08.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.530+0000 7f61a7e00640 1 -- 172.21.15.133:0/2322682979 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a000ae40 con 0x7f61b00cdee0 2024-03-07T06:52:08.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.530+0000 7f61b7000640 1 -- 172.21.15.133:0/2322682979 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b006d860 con 0x7f61b00cdee0 2024-03-07T06:52:08.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.531+0000 7f61b7000640 1 -- 172.21.15.133:0/2322682979 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b0062c90 con 0x7f61b00cdee0 2024-03-07T06:52:08.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.532+0000 7f61a7e00640 1 -- 172.21.15.133:0/2322682979 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f61a0019020 con 0x7f61b00cdee0 2024-03-07T06:52:08.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.533+0000 7f61a7e00640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f618c06b4d0 0x7f618c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:08.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.533+0000 7f61a7e00640 1 -- 172.21.15.133:0/2322682979 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f61a008a2c0 con 0x7f61b00cdee0 2024-03-07T06:52:08.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.534+0000 7f61af400640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f618c06b4d0 0x7f618c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:08.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.534+0000 7f61af400640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f618c06b4d0 0x7f618c06d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6198006fd0 tx=0x7f6198008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:08.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.538+0000 7f61a7e00640 1 -- 172.21.15.133:0/2322682979 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f61a0020030 con 0x7f61b00cdee0 2024-03-07T06:52:08.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.745+0000 7f61b7000640 1 -- 172.21.15.133:0/2322682979 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f61b015b320 con 0x7f61b00cdee0 2024-03-07T06:52:08.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.746+0000 7f61a7e00640 1 -- 172.21.15.133:0/2322682979 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f61a00590b0 con 0x7f61b00cdee0 2024-03-07T06:52:08.744 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-03-07T06:52:08.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.748+0000 7f61b7000640 1 -- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f618c06b4d0 msgr2=0x7f618c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.748+0000 7f61b7000640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f618c06b4d0 0x7f618c06d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6198006fd0 tx=0x7f6198008040 comp rx=0 tx=0).stop 2024-03-07T06:52:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.749+0000 7f61b7000640 1 -- 172.21.15.133:0/2322682979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61b00cdee0 msgr2=0x7f61b0150c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.749+0000 7f61b7000640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61b00cdee0 0x7f61b0150c70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f61a000c7f0 tx=0x7f61a000ccb0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.749+0000 7f61b7000640 1 -- 172.21.15.133:0/2322682979 shutdown_connections 2024-03-07T06:52:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.749+0000 7f61b7000640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f61b00b02c0 0x7f61b00b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.749+0000 7f61b7000640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f618c06b4d0 0x7f618c06d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.749+0000 7f61b7000640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f61b015fbb0 0x7f61b006e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.749+0000 7f61b7000640 1 --2- 172.21.15.133:0/2322682979 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61b00cdee0 0x7f61b0150c70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.749+0000 7f61b7000640 1 -- 172.21.15.133:0/2322682979 >> 172.21.15.133:0/2322682979 conn(0x7f61b00b3120 msgr2=0x7f61b00cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.749+0000 7f61b7000640 1 -- 172.21.15.133:0/2322682979 shutdown_connections 2024-03-07T06:52:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:08.749+0000 7f61b7000640 1 -- 172.21.15.133:0/2322682979 wait complete. 2024-03-07T06:52:08.766 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-07T06:52:08.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:08.766 DEBUG:teuthology.orchestra.run.smithi133:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-07T06:52:08.866 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:52:08.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:08.866 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-07T06:52:08.916 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:52:08.916 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:52:08.972 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:52:08.972 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:52:09.023 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:52:09.024 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:52:09.075 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:52:09.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.294+0000 7f961a400640 1 Processor -- start 2024-03-07T06:52:09.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.294+0000 7f961a400640 1 -- start start 2024-03-07T06:52:09.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.295+0000 7f961a400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9614140cb0 0x7f9614162290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f961a400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96140b70d0 0x7f96140b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f961a400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96140b9300 0x7f96140b9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f961a400640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96141435c0 con 0x7f96140b70d0 2024-03-07T06:52:09.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f961a400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9614144420 con 0x7f9614140cb0 2024-03-07T06:52:09.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f961a400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9614143810 con 0x7f96140b9300 2024-03-07T06:52:09.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f960be00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96140b70d0 0x7f96140b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f960be00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f96140b70d0 0x7f96140b7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47550/0 (socket says 172.21.15.133:47550) 2024-03-07T06:52:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f960be00640 1 -- 172.21.15.133:0/4105470551 learned_addr learned my addr 172.21.15.133:0/4105470551 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f9619000640 1 --2- 172.21.15.133:0/4105470551 >> v2:172.21.15.161:3300/0 conn(0x7f9614140cb0 0x7f9614162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f960be00640 1 -- 172.21.15.133:0/4105470551 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96140b9300 msgr2=0x7f96140b9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f9619a00640 1 --2- 172.21.15.133:0/4105470551 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96140b9300 0x7f96140b9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f960be00640 1 --2- 172.21.15.133:0/4105470551 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96140b9300 0x7f96140b9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f960be00640 1 -- 172.21.15.133:0/4105470551 >> v2:172.21.15.161:3300/0 conn(0x7f9614140cb0 msgr2=0x7f9614162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f960be00640 1 --2- 172.21.15.133:0/4105470551 >> v2:172.21.15.161:3300/0 conn(0x7f9614140cb0 0x7f9614162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f960be00640 1 -- 172.21.15.133:0/4105470551 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9614143af0 con 0x7f96140b70d0 2024-03-07T06:52:09.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.296+0000 7f9619000640 1 --2- 172.21.15.133:0/4105470551 >> v2:172.21.15.161:3300/0 conn(0x7f9614140cb0 0x7f9614162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:09.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.297+0000 7f960be00640 1 --2- 172.21.15.133:0/4105470551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96140b70d0 0x7f96140b7560 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f96140d3ca0 tx=0x7f95fc02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f117fccd77500557 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:09.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.298+0000 7f9613e00640 1 -- 172.21.15.133:0/4105470551 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc035020 con 0x7f96140b70d0 2024-03-07T06:52:09.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.298+0000 7f9613e00640 1 -- 172.21.15.133:0/4105470551 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc02d040 con 0x7f96140b70d0 2024-03-07T06:52:09.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.298+0000 7f9613e00640 1 -- 172.21.15.133:0/4105470551 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc031a80 con 0x7f96140b70d0 2024-03-07T06:52:09.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.298+0000 7f961a400640 1 -- 172.21.15.133:0/4105470551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96140b70d0 msgr2=0x7f96140b7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:09.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.298+0000 7f961a400640 1 --2- 172.21.15.133:0/4105470551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96140b70d0 0x7f96140b7560 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f96140d3ca0 tx=0x7f95fc02b900 comp rx=0 tx=0).stop 2024-03-07T06:52:09.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.298+0000 7f961a400640 1 -- 172.21.15.133:0/4105470551 shutdown_connections 2024-03-07T06:52:09.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.298+0000 7f961a400640 1 --2- 172.21.15.133:0/4105470551 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96140b9300 0x7f96140b9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.298+0000 7f961a400640 1 --2- 172.21.15.133:0/4105470551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96140b70d0 0x7f96140b7560 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.298+0000 7f961a400640 1 --2- 172.21.15.133:0/4105470551 >> v2:172.21.15.161:3300/0 conn(0x7f9614140cb0 0x7f9614162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.298+0000 7f961a400640 1 -- 172.21.15.133:0/4105470551 >> 172.21.15.133:0/4105470551 conn(0x7f96140674e0 msgr2=0x7f96140d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:09.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.299+0000 7f961a400640 1 -- 172.21.15.133:0/4105470551 shutdown_connections 2024-03-07T06:52:09.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.299+0000 7f961a400640 1 -- 172.21.15.133:0/4105470551 wait complete. 2024-03-07T06:52:09.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.300+0000 7f961a400640 1 Processor -- start 2024-03-07T06:52:09.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.300+0000 7f961a400640 1 -- start start 2024-03-07T06:52:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.301+0000 7f961a400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96140b70d0 0x7f961413c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.301+0000 7f961a400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96140b9300 0x7f9614145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.301+0000 7f961a400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9614140cb0 0x7f961413e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.301+0000 7f961a400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96141627d0 con 0x7f9614140cb0 2024-03-07T06:52:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.301+0000 7f961a400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96141440b0 con 0x7f96140b9300 2024-03-07T06:52:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.301+0000 7f961a400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9614161bb0 con 0x7f96140b70d0 2024-03-07T06:52:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.302+0000 7f9619a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9614140cb0 0x7f961413e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.302+0000 7f9619a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9614140cb0 0x7f961413e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47554/0 (socket says 172.21.15.133:47554) 2024-03-07T06:52:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.302+0000 7f9619a00640 1 -- 172.21.15.133:0/298322096 learned_addr learned my addr 172.21.15.133:0/298322096 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.302+0000 7f9619000640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96140b70d0 0x7f961413c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.302+0000 7f960be00640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96140b9300 0x7f9614145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.302+0000 7f9619a00640 1 -- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96140b70d0 msgr2=0x7f961413c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.302+0000 7f9619a00640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96140b70d0 0x7f961413c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.302+0000 7f9619a00640 1 -- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96140b9300 msgr2=0x7f9614145e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:09.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.302+0000 7f9619a00640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96140b9300 0x7f9614145e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.302+0000 7f9619a00640 1 -- 172.21.15.133:0/298322096 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961413d5e0 con 0x7f9614140cb0 2024-03-07T06:52:09.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.302+0000 7f960be00640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96140b9300 0x7f9614145e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:52:09.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.302+0000 7f9619a00640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9614140cb0 0x7f961413e680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c7f0 tx=0x7f960c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:09.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.303+0000 7f9611600640 1 -- 172.21.15.133:0/298322096 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c015070 con 0x7f9614140cb0 2024-03-07T06:52:09.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.303+0000 7f9611600640 1 -- 172.21.15.133:0/298322096 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c004220 con 0x7f9614140cb0 2024-03-07T06:52:09.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.303+0000 7f9611600640 1 -- 172.21.15.133:0/298322096 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c007620 con 0x7f9614140cb0 2024-03-07T06:52:09.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.303+0000 7f961a400640 1 -- 172.21.15.133:0/298322096 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961413d8c0 con 0x7f9614140cb0 2024-03-07T06:52:09.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.303+0000 7f961a400640 1 -- 172.21.15.133:0/298322096 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961413de80 con 0x7f9614140cb0 2024-03-07T06:52:09.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.304+0000 7f961a400640 1 -- 172.21.15.133:0/298322096 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9614062c90 con 0x7f9614140cb0 2024-03-07T06:52:09.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.305+0000 7f9611600640 1 -- 172.21.15.133:0/298322096 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f960c0077c0 con 0x7f9614140cb0 2024-03-07T06:52:09.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.306+0000 7f9611600640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f95f006b4d0 0x7f95f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.306+0000 7f9611600640 1 -- 172.21.15.133:0/298322096 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f960c08a190 con 0x7f9614140cb0 2024-03-07T06:52:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.307+0000 7f9619000640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f95f006b4d0 0x7f95f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.308+0000 7f9619000640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f95f006b4d0 0x7f95f006d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9604007130 tx=0x7f9604007060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:09.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.311+0000 7f9611600640 1 -- 172.21.15.133:0/298322096 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f960c058ef0 con 0x7f9614140cb0 2024-03-07T06:52:09.506 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.507+0000 7f961a400640 1 -- 172.21.15.133:0/298322096 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96140b7c00 con 0x7f9614140cb0 2024-03-07T06:52:09.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.510+0000 7f9611600640 1 -- 172.21.15.133:0/298322096 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16901 (secure 0 0 0) 0x7f960c0587f0 con 0x7f9614140cb0 2024-03-07T06:52:09.509 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:09.509 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:56.413096+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:52:09.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.513+0000 7f961a400640 1 -- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f95f006b4d0 msgr2=0x7f95f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:09.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.513+0000 7f961a400640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f95f006b4d0 0x7f95f006d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9604007130 tx=0x7f9604007060 comp rx=0 tx=0).stop 2024-03-07T06:52:09.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.513+0000 7f961a400640 1 -- 172.21.15.133:0/298322096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9614140cb0 msgr2=0x7f961413e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:09.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.513+0000 7f961a400640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9614140cb0 0x7f961413e680 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c7f0 tx=0x7f960c00ccb0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.513+0000 7f961a400640 1 -- 172.21.15.133:0/298322096 shutdown_connections 2024-03-07T06:52:09.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.513+0000 7f961a400640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9614140cb0 0x7f961413e680 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.513+0000 7f961a400640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f95f006b4d0 0x7f95f006d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.513+0000 7f961a400640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f96140b9300 0x7f9614145e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.513+0000 7f961a400640 1 --2- 172.21.15.133:0/298322096 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f96140b70d0 0x7f961413c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.513+0000 7f961a400640 1 -- 172.21.15.133:0/298322096 >> 172.21.15.133:0/298322096 conn(0x7f96140674e0 msgr2=0x7f9614160630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:09.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.513+0000 7f961a400640 1 -- 172.21.15.133:0/298322096 shutdown_connections 2024-03-07T06:52:09.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.513+0000 7f961a400640 1 -- 172.21.15.133:0/298322096 wait complete. 2024-03-07T06:52:09.534 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:51:12.858333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:51:13.615348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:52:09.535 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:52:09.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.747+0000 7f853ae00640 1 Processor -- start 2024-03-07T06:52:09.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.747+0000 7f853ae00640 1 -- start start 2024-03-07T06:52:09.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.749+0000 7f853ae00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f853415b370 0x7f85340cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.749+0000 7f853ae00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f853406d180 0x7f85340cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.749+0000 7f853ae00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853406d740 con 0x7f853415b750 2024-03-07T06:52:09.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.749+0000 7f853ae00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f853406da20 con 0x7f853406d180 2024-03-07T06:52:09.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.749+0000 7f853ae00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853406dd00 con 0x7f853415b370 2024-03-07T06:52:09.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.749+0000 7f8538e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f853415b750 0x7f853415bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47842/0 (socket says 172.21.15.133:47842) 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.749+0000 7f8538e00640 1 -- 172.21.15.133:0/3310278994 learned_addr learned my addr 172.21.15.133:0/3310278994 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.749+0000 7f8533e00640 1 --2- 172.21.15.133:0/3310278994 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f853415b370 0x7f85340cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.749+0000 7f8533400640 1 --2- 172.21.15.133:0/3310278994 >> v2:172.21.15.161:3300/0 conn(0x7f853406d180 0x7f85340cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 -- 172.21.15.133:0/3310278994 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302414221 0 0) 0x7f853406d740 con 0x7f853415b750 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 -- 172.21.15.133:0/3310278994 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8518005730 con 0x7f853415b750 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 -- 172.21.15.133:0/3310278994 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2121064878 0 0) 0x7f8518005730 con 0x7f853415b750 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 -- 172.21.15.133:0/3310278994 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f853415b370 msgr2=0x7f85340cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 --2- 172.21.15.133:0/3310278994 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f853415b370 0x7f85340cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 -- 172.21.15.133:0/3310278994 >> v2:172.21.15.161:3300/0 conn(0x7f853406d180 msgr2=0x7f85340cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 --2- 172.21.15.133:0/3310278994 >> v2:172.21.15.161:3300/0 conn(0x7f853406d180 0x7f85340cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 -- 172.21.15.133:0/3310278994 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853406dfe0 con 0x7f853415b750 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 -- 172.21.15.133:0/3310278994 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f852c0019f0 con 0x7f853415b750 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 -- 172.21.15.133:0/3310278994 >> v1:172.21.15.133:6789/0 conn(0x7f853415b750 legacy=0x7f853415bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 --2- 172.21.15.133:0/3310278994 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f853415b370 0x7f8518005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 --2- 172.21.15.133:0/3310278994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8518006190 0x7f8518006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 --2- 172.21.15.133:0/3310278994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518006f50 0x7f85180073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 -- 172.21.15.133:0/3310278994 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853406da20 con 0x7f8518006190 2024-03-07T06:52:09.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 -- 172.21.15.133:0/3310278994 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853406dd00 con 0x7f8518006f50 2024-03-07T06:52:09.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8532a00640 1 -- 172.21.15.133:0/3310278994 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f851800dc80 con 0x7f853415b370 2024-03-07T06:52:09.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.750+0000 7f8533400640 1 --2- 172.21.15.133:0/3310278994 >> v2:172.21.15.161:3300/0 conn(0x7f853406d180 0x7f85340cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:52:09.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.751+0000 7f8533400640 1 --2- 172.21.15.133:0/3310278994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8518006190 0x7f8518006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.751+0000 7f8538e00640 1 --2- 172.21.15.133:0/3310278994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518006f50 0x7f85180073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.751+0000 7f8533e00640 1 --2- 172.21.15.133:0/3310278994 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f853415b370 0x7f8518005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.751+0000 7f8533400640 1 -- 172.21.15.133:0/3310278994 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f853415b370 msgr2=0x7f8518005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:09.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.751+0000 7f8533400640 1 --2- 172.21.15.133:0/3310278994 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f853415b370 0x7f8518005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.751+0000 7f8533400640 1 -- 172.21.15.133:0/3310278994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518006f50 msgr2=0x7f85180073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:09.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.751+0000 7f8533400640 1 --2- 172.21.15.133:0/3310278994 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518006f50 0x7f85180073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.751+0000 7f8533400640 1 -- 172.21.15.133:0/3310278994 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8518005730 con 0x7f8518006190 2024-03-07T06:52:09.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.751+0000 7f8533e00640 1 --2- 172.21.15.133:0/3310278994 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f853415b370 0x7f8518005bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:09.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.752+0000 7f8533400640 1 --2- 172.21.15.133:0/3310278994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8518006190 0x7f8518006a10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f85200098a0 tx=0x7f8518007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa7168b46d8a831 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:09.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.753+0000 7f8532a00640 1 -- 172.21.15.133:0/3310278994 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852000f040 con 0x7f8518006190 2024-03-07T06:52:09.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.753+0000 7f8532a00640 1 -- 172.21.15.133:0/3310278994 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8520037600 con 0x7f8518006190 2024-03-07T06:52:09.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.753+0000 7f8532a00640 1 -- 172.21.15.133:0/3310278994 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85200379b0 con 0x7f8518006190 2024-03-07T06:52:09.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.753+0000 7f853ae00640 1 -- 172.21.15.133:0/3310278994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8518006190 msgr2=0x7f8518006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:09.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.753+0000 7f853ae00640 1 --2- 172.21.15.133:0/3310278994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8518006190 0x7f8518006a10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f85200098a0 tx=0x7f8518007a40 comp rx=0 tx=0).stop 2024-03-07T06:52:09.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.753+0000 7f8538e00640 1 -- 172.21.15.133:0/3310278994 reap_dead start 2024-03-07T06:52:09.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.753+0000 7f853ae00640 1 -- 172.21.15.133:0/3310278994 shutdown_connections 2024-03-07T06:52:09.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.753+0000 7f853ae00640 1 -- 172.21.15.133:0/3310278994 >> 172.21.15.133:0/3310278994 conn(0x7f85340b1d30 msgr2=0x7f85340b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:09.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.753+0000 7f853ae00640 1 -- 172.21.15.133:0/3310278994 shutdown_connections 2024-03-07T06:52:09.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.753+0000 7f853ae00640 1 -- 172.21.15.133:0/3310278994 wait complete. 2024-03-07T06:52:09.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.755+0000 7f853ae00640 1 Processor -- start 2024-03-07T06:52:09.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.755+0000 7f853ae00640 1 -- start start 2024-03-07T06:52:09.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.756+0000 7f853ae00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518006190 0x7f85341e1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.756+0000 7f853ae00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8534134f60 0x7f85341e41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.756+0000 7f853ae00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85341e4720 0x7f853414eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.756+0000 7f853ae00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85340b4050 con 0x7f8534134f60 2024-03-07T06:52:09.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.756+0000 7f853ae00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85340b3ed0 con 0x7f8518006190 2024-03-07T06:52:09.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.756+0000 7f853ae00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85340b41d0 con 0x7f85341e4720 2024-03-07T06:52:09.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.756+0000 7f8533400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8534134f60 0x7f85341e41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.756+0000 7f8533400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8534134f60 0x7f85341e41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47562/0 (socket says 172.21.15.133:47562) 2024-03-07T06:52:09.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.756+0000 7f8533400640 1 -- 172.21.15.133:0/1132976322 learned_addr learned my addr 172.21.15.133:0/1132976322 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:09.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.757+0000 7f8533e00640 1 --2- 172.21.15.133:0/1132976322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518006190 0x7f85341e1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.757+0000 7f8538e00640 1 --2- 172.21.15.133:0/1132976322 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85341e4720 0x7f853414eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.757+0000 7f8533400640 1 -- 172.21.15.133:0/1132976322 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85341e4720 msgr2=0x7f853414eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:09.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.757+0000 7f8533400640 1 --2- 172.21.15.133:0/1132976322 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85341e4720 0x7f853414eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.757+0000 7f8533400640 1 -- 172.21.15.133:0/1132976322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518006190 msgr2=0x7f85341e1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:09.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.757+0000 7f8533400640 1 --2- 172.21.15.133:0/1132976322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518006190 0x7f85341e1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.757+0000 7f8533400640 1 -- 172.21.15.133:0/1132976322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85341e4ba0 con 0x7f8534134f60 2024-03-07T06:52:09.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.757+0000 7f8533400640 1 --2- 172.21.15.133:0/1132976322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8534134f60 0x7f85341e41e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8520009990 tx=0x7f8520004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:09.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.758+0000 7f852be00640 1 -- 172.21.15.133:0/1132976322 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852000f000 con 0x7f8534134f60 2024-03-07T06:52:09.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.758+0000 7f852be00640 1 -- 172.21.15.133:0/1132976322 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852002fa60 con 0x7f8534134f60 2024-03-07T06:52:09.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.758+0000 7f852be00640 1 -- 172.21.15.133:0/1132976322 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852002fe30 con 0x7f8534134f60 2024-03-07T06:52:09.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.758+0000 7f853ae00640 1 -- 172.21.15.133:0/1132976322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85341358c0 con 0x7f8534134f60 2024-03-07T06:52:09.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.758+0000 7f853ae00640 1 -- 172.21.15.133:0/1132976322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8534135e80 con 0x7f8534134f60 2024-03-07T06:52:09.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.759+0000 7f853ae00640 1 -- 172.21.15.133:0/1132976322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8534062c90 con 0x7f8534134f60 2024-03-07T06:52:09.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.760+0000 7f852be00640 1 -- 172.21.15.133:0/1132976322 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8520042440 con 0x7f8534134f60 2024-03-07T06:52:09.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.761+0000 7f852be00640 1 --2- 172.21.15.133:0/1132976322 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f851406b500 0x7f851406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:09.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.761+0000 7f852be00640 1 -- 172.21.15.133:0/1132976322 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f85200ac900 con 0x7f8534134f60 2024-03-07T06:52:09.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.761+0000 7f8533e00640 1 --2- 172.21.15.133:0/1132976322 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f851406b500 0x7f851406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:09.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.763+0000 7f8533e00640 1 --2- 172.21.15.133:0/1132976322 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f851406b500 0x7f851406d9c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f85240098f0 tx=0x7f8524008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:09.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.765+0000 7f852be00640 1 -- 172.21.15.133:0/1132976322 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8520033030 con 0x7f8534134f60 2024-03-07T06:52:09.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.960+0000 7f853ae00640 1 -- 172.21.15.133:0/1132976322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8534072020 con 0x7f8534134f60 2024-03-07T06:52:09.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.961+0000 7f852be00640 1 -- 172.21.15.133:0/1132976322 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f852007b700 con 0x7f8534134f60 2024-03-07T06:52:09.961 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:52:09.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.964+0000 7f853ae00640 1 -- 172.21.15.133:0/1132976322 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f851406b500 msgr2=0x7f851406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:09.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.964+0000 7f853ae00640 1 --2- 172.21.15.133:0/1132976322 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f851406b500 0x7f851406d9c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f85240098f0 tx=0x7f8524008040 comp rx=0 tx=0).stop 2024-03-07T06:52:09.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.964+0000 7f853ae00640 1 -- 172.21.15.133:0/1132976322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8534134f60 msgr2=0x7f85341e41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:09.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.964+0000 7f853ae00640 1 --2- 172.21.15.133:0/1132976322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8534134f60 0x7f85341e41e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8520009990 tx=0x7f8520004730 comp rx=0 tx=0).stop 2024-03-07T06:52:09.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.964+0000 7f853ae00640 1 -- 172.21.15.133:0/1132976322 shutdown_connections 2024-03-07T06:52:09.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.964+0000 7f853ae00640 1 --2- 172.21.15.133:0/1132976322 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f85341e4720 0x7f853414eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.964+0000 7f853ae00640 1 --2- 172.21.15.133:0/1132976322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8534134f60 0x7f85341e41e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.964+0000 7f853ae00640 1 --2- 172.21.15.133:0/1132976322 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f851406b500 0x7f851406d9c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.964+0000 7f853ae00640 1 --2- 172.21.15.133:0/1132976322 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8518006190 0x7f85341e1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:09.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.964+0000 7f853ae00640 1 -- 172.21.15.133:0/1132976322 >> 172.21.15.133:0/1132976322 conn(0x7f85340b1d30 msgr2=0x7f8534150920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:09.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.964+0000 7f853ae00640 1 -- 172.21.15.133:0/1132976322 shutdown_connections 2024-03-07T06:52:09.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:09.964+0000 7f853ae00640 1 -- 172.21.15.133:0/1132976322 wait complete. 2024-03-07T06:52:09.984 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:52:10.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.200+0000 7f9c29000640 1 Processor -- start 2024-03-07T06:52:10.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.200+0000 7f9c29000640 1 -- start start 2024-03-07T06:52:10.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.202+0000 7f9c29000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9c240cf610 0x7f9c240cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.202+0000 7f9c29000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9c240cda20 0x7f9c240719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.202+0000 7f9c29000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c2406e150 con 0x7f9c24145670 2024-03-07T06:52:10.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.202+0000 7f9c29000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c240cde70 con 0x7f9c240cda20 2024-03-07T06:52:10.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.202+0000 7f9c29000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c2406ed60 con 0x7f9c240cf610 2024-03-07T06:52:10.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.202+0000 7f9c22800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9c240cda20 0x7f9c240719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:10.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.202+0000 7f9c22800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9c240cda20 0x7f9c240719a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47566/0 (socket says 172.21.15.133:47566) 2024-03-07T06:52:10.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.202+0000 7f9c22800640 1 -- 172.21.15.133:0/2569706190 learned_addr learned my addr 172.21.15.133:0/2569706190 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:10.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.202+0000 7f9c21400640 1 --2- 172.21.15.133:0/2569706190 >> v2:172.21.15.161:3300/0 conn(0x7f9c240cf610 0x7f9c240cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:10.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.203+0000 7f9c22800640 1 -- 172.21.15.133:0/2569706190 >> v2:172.21.15.161:3300/0 conn(0x7f9c240cf610 msgr2=0x7f9c240cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:10.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.203+0000 7f9c22800640 1 --2- 172.21.15.133:0/2569706190 >> v2:172.21.15.161:3300/0 conn(0x7f9c240cf610 0x7f9c240cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.203+0000 7f9c22800640 1 -- 172.21.15.133:0/2569706190 >> v1:172.21.15.161:6789/0 conn(0x7f9c24145670 legacy=0x7f9c2415cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:10.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.203+0000 7f9c22800640 1 -- 172.21.15.133:0/2569706190 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2406e430 con 0x7f9c240cda20 2024-03-07T06:52:10.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.204+0000 7f9c22800640 1 --2- 172.21.15.133:0/2569706190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c240cda20 0x7f9c240719a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009e50 tx=0x7f9c1c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25da1ea10eb9e7aa server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:10.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.204+0000 7f9c20a00640 1 -- 172.21.15.133:0/2569706190 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c02c9f0 con 0x7f9c240cda20 2024-03-07T06:52:10.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.204+0000 7f9c20a00640 1 -- 172.21.15.133:0/2569706190 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c02cb90 con 0x7f9c240cda20 2024-03-07T06:52:10.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.204+0000 7f9c20a00640 1 -- 172.21.15.133:0/2569706190 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0343f0 con 0x7f9c240cda20 2024-03-07T06:52:10.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.205+0000 7f9c29000640 1 -- 172.21.15.133:0/2569706190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c240cda20 msgr2=0x7f9c240719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:10.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.205+0000 7f9c29000640 1 --2- 172.21.15.133:0/2569706190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c240cda20 0x7f9c240719a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009e50 tx=0x7f9c1c02be50 comp rx=0 tx=0).stop 2024-03-07T06:52:10.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.205+0000 7f9c29000640 1 -- 172.21.15.133:0/2569706190 shutdown_connections 2024-03-07T06:52:10.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.205+0000 7f9c29000640 1 --2- 172.21.15.133:0/2569706190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c240cda20 0x7f9c240719a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.205+0000 7f9c29000640 1 --2- 172.21.15.133:0/2569706190 >> v2:172.21.15.161:3300/0 conn(0x7f9c240cf610 0x7f9c240cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.205+0000 7f9c29000640 1 -- 172.21.15.133:0/2569706190 >> 172.21.15.133:0/2569706190 conn(0x7f9c240b1d30 msgr2=0x7f9c240b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:10.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.205+0000 7f9c29000640 1 -- 172.21.15.133:0/2569706190 shutdown_connections 2024-03-07T06:52:10.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.205+0000 7f9c29000640 1 -- 172.21.15.133:0/2569706190 wait complete. 2024-03-07T06:52:10.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.207+0000 7f9c29000640 1 Processor -- start 2024-03-07T06:52:10.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.207+0000 7f9c29000640 1 -- start start 2024-03-07T06:52:10.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c29000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c240cda20 0x7f9c241f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c29000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c240cf610 0x7f9c240b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c29000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c24145670 0x7f9c24072520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c29000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c24145a50 con 0x7f9c240cf610 2024-03-07T06:52:10.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c29000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2406e9f0 con 0x7f9c240cda20 2024-03-07T06:52:10.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c29000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c24145bd0 con 0x7f9c24145670 2024-03-07T06:52:10.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c21400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c240cf610 0x7f9c240b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:10.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c21400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c240cf610 0x7f9c240b2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47582/0 (socket says 172.21.15.133:47582) 2024-03-07T06:52:10.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c21400640 1 -- 172.21.15.133:0/2895902121 learned_addr learned my addr 172.21.15.133:0/2895902121 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:10.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c21400640 1 -- 172.21.15.133:0/2895902121 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c24145670 msgr2=0x7f9c24072520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:10.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c22800640 1 --2- 172.21.15.133:0/2895902121 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c24145670 0x7f9c24072520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:10.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c21e00640 1 --2- 172.21.15.133:0/2895902121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c240cda20 0x7f9c241f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:10.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c21400640 1 --2- 172.21.15.133:0/2895902121 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c24145670 0x7f9c24072520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c21400640 1 -- 172.21.15.133:0/2895902121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c240cda20 msgr2=0x7f9c241f6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:10.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c21400640 1 --2- 172.21.15.133:0/2895902121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c240cda20 0x7f9c241f6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.208+0000 7f9c21400640 1 -- 172.21.15.133:0/2895902121 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c240b3930 con 0x7f9c240cf610 2024-03-07T06:52:10.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.209+0000 7f9c21400640 1 --2- 172.21.15.133:0/2895902121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c240cf610 0x7f9c240b2f50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00d620 tx=0x7f9c0c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:10.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.209+0000 7f9c1a000640 1 -- 172.21.15.133:0/2895902121 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00b620 con 0x7f9c240cf610 2024-03-07T06:52:10.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.209+0000 7f9c1a000640 1 -- 172.21.15.133:0/2895902121 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c019070 con 0x7f9c240cf610 2024-03-07T06:52:10.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.209+0000 7f9c1a000640 1 -- 172.21.15.133:0/2895902121 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00ba00 con 0x7f9c240cf610 2024-03-07T06:52:10.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.209+0000 7f9c29000640 1 -- 172.21.15.133:0/2895902121 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c240b3c10 con 0x7f9c240cf610 2024-03-07T06:52:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.209+0000 7f9c29000640 1 -- 172.21.15.133:0/2895902121 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c240b41d0 con 0x7f9c240cf610 2024-03-07T06:52:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.210+0000 7f9c29000640 1 -- 172.21.15.133:0/2895902121 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c24062c90 con 0x7f9c240cf610 2024-03-07T06:52:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.211+0000 7f9c1a000640 1 -- 172.21.15.133:0/2895902121 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9c0c00bba0 con 0x7f9c240cf610 2024-03-07T06:52:10.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.212+0000 7f9c1a000640 1 --2- 172.21.15.133:0/2895902121 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9c0006b500 0x7f9c0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.212+0000 7f9c1a000640 1 -- 172.21.15.133:0/2895902121 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f9c0c08a720 con 0x7f9c240cf610 2024-03-07T06:52:10.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.213+0000 7f9c21e00640 1 --2- 172.21.15.133:0/2895902121 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9c0006b500 0x7f9c0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:10.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.214+0000 7f9c21e00640 1 --2- 172.21.15.133:0/2895902121 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9c0006b500 0x7f9c0006d9c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9c24072390 tx=0x7f9c14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:10.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.216+0000 7f9c1a000640 1 -- 172.21.15.133:0/2895902121 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9c0c059480 con 0x7f9c240cf610 2024-03-07T06:52:10.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.412+0000 7f9c29000640 1 -- 172.21.15.133:0/2895902121 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c2406f440 con 0x7f9c240cf610 2024-03-07T06:52:10.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.413+0000 7f9c1a000640 1 -- 172.21.15.133:0/2895902121 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9c0c058d80 con 0x7f9c240cf610 2024-03-07T06:52:10.412 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:52:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.415+0000 7f9c29000640 1 -- 172.21.15.133:0/2895902121 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9c0006b500 msgr2=0x7f9c0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.415+0000 7f9c29000640 1 --2- 172.21.15.133:0/2895902121 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9c0006b500 0x7f9c0006d9c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9c24072390 tx=0x7f9c14009040 comp rx=0 tx=0).stop 2024-03-07T06:52:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.415+0000 7f9c29000640 1 -- 172.21.15.133:0/2895902121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c240cf610 msgr2=0x7f9c240b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.415+0000 7f9c29000640 1 --2- 172.21.15.133:0/2895902121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c240cf610 0x7f9c240b2f50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c00d620 tx=0x7f9c0c00dae0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.416+0000 7f9c29000640 1 -- 172.21.15.133:0/2895902121 shutdown_connections 2024-03-07T06:52:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.416+0000 7f9c29000640 1 --2- 172.21.15.133:0/2895902121 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9c24145670 0x7f9c24072520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.416+0000 7f9c29000640 1 --2- 172.21.15.133:0/2895902121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c240cf610 0x7f9c240b2f50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.416+0000 7f9c29000640 1 --2- 172.21.15.133:0/2895902121 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9c0006b500 0x7f9c0006d9c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.416+0000 7f9c29000640 1 --2- 172.21.15.133:0/2895902121 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9c240cda20 0x7f9c241f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.416+0000 7f9c29000640 1 -- 172.21.15.133:0/2895902121 >> 172.21.15.133:0/2895902121 conn(0x7f9c240b1d30 msgr2=0x7f9c241434c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:10.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.416+0000 7f9c29000640 1 -- 172.21.15.133:0/2895902121 shutdown_connections 2024-03-07T06:52:10.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.416+0000 7f9c29000640 1 -- 172.21.15.133:0/2895902121 wait complete. 2024-03-07T06:52:10.432 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:52:10.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.638+0000 7f4031000640 1 Processor -- start 2024-03-07T06:52:10.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.638+0000 7f4031000640 1 -- start start 2024-03-07T06:52:10.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.640+0000 7f4031000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f402c15b370 0x7f402c0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.640+0000 7f4031000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f402c0b3ed0 0x7f402c0cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.640+0000 7f4031000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402c06d740 con 0x7f402c06d180 2024-03-07T06:52:10.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.640+0000 7f4031000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402c06da20 con 0x7f402c15b370 2024-03-07T06:52:10.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.640+0000 7f4031000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402c06dd00 con 0x7f402c0b3ed0 2024-03-07T06:52:10.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.640+0000 7f4029400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f402c06d180 0x7f402c15b750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47850/0 (socket says 172.21.15.133:47850) 2024-03-07T06:52:10.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.640+0000 7f4029400640 1 -- 172.21.15.133:0/763813219 learned_addr learned my addr 172.21.15.133:0/763813219 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:10.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.640+0000 7f4029e00640 1 --2- 172.21.15.133:0/763813219 >> v2:172.21.15.161:3300/0 conn(0x7f402c15b370 0x7f402c0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f402a800640 1 --2- 172.21.15.133:0/763813219 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f402c0b3ed0 0x7f402c0cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 -- 172.21.15.133:0/763813219 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249702083 0 0) 0x7f402c06d740 con 0x7f402c06d180 2024-03-07T06:52:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 -- 172.21.15.133:0/763813219 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4010003520 con 0x7f402c06d180 2024-03-07T06:52:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 -- 172.21.15.133:0/763813219 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3152865299 0 0) 0x7f4010003520 con 0x7f402c06d180 2024-03-07T06:52:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 -- 172.21.15.133:0/763813219 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f402c0b3ed0 msgr2=0x7f402c0cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 --2- 172.21.15.133:0/763813219 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f402c0b3ed0 0x7f402c0cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 -- 172.21.15.133:0/763813219 >> v2:172.21.15.161:3300/0 conn(0x7f402c15b370 msgr2=0x7f402c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 --2- 172.21.15.133:0/763813219 >> v2:172.21.15.161:3300/0 conn(0x7f402c15b370 0x7f402c0cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 -- 172.21.15.133:0/763813219 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c06dfe0 con 0x7f402c06d180 2024-03-07T06:52:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 -- 172.21.15.133:0/763813219 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f4018001670 con 0x7f402c06d180 2024-03-07T06:52:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 -- 172.21.15.133:0/763813219 >> v1:172.21.15.133:6789/0 conn(0x7f402c06d180 legacy=0x7f402c15b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 --2- 172.21.15.133:0/763813219 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f402c0b3ed0 0x7f40100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 --2- 172.21.15.133:0/763813219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4010003f30 0x7f4010004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 --2- 172.21.15.133:0/763813219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4010004d40 0x7f40100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 -- 172.21.15.133:0/763813219 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c06da20 con 0x7f4010003f30 2024-03-07T06:52:10.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 -- 172.21.15.133:0/763813219 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c06dd00 con 0x7f4010004d40 2024-03-07T06:52:10.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4028a00640 1 -- 172.21.15.133:0/763813219 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401000b9c0 con 0x7f402c0b3ed0 2024-03-07T06:52:10.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.641+0000 7f4029400640 1 --2- 172.21.15.133:0/763813219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4010003f30 0x7f4010004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:10.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.642+0000 7f4029400640 1 -- 172.21.15.133:0/763813219 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f402c0b3ed0 msgr2=0x7f40100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:10.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.642+0000 7f4029400640 1 --2- 172.21.15.133:0/763813219 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f402c0b3ed0 0x7f40100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.642+0000 7f4029400640 1 -- 172.21.15.133:0/763813219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4010004d40 msgr2=0x7f40100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:10.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.642+0000 7f4029400640 1 --2- 172.21.15.133:0/763813219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4010004d40 0x7f40100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.642+0000 7f4029400640 1 -- 172.21.15.133:0/763813219 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4010003520 con 0x7f4010003f30 2024-03-07T06:52:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.643+0000 7f4029400640 1 --2- 172.21.15.133:0/763813219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4010003f30 0x7f4010004800 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f401800c290 tx=0x7f4010005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b1f7cf7742810a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.643+0000 7f4028a00640 1 -- 172.21.15.133:0/763813219 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018007d50 con 0x7f4010003f30 2024-03-07T06:52:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.643+0000 7f4028a00640 1 -- 172.21.15.133:0/763813219 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4018030070 con 0x7f4010003f30 2024-03-07T06:52:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.643+0000 7f4028a00640 1 -- 172.21.15.133:0/763813219 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401802cdb0 con 0x7f4010003f30 2024-03-07T06:52:10.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.644+0000 7f4031000640 1 -- 172.21.15.133:0/763813219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4010003f30 msgr2=0x7f4010004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:10.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.644+0000 7f4031000640 1 --2- 172.21.15.133:0/763813219 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4010003f30 0x7f4010004800 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f401800c290 tx=0x7f4010005830 comp rx=0 tx=0).stop 2024-03-07T06:52:10.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.644+0000 7f402a800640 1 -- 172.21.15.133:0/763813219 reap_dead start 2024-03-07T06:52:10.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.644+0000 7f4031000640 1 -- 172.21.15.133:0/763813219 shutdown_connections 2024-03-07T06:52:10.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.644+0000 7f4031000640 1 -- 172.21.15.133:0/763813219 >> 172.21.15.133:0/763813219 conn(0x7f402c0b1d30 msgr2=0x7f402c0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:10.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.644+0000 7f4031000640 1 -- 172.21.15.133:0/763813219 shutdown_connections 2024-03-07T06:52:10.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.644+0000 7f4031000640 1 -- 172.21.15.133:0/763813219 wait complete. 2024-03-07T06:52:10.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.645+0000 7f4031000640 1 Processor -- start 2024-03-07T06:52:10.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.646+0000 7f4031000640 1 -- start start 2024-03-07T06:52:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.646+0000 7f4031000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4010003f30 0x7f402c147890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.646+0000 7f4031000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402c149f80 0x7f402c141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f4031000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f402c142050 0x7f402c140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f4031000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c15bfe0 con 0x7f402c149f80 2024-03-07T06:52:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f4031000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c15be60 con 0x7f402c142050 2024-03-07T06:52:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f4031000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402c15c160 con 0x7f4010003f30 2024-03-07T06:52:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f4029400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402c149f80 0x7f402c141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f4029400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402c149f80 0x7f402c141b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47592/0 (socket says 172.21.15.133:47592) 2024-03-07T06:52:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f4029400640 1 -- 172.21.15.133:0/3615750760 learned_addr learned my addr 172.21.15.133:0/3615750760 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f4029e00640 1 --2- 172.21.15.133:0/3615750760 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4010003f30 0x7f402c147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:10.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f4029400640 1 -- 172.21.15.133:0/3615750760 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4010003f30 msgr2=0x7f402c147890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f4029400640 1 --2- 172.21.15.133:0/3615750760 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4010003f30 0x7f402c147890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f402a800640 1 --2- 172.21.15.133:0/3615750760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f402c142050 0x7f402c140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f4029400640 1 -- 172.21.15.133:0/3615750760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f402c142050 msgr2=0x7f402c140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f4029400640 1 --2- 172.21.15.133:0/3615750760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f402c142050 0x7f402c140380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f4029400640 1 -- 172.21.15.133:0/3615750760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c1442e0 con 0x7f402c149f80 2024-03-07T06:52:10.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.647+0000 7f4029400640 1 --2- 172.21.15.133:0/3615750760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402c149f80 0x7f402c141b10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f401800c290 tx=0x7f402c06e1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:10.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.648+0000 7f4022000640 1 -- 172.21.15.133:0/3615750760 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018045070 con 0x7f402c149f80 2024-03-07T06:52:10.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.648+0000 7f4022000640 1 -- 172.21.15.133:0/3615750760 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4018004bf0 con 0x7f402c149f80 2024-03-07T06:52:10.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.648+0000 7f4022000640 1 -- 172.21.15.133:0/3615750760 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018007070 con 0x7f402c149f80 2024-03-07T06:52:10.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.648+0000 7f4031000640 1 -- 172.21.15.133:0/3615750760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402c148200 con 0x7f402c149f80 2024-03-07T06:52:10.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.648+0000 7f4031000640 1 -- 172.21.15.133:0/3615750760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402c1487c0 con 0x7f402c149f80 2024-03-07T06:52:10.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.649+0000 7f4031000640 1 -- 172.21.15.133:0/3615750760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402c142460 con 0x7f402c149f80 2024-03-07T06:52:10.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.650+0000 7f4022000640 1 -- 172.21.15.133:0/3615750760 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4018007210 con 0x7f402c149f80 2024-03-07T06:52:10.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.651+0000 7f4022000640 1 --2- 172.21.15.133:0/3615750760 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f400c06b500 0x7f400c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:10.649 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.651+0000 7f4022000640 1 -- 172.21.15.133:0/3615750760 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f40180ab270 con 0x7f402c149f80 2024-03-07T06:52:10.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.651+0000 7f4029e00640 1 --2- 172.21.15.133:0/3615750760 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f400c06b500 0x7f400c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:10.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.652+0000 7f4029e00640 1 --2- 172.21.15.133:0/3615750760 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f400c06b500 0x7f400c06d9c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f402c1432d0 tx=0x7f401c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:10.653 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.655+0000 7f4022000640 1 -- 172.21.15.133:0/3615750760 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f402c142460 con 0x7f402c149f80 2024-03-07T06:52:10.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.851+0000 7f4031000640 1 -- 172.21.15.133:0/3615750760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f402c062c90 con 0x7f402c149f80 2024-03-07T06:52:10.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.853+0000 7f4022000640 1 -- 172.21.15.133:0/3615750760 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f401807a1e0 con 0x7f402c149f80 2024-03-07T06:52:10.851 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:52:10.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.855+0000 7f4031000640 1 -- 172.21.15.133:0/3615750760 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f400c06b500 msgr2=0x7f400c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:10.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.855+0000 7f4031000640 1 --2- 172.21.15.133:0/3615750760 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f400c06b500 0x7f400c06d9c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f402c1432d0 tx=0x7f401c008040 comp rx=0 tx=0).stop 2024-03-07T06:52:10.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.855+0000 7f4031000640 1 -- 172.21.15.133:0/3615750760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402c149f80 msgr2=0x7f402c141b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:10.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.855+0000 7f4031000640 1 --2- 172.21.15.133:0/3615750760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402c149f80 0x7f402c141b10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f401800c290 tx=0x7f402c06e1b0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.855+0000 7f4031000640 1 -- 172.21.15.133:0/3615750760 shutdown_connections 2024-03-07T06:52:10.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.855+0000 7f4031000640 1 --2- 172.21.15.133:0/3615750760 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f400c06b500 0x7f400c06d9c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.855+0000 7f4031000640 1 --2- 172.21.15.133:0/3615750760 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f402c142050 0x7f402c140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.855+0000 7f4031000640 1 --2- 172.21.15.133:0/3615750760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f402c149f80 0x7f402c141b10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.855+0000 7f4031000640 1 --2- 172.21.15.133:0/3615750760 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4010003f30 0x7f402c147890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:10.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.855+0000 7f4031000640 1 -- 172.21.15.133:0/3615750760 >> 172.21.15.133:0/3615750760 conn(0x7f402c0b1d30 msgr2=0x7f402c1531d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:10.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.855+0000 7f4031000640 1 -- 172.21.15.133:0/3615750760 shutdown_connections 2024-03-07T06:52:10.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:10.855+0000 7f4031000640 1 -- 172.21.15.133:0/3615750760 wait complete. 2024-03-07T06:52:10.874 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-07T06:52:11.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.082+0000 7f0238a00640 1 Processor -- start 2024-03-07T06:52:11.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.082+0000 7f0238a00640 1 -- start start 2024-03-07T06:52:11.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.084+0000 7f0238a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0234140cb0 0x7f0234162290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:11.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.084+0000 7f0238a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f02340d2b00 0x7f02340b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:11.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.084+0000 7f0238a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0234143810 con 0x7f02340b70d0 2024-03-07T06:52:11.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.084+0000 7f0238a00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02341435c0 con 0x7f02340d2b00 2024-03-07T06:52:11.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.084+0000 7f0238a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0234144420 con 0x7f0234140cb0 2024-03-07T06:52:11.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.084+0000 7f0233200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f02340d2b00 0x7f02340b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:11.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.084+0000 7f0233200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f02340d2b00 0x7f02340b7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38640/0 (socket says 172.21.15.133:38640) 2024-03-07T06:52:11.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.084+0000 7f0233200640 1 -- 172.21.15.133:0/1704830916 learned_addr learned my addr 172.21.15.133:0/1704830916 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:11.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.085+0000 7f0232800640 1 --2- 172.21.15.133:0/1704830916 >> v2:172.21.15.161:3300/0 conn(0x7f0234140cb0 0x7f0234162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:11.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.085+0000 7f0233200640 1 -- 172.21.15.133:0/1704830916 >> v2:172.21.15.161:3300/0 conn(0x7f0234140cb0 msgr2=0x7f0234162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:11.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.085+0000 7f0233200640 1 --2- 172.21.15.133:0/1704830916 >> v2:172.21.15.161:3300/0 conn(0x7f0234140cb0 0x7f0234162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.085+0000 7f0233200640 1 -- 172.21.15.133:0/1704830916 >> v1:172.21.15.161:6789/0 conn(0x7f02340b70d0 legacy=0x7f02340b9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:11.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.085+0000 7f0233200640 1 -- 172.21.15.133:0/1704830916 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0234143af0 con 0x7f02340d2b00 2024-03-07T06:52:11.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.086+0000 7f0233200640 1 --2- 172.21.15.133:0/1704830916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02340d2b00 0x7f02340b7560 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f02340d3ca0 tx=0x7f022c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f419e71350bf92a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:11.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.087+0000 7f0231400640 1 -- 172.21.15.133:0/1704830916 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c02c9f0 con 0x7f02340d2b00 2024-03-07T06:52:11.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.087+0000 7f0231400640 1 -- 172.21.15.133:0/1704830916 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022c02cb90 con 0x7f02340d2b00 2024-03-07T06:52:11.085 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.087+0000 7f0231400640 1 -- 172.21.15.133:0/1704830916 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c0343f0 con 0x7f02340d2b00 2024-03-07T06:52:11.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.087+0000 7f0238a00640 1 -- 172.21.15.133:0/1704830916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02340d2b00 msgr2=0x7f02340b7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:11.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.087+0000 7f0238a00640 1 --2- 172.21.15.133:0/1704830916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02340d2b00 0x7f02340b7560 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f02340d3ca0 tx=0x7f022c02be50 comp rx=0 tx=0).stop 2024-03-07T06:52:11.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.087+0000 7f0238a00640 1 -- 172.21.15.133:0/1704830916 shutdown_connections 2024-03-07T06:52:11.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.087+0000 7f0238a00640 1 --2- 172.21.15.133:0/1704830916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02340d2b00 0x7f02340b7560 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.087+0000 7f0238a00640 1 --2- 172.21.15.133:0/1704830916 >> v2:172.21.15.161:3300/0 conn(0x7f0234140cb0 0x7f0234162290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.087+0000 7f0238a00640 1 -- 172.21.15.133:0/1704830916 >> 172.21.15.133:0/1704830916 conn(0x7f02340674e0 msgr2=0x7f02340d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:11.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.087+0000 7f0238a00640 1 -- 172.21.15.133:0/1704830916 shutdown_connections 2024-03-07T06:52:11.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.087+0000 7f0238a00640 1 -- 172.21.15.133:0/1704830916 wait complete. 2024-03-07T06:52:11.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.090+0000 7f0238a00640 1 Processor -- start 2024-03-07T06:52:11.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.090+0000 7f0238a00640 1 -- start start 2024-03-07T06:52:11.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.091+0000 7f0238a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02340b70d0 0x7f023415ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:11.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.091+0000 7f0238a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02340d2b00 0x7f023413fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:11.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.091+0000 7f0238a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0234140cb0 0x7f0234146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:11.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.091+0000 7f0238a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02341627d0 con 0x7f02340b70d0 2024-03-07T06:52:11.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.091+0000 7f0238a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02341440b0 con 0x7f02340d2b00 2024-03-07T06:52:11.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.091+0000 7f0238a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0234161bb0 con 0x7f0234140cb0 2024-03-07T06:52:11.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.091+0000 7f0232800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02340b70d0 0x7f023415ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:11.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.091+0000 7f0232800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02340b70d0 0x7f023415ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38644/0 (socket says 172.21.15.133:38644) 2024-03-07T06:52:11.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.091+0000 7f0232800640 1 -- 172.21.15.133:0/2230380164 learned_addr learned my addr 172.21.15.133:0/2230380164 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:11.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.092+0000 7f0231e00640 1 --2- 172.21.15.133:0/2230380164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02340d2b00 0x7f023413fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:11.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.092+0000 7f0233200640 1 --2- 172.21.15.133:0/2230380164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0234140cb0 0x7f0234146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:11.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.092+0000 7f0232800640 1 -- 172.21.15.133:0/2230380164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0234140cb0 msgr2=0x7f0234146080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:11.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.092+0000 7f0232800640 1 --2- 172.21.15.133:0/2230380164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0234140cb0 0x7f0234146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.092+0000 7f0232800640 1 -- 172.21.15.133:0/2230380164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02340d2b00 msgr2=0x7f023413fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:11.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.092+0000 7f0232800640 1 --2- 172.21.15.133:0/2230380164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02340d2b00 0x7f023413fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.092+0000 7f0232800640 1 -- 172.21.15.133:0/2230380164 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0234160680 con 0x7f02340b70d0 2024-03-07T06:52:11.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.092+0000 7f0232800640 1 --2- 172.21.15.133:0/2230380164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02340b70d0 0x7f023415ffe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f022400d800 tx=0x7f022400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:11.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.092+0000 7f022aa00640 1 -- 172.21.15.133:0/2230380164 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022400b6c0 con 0x7f02340b70d0 2024-03-07T06:52:11.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.092+0000 7f022aa00640 1 -- 172.21.15.133:0/2230380164 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0224019070 con 0x7f02340b70d0 2024-03-07T06:52:11.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.092+0000 7f022aa00640 1 -- 172.21.15.133:0/2230380164 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022400ba80 con 0x7f02340b70d0 2024-03-07T06:52:11.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.092+0000 7f0238a00640 1 -- 172.21.15.133:0/2230380164 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0234155b30 con 0x7f02340b70d0 2024-03-07T06:52:11.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.093+0000 7f0238a00640 1 -- 172.21.15.133:0/2230380164 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02341560f0 con 0x7f02340b70d0 2024-03-07T06:52:11.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.094+0000 7f0238a00640 1 -- 172.21.15.133:0/2230380164 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0234062c90 con 0x7f02340b70d0 2024-03-07T06:52:11.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.095+0000 7f022aa00640 1 -- 172.21.15.133:0/2230380164 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f022400bc20 con 0x7f02340b70d0 2024-03-07T06:52:11.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.096+0000 7f022aa00640 1 --2- 172.21.15.133:0/2230380164 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f021406b4d0 0x7f021406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:11.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.096+0000 7f022aa00640 1 -- 172.21.15.133:0/2230380164 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f022408a7e0 con 0x7f02340b70d0 2024-03-07T06:52:11.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.096+0000 7f0231e00640 1 --2- 172.21.15.133:0/2230380164 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f021406b4d0 0x7f021406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:11.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.097+0000 7f0231e00640 1 --2- 172.21.15.133:0/2230380164 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f021406b4d0 0x7f021406d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0220006fd0 tx=0x7f0220008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:11.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.100+0000 7f022aa00640 1 -- 172.21.15.133:0/2230380164 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0224059540 con 0x7f02340b70d0 2024-03-07T06:52:11.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.299+0000 7f0238a00640 1 -- 172.21.15.133:0/2230380164 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f02341412c0 con 0x7f02340b70d0 2024-03-07T06:52:11.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.300+0000 7f022aa00640 1 -- 172.21.15.133:0/2230380164 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0224058e40 con 0x7f02340b70d0 2024-03-07T06:52:11.299 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-03-07T06:52:11.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.302+0000 7f0238a00640 1 -- 172.21.15.133:0/2230380164 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f021406b4d0 msgr2=0x7f021406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:11.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.302+0000 7f0238a00640 1 --2- 172.21.15.133:0/2230380164 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f021406b4d0 0x7f021406d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0220006fd0 tx=0x7f0220008040 comp rx=0 tx=0).stop 2024-03-07T06:52:11.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.302+0000 7f0238a00640 1 -- 172.21.15.133:0/2230380164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02340b70d0 msgr2=0x7f023415ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:11.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.302+0000 7f0238a00640 1 --2- 172.21.15.133:0/2230380164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02340b70d0 0x7f023415ffe0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f022400d800 tx=0x7f022400dcc0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.302+0000 7f0238a00640 1 -- 172.21.15.133:0/2230380164 shutdown_connections 2024-03-07T06:52:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.302+0000 7f0238a00640 1 --2- 172.21.15.133:0/2230380164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0234140cb0 0x7f0234146080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.302+0000 7f0238a00640 1 --2- 172.21.15.133:0/2230380164 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f021406b4d0 0x7f021406d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.302+0000 7f0238a00640 1 --2- 172.21.15.133:0/2230380164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f02340d2b00 0x7f023413fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.302+0000 7f0238a00640 1 --2- 172.21.15.133:0/2230380164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02340b70d0 0x7f023415ffe0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.302+0000 7f0238a00640 1 -- 172.21.15.133:0/2230380164 >> 172.21.15.133:0/2230380164 conn(0x7f02340674e0 msgr2=0x7f02341414e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.303+0000 7f0238a00640 1 -- 172.21.15.133:0/2230380164 shutdown_connections 2024-03-07T06:52:11.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.303+0000 7f0238a00640 1 -- 172.21.15.133:0/2230380164 wait complete. 2024-03-07T06:52:11.320 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-07T06:52:11.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:11.320 DEBUG:teuthology.orchestra.run.smithi161:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-07T06:52:11.417 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi161.front.sepia.ceph.com 2024-03-07T06:52:11.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:11.418 DEBUG:teuthology.orchestra.run.smithi161:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-07T06:52:11.466 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-07T06:52:11.467 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:52:11.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.681+0000 7f41e6800640 1 Processor -- start 2024-03-07T06:52:11.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.681+0000 7f41e6800640 1 -- start start 2024-03-07T06:52:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.682+0000 7f41e6800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41e015b1b0 0x7f41e00b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.682+0000 7f41e6800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f41e00ce5f0 0x7f41e0071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.682+0000 7f41e6800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e006f140 con 0x7f41e00af950 2024-03-07T06:52:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.682+0000 7f41e6800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e0071610 con 0x7f41e00ce5f0 2024-03-07T06:52:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.682+0000 7f41e6800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e006f420 con 0x7f41e015b1b0 2024-03-07T06:52:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.683+0000 7f41dfe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f41e00ce5f0 0x7f41e0071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.683+0000 7f41dfe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f41e00ce5f0 0x7f41e0071d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38662/0 (socket says 172.21.15.133:38662) 2024-03-07T06:52:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.683+0000 7f41d7e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f41e00af950 0x7f41e00cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33316/0 (socket says 172.21.15.133:33316) 2024-03-07T06:52:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.683+0000 7f41dfe00640 1 -- 172.21.15.133:0/654757013 learned_addr learned my addr 172.21.15.133:0/654757013 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:11.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.683+0000 7f41df400640 1 --2- 172.21.15.133:0/654757013 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41e015b1b0 0x7f41e00b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.683+0000 7f41dfe00640 1 -- 172.21.15.133:0/654757013 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41e015b1b0 msgr2=0x7f41e00b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.683+0000 7f41dfe00640 1 --2- 172.21.15.133:0/654757013 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41e015b1b0 0x7f41e00b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.683+0000 7f41dfe00640 1 -- 172.21.15.133:0/654757013 >> v1:172.21.15.133:6789/0 conn(0x7f41e00af950 legacy=0x7f41e00cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.683+0000 7f41dfe00640 1 -- 172.21.15.133:0/654757013 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e0071860 con 0x7f41e00ce5f0 2024-03-07T06:52:11.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.683+0000 7f41dea00640 1 -- 172.21.15.133:0/654757013 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4197687567 0 0) 0x7f41e006f140 con 0x7f41e00af950 2024-03-07T06:52:11.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.684+0000 7f41dfe00640 1 --2- 172.21.15.133:0/654757013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e00ce5f0 0x7f41e0071d80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009e50 tx=0x7f41d802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9fe9167082a13d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:11.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.685+0000 7f41dea00640 1 -- 172.21.15.133:0/654757013 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d802ca70 con 0x7f41e00ce5f0 2024-03-07T06:52:11.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.685+0000 7f41dea00640 1 -- 172.21.15.133:0/654757013 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d802cc10 con 0x7f41e00ce5f0 2024-03-07T06:52:11.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.685+0000 7f41dea00640 1 -- 172.21.15.133:0/654757013 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d80343f0 con 0x7f41e00ce5f0 2024-03-07T06:52:11.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.685+0000 7f41e6800640 1 -- 172.21.15.133:0/654757013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e00ce5f0 msgr2=0x7f41e0071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:11.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.685+0000 7f41e6800640 1 --2- 172.21.15.133:0/654757013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e00ce5f0 0x7f41e0071d80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009e50 tx=0x7f41d802bed0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.685+0000 7f41e6800640 1 -- 172.21.15.133:0/654757013 shutdown_connections 2024-03-07T06:52:11.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.685+0000 7f41e6800640 1 --2- 172.21.15.133:0/654757013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e00ce5f0 0x7f41e0071d80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.685+0000 7f41e6800640 1 --2- 172.21.15.133:0/654757013 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41e015b1b0 0x7f41e00b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.685+0000 7f41e6800640 1 -- 172.21.15.133:0/654757013 >> 172.21.15.133:0/654757013 conn(0x7f41e00b3120 msgr2=0x7f41e00b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:11.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.685+0000 7f41e6800640 1 -- 172.21.15.133:0/654757013 shutdown_connections 2024-03-07T06:52:11.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.686+0000 7f41e6800640 1 -- 172.21.15.133:0/654757013 wait complete. 2024-03-07T06:52:11.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.687+0000 7f41e6800640 1 Processor -- start 2024-03-07T06:52:11.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.687+0000 7f41e6800640 1 -- start start 2024-03-07T06:52:11.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.688+0000 7f41e6800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e00af950 0x7f41e0153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:11.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.688+0000 7f41e6800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41e00ce5f0 0x7f41e00aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:11.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.688+0000 7f41e6800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41e015b1b0 0x7f41e0157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:11.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.688+0000 7f41e6800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e015b590 con 0x7f41e00af950 2024-03-07T06:52:11.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.688+0000 7f41e6800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e006ee30 con 0x7f41e015b1b0 2024-03-07T06:52:11.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.688+0000 7f41e6800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e015b710 con 0x7f41e00ce5f0 2024-03-07T06:52:11.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.688+0000 7f41df400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e00af950 0x7f41e0153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:11.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.688+0000 7f41df400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e00af950 0x7f41e0153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38664/0 (socket says 172.21.15.133:38664) 2024-03-07T06:52:11.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.688+0000 7f41df400640 1 -- 172.21.15.133:0/423802969 learned_addr learned my addr 172.21.15.133:0/423802969 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:11.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.689+0000 7f41d7e00640 1 --2- 172.21.15.133:0/423802969 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41e00ce5f0 0x7f41e00aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:11.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.689+0000 7f41df400640 1 -- 172.21.15.133:0/423802969 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41e00ce5f0 msgr2=0x7f41e00aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:11.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.689+0000 7f41df400640 1 --2- 172.21.15.133:0/423802969 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41e00ce5f0 0x7f41e00aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.689+0000 7f41dfe00640 1 --2- 172.21.15.133:0/423802969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41e015b1b0 0x7f41e0157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:11.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.689+0000 7f41df400640 1 -- 172.21.15.133:0/423802969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41e015b1b0 msgr2=0x7f41e0157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:11.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.689+0000 7f41df400640 1 --2- 172.21.15.133:0/423802969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41e015b1b0 0x7f41e0157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.689+0000 7f41df400640 1 -- 172.21.15.133:0/423802969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e00af110 con 0x7f41e00af950 2024-03-07T06:52:11.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.689+0000 7f41df400640 1 --2- 172.21.15.133:0/423802969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e00af950 0x7f41e0153e40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f41d000d620 tx=0x7f41d000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:11.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.689+0000 7f41d7400640 1 -- 172.21.15.133:0/423802969 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000b620 con 0x7f41e00af950 2024-03-07T06:52:11.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.689+0000 7f41d7400640 1 -- 172.21.15.133:0/423802969 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d0019070 con 0x7f41e00af950 2024-03-07T06:52:11.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.689+0000 7f41d7400640 1 -- 172.21.15.133:0/423802969 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000ba00 con 0x7f41e00af950 2024-03-07T06:52:11.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.690+0000 7f41e6800640 1 -- 172.21.15.133:0/423802969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e0157ec0 con 0x7f41e00af950 2024-03-07T06:52:11.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.690+0000 7f41e6800640 1 -- 172.21.15.133:0/423802969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e00ae0f0 con 0x7f41e00af950 2024-03-07T06:52:11.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.691+0000 7f41e6800640 1 -- 172.21.15.133:0/423802969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e00b06a0 con 0x7f41e00af950 2024-03-07T06:52:11.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.692+0000 7f41d7400640 1 -- 172.21.15.133:0/423802969 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f41d000bba0 con 0x7f41e00af950 2024-03-07T06:52:11.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.693+0000 7f41d7400640 1 --2- 172.21.15.133:0/423802969 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f41bc06b4d0 0x7f41bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:11.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.693+0000 7f41d7400640 1 -- 172.21.15.133:0/423802969 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f41d008a760 con 0x7f41e00af950 2024-03-07T06:52:11.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.693+0000 7f41d7e00640 1 --2- 172.21.15.133:0/423802969 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f41bc06b4d0 0x7f41bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:11.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.694+0000 7f41d7e00640 1 --2- 172.21.15.133:0/423802969 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f41bc06b4d0 0x7f41bc06d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41c8007fd0 tx=0x7f41c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:11.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.697+0000 7f41d7400640 1 -- 172.21.15.133:0/423802969 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f41e00b06a0 con 0x7f41e00af950 2024-03-07T06:52:11.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.931+0000 7f41e6800640 1 -- 172.21.15.133:0/423802969 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41e00cc660 con 0x7f41e00af950 2024-03-07T06:52:11.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.932+0000 7f41d7400640 1 -- 172.21.15.133:0/423802969 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4983 (secure 0 0 0) 0x7f41e00af110 con 0x7f41e00af950 2024-03-07T06:52:11.931 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:11.931 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1116649196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1116649196},{"type":"v1","addr":"172.21.15.161:6835","nonce":1116649196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3485625105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3485625105},{"type":"v1","addr":"172.21.15.133:6835","nonce":3485625105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3866978630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3866978630},{"type":"v1","addr":"172.21.15.133:6833","nonce":3866978630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:51:14.594513+0000","modified":"2024-03-07T06:51:17.675044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2885273912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2885273912},{"type":"v1","addr":"172.21.15.161:6837","nonce":2885273912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-03-07T06:52:11.931 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 8 2024-03-07T06:52:11.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.934+0000 7f41e6800640 1 -- 172.21.15.133:0/423802969 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f41bc06b4d0 msgr2=0x7f41bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:11.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.934+0000 7f41e6800640 1 --2- 172.21.15.133:0/423802969 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f41bc06b4d0 0x7f41bc06d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f41c8007fd0 tx=0x7f41c8009040 comp rx=0 tx=0).stop 2024-03-07T06:52:11.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.934+0000 7f41e6800640 1 -- 172.21.15.133:0/423802969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e00af950 msgr2=0x7f41e0153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:11.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.934+0000 7f41e6800640 1 --2- 172.21.15.133:0/423802969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e00af950 0x7f41e0153e40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f41d000d620 tx=0x7f41d000dae0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.935+0000 7f41e6800640 1 -- 172.21.15.133:0/423802969 shutdown_connections 2024-03-07T06:52:11.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.935+0000 7f41e6800640 1 --2- 172.21.15.133:0/423802969 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f41bc06b4d0 0x7f41bc06d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.935+0000 7f41e6800640 1 --2- 172.21.15.133:0/423802969 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41e015b1b0 0x7f41e0157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.935+0000 7f41e6800640 1 --2- 172.21.15.133:0/423802969 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41e00ce5f0 0x7f41e00aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.935+0000 7f41e6800640 1 --2- 172.21.15.133:0/423802969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41e00af950 0x7f41e0153e40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:11.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.935+0000 7f41e6800640 1 -- 172.21.15.133:0/423802969 >> 172.21.15.133:0/423802969 conn(0x7f41e00b3120 msgr2=0x7f41e015a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:11.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.935+0000 7f41e6800640 1 -- 172.21.15.133:0/423802969 shutdown_connections 2024-03-07T06:52:11.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:11.935+0000 7f41e6800640 1 -- 172.21.15.133:0/423802969 wait complete. 2024-03-07T06:52:11.952 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:52:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.168+0000 7fa32a400640 1 Processor -- start 2024-03-07T06:52:12.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.168+0000 7fa32a400640 1 -- start start 2024-03-07T06:52:12.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.170+0000 7fa32a400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32415fbb0 0x7fa3240cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.170+0000 7fa32a400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa324072950 0x7fa3240cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.170+0000 7fa32a400640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3240731b0 con 0x7fa32415ff90 2024-03-07T06:52:12.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.170+0000 7fa32a400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa324072f60 con 0x7fa324072950 2024-03-07T06:52:12.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.170+0000 7fa32a400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa324073490 con 0x7fa32415fbb0 2024-03-07T06:52:12.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.170+0000 7fa323e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa32415ff90 0x7fa3240acd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33324/0 (socket says 172.21.15.133:33324) 2024-03-07T06:52:12.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.170+0000 7fa323e00640 1 -- 172.21.15.133:0/1789050452 learned_addr learned my addr 172.21.15.133:0/1789050452 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa323400640 1 --2- 172.21.15.133:0/1789050452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32415fbb0 0x7fa3240cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322a00640 1 --2- 172.21.15.133:0/1789050452 >> v2:172.21.15.161:3300/0 conn(0x7fa324072950 0x7fa3240cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 -- 172.21.15.133:0/1789050452 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474325766 0 0) 0x7fa3240731b0 con 0x7fa32415ff90 2024-03-07T06:52:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 -- 172.21.15.133:0/1789050452 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa308003520 con 0x7fa32415ff90 2024-03-07T06:52:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 -- 172.21.15.133:0/1789050452 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 778349792 0 0) 0x7fa308003520 con 0x7fa32415ff90 2024-03-07T06:52:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 -- 172.21.15.133:0/1789050452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32415fbb0 msgr2=0x7fa3240cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 --2- 172.21.15.133:0/1789050452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32415fbb0 0x7fa3240cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 -- 172.21.15.133:0/1789050452 >> v2:172.21.15.161:3300/0 conn(0x7fa324072950 msgr2=0x7fa3240cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 --2- 172.21.15.133:0/1789050452 >> v2:172.21.15.161:3300/0 conn(0x7fa324072950 0x7fa3240cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 -- 172.21.15.133:0/1789050452 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa324072150 con 0x7fa32415ff90 2024-03-07T06:52:12.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 -- 172.21.15.133:0/1789050452 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fa31c001670 con 0x7fa32415ff90 2024-03-07T06:52:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 -- 172.21.15.133:0/1789050452 >> v1:172.21.15.133:6789/0 conn(0x7fa32415ff90 legacy=0x7fa3240acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 --2- 172.21.15.133:0/1789050452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32415fbb0 0x7fa3080039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 --2- 172.21.15.133:0/1789050452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa308003f30 0x7fa308004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 --2- 172.21.15.133:0/1789050452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa308004d40 0x7fa3080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 -- 172.21.15.133:0/1789050452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa324072f60 con 0x7fa308003f30 2024-03-07T06:52:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 -- 172.21.15.133:0/1789050452 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa324073490 con 0x7fa308004d40 2024-03-07T06:52:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.171+0000 7fa322000640 1 -- 172.21.15.133:0/1789050452 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30800ba10 con 0x7fa32415fbb0 2024-03-07T06:52:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.172+0000 7fa322a00640 1 --2- 172.21.15.133:0/1789050452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa308003f30 0x7fa308004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.172+0000 7fa322a00640 1 -- 172.21.15.133:0/1789050452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32415fbb0 msgr2=0x7fa3080039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.172+0000 7fa322a00640 1 --2- 172.21.15.133:0/1789050452 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa32415fbb0 0x7fa3080039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.172+0000 7fa322a00640 1 -- 172.21.15.133:0/1789050452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa308004d40 msgr2=0x7fa3080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.172+0000 7fa322a00640 1 --2- 172.21.15.133:0/1789050452 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa308004d40 0x7fa3080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.172+0000 7fa322a00640 1 -- 172.21.15.133:0/1789050452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa308003520 con 0x7fa308003f30 2024-03-07T06:52:12.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.173+0000 7fa322a00640 1 --2- 172.21.15.133:0/1789050452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa308003f30 0x7fa308004800 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009af0 tx=0x7fa308005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a479305f933d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:12.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.173+0000 7fa322000640 1 -- 172.21.15.133:0/1789050452 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c039070 con 0x7fa308003f30 2024-03-07T06:52:12.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.173+0000 7fa322000640 1 -- 172.21.15.133:0/1789050452 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c02c710 con 0x7fa308003f30 2024-03-07T06:52:12.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.173+0000 7fa322000640 1 -- 172.21.15.133:0/1789050452 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c009690 con 0x7fa308003f30 2024-03-07T06:52:12.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.174+0000 7fa32a400640 1 -- 172.21.15.133:0/1789050452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa308003f30 msgr2=0x7fa308004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:12.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.174+0000 7fa32a400640 1 --2- 172.21.15.133:0/1789050452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa308003f30 0x7fa308004800 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009af0 tx=0x7fa308005830 comp rx=0 tx=0).stop 2024-03-07T06:52:12.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.174+0000 7fa323e00640 1 -- 172.21.15.133:0/1789050452 reap_dead start 2024-03-07T06:52:12.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.174+0000 7fa32a400640 1 -- 172.21.15.133:0/1789050452 shutdown_connections 2024-03-07T06:52:12.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.174+0000 7fa32a400640 1 -- 172.21.15.133:0/1789050452 >> 172.21.15.133:0/1789050452 conn(0x7fa3240b3120 msgr2=0x7fa3240b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:12.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.174+0000 7fa32a400640 1 -- 172.21.15.133:0/1789050452 shutdown_connections 2024-03-07T06:52:12.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.174+0000 7fa32a400640 1 -- 172.21.15.133:0/1789050452 wait complete. 2024-03-07T06:52:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.176+0000 7fa32a400640 1 Processor -- start 2024-03-07T06:52:12.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.176+0000 7fa32a400640 1 -- start start 2024-03-07T06:52:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.177+0000 7fa32a400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa308003f30 0x7fa32409d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.177+0000 7fa32a400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3241593f0 0x7fa32409be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.177+0000 7fa32a400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3240cbb10 0x7fa324158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.177+0000 7fa32a400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240b1a00 con 0x7fa308003f30 2024-03-07T06:52:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.177+0000 7fa32a400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa324160980 con 0x7fa3240cbb10 2024-03-07T06:52:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.177+0000 7fa32a400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3241603a0 con 0x7fa3241593f0 2024-03-07T06:52:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.177+0000 7fa323400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa308003f30 0x7fa32409d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.177+0000 7fa323400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa308003f30 0x7fa32409d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38690/0 (socket says 172.21.15.133:38690) 2024-03-07T06:52:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.177+0000 7fa323400640 1 -- 172.21.15.133:0/3116051411 learned_addr learned my addr 172.21.15.133:0/3116051411 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:12.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.178+0000 7fa322a00640 1 --2- 172.21.15.133:0/3116051411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3241593f0 0x7fa32409be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.178+0000 7fa323e00640 1 --2- 172.21.15.133:0/3116051411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3240cbb10 0x7fa324158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.178+0000 7fa323400640 1 -- 172.21.15.133:0/3116051411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3241593f0 msgr2=0x7fa32409be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.178+0000 7fa323400640 1 --2- 172.21.15.133:0/3116051411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3241593f0 0x7fa32409be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.178+0000 7fa323400640 1 -- 172.21.15.133:0/3116051411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3240cbb10 msgr2=0x7fa324158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.178+0000 7fa323400640 1 --2- 172.21.15.133:0/3116051411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3240cbb10 0x7fa324158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.178+0000 7fa323400640 1 -- 172.21.15.133:0/3116051411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32409c3c0 con 0x7fa308003f30 2024-03-07T06:52:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.178+0000 7fa323400640 1 --2- 172.21.15.133:0/3116051411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa308003f30 0x7fa32409d580 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa31400ef10 tx=0x7fa31400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.178+0000 7fa31b400640 1 -- 172.21.15.133:0/3116051411 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa314018020 con 0x7fa308003f30 2024-03-07T06:52:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.178+0000 7fa31b400640 1 -- 172.21.15.133:0/3116051411 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa314014400 con 0x7fa308003f30 2024-03-07T06:52:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.178+0000 7fa31b400640 1 -- 172.21.15.133:0/3116051411 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3140147d0 con 0x7fa308003f30 2024-03-07T06:52:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.178+0000 7fa32a400640 1 -- 172.21.15.133:0/3116051411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32409b3e0 con 0x7fa308003f30 2024-03-07T06:52:12.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.178+0000 7fa32a400640 1 -- 172.21.15.133:0/3116051411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32409b680 con 0x7fa308003f30 2024-03-07T06:52:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.180+0000 7fa32a400640 1 -- 172.21.15.133:0/3116051411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa324062c90 con 0x7fa308003f30 2024-03-07T06:52:12.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.180+0000 7fa31b400640 1 -- 172.21.15.133:0/3116051411 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa314005d10 con 0x7fa308003f30 2024-03-07T06:52:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.181+0000 7fa31b400640 1 --2- 172.21.15.133:0/3116051411 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa30006b500 0x7fa30006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.181+0000 7fa31b400640 1 -- 172.21.15.133:0/3116051411 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fa31408ab80 con 0x7fa308003f30 2024-03-07T06:52:12.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.182+0000 7fa322a00640 1 --2- 172.21.15.133:0/3116051411 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa30006b500 0x7fa30006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:12.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.183+0000 7fa322a00640 1 --2- 172.21.15.133:0/3116051411 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa30006b500 0x7fa30006d9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa30c036000 tx=0x7fa30c030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:12.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.186+0000 7fa31b400640 1 -- 172.21.15.133:0/3116051411 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa314022030 con 0x7fa308003f30 2024-03-07T06:52:12.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.383+0000 7fa32a400640 1 -- 172.21.15.133:0/3116051411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa324072d40 con 0x7fa308003f30 2024-03-07T06:52:12.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.385+0000 7fa31b400640 1 -- 172.21.15.133:0/3116051411 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16901 (secure 0 0 0) 0x7fa314059970 con 0x7fa308003f30 2024-03-07T06:52:12.384 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:12.384 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:56.413096+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:52:12.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.388+0000 7fa32a400640 1 -- 172.21.15.133:0/3116051411 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa30006b500 msgr2=0x7fa30006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:12.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.388+0000 7fa32a400640 1 --2- 172.21.15.133:0/3116051411 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa30006b500 0x7fa30006d9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa30c036000 tx=0x7fa30c030000 comp rx=0 tx=0).stop 2024-03-07T06:52:12.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.388+0000 7fa32a400640 1 -- 172.21.15.133:0/3116051411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa308003f30 msgr2=0x7fa32409d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:12.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.388+0000 7fa32a400640 1 --2- 172.21.15.133:0/3116051411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa308003f30 0x7fa32409d580 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa31400ef10 tx=0x7fa31400b480 comp rx=0 tx=0).stop 2024-03-07T06:52:12.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.388+0000 7fa32a400640 1 -- 172.21.15.133:0/3116051411 shutdown_connections 2024-03-07T06:52:12.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.388+0000 7fa32a400640 1 --2- 172.21.15.133:0/3116051411 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa30006b500 0x7fa30006d9c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.388+0000 7fa32a400640 1 --2- 172.21.15.133:0/3116051411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3240cbb10 0x7fa324158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.388+0000 7fa32a400640 1 --2- 172.21.15.133:0/3116051411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3241593f0 0x7fa32409be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.388+0000 7fa32a400640 1 --2- 172.21.15.133:0/3116051411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa308003f30 0x7fa32409d580 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.388+0000 7fa32a400640 1 -- 172.21.15.133:0/3116051411 >> 172.21.15.133:0/3116051411 conn(0x7fa3240b3120 msgr2=0x7fa324074300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:12.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.388+0000 7fa32a400640 1 -- 172.21.15.133:0/3116051411 shutdown_connections 2024-03-07T06:52:12.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.388+0000 7fa32a400640 1 -- 172.21.15.133:0/3116051411 wait complete. 2024-03-07T06:52:12.406 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:51:12.858333+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:51:13.615348+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:52:12.407 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:52:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.620+0000 7fe65ca00640 1 Processor -- start 2024-03-07T06:52:12.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.620+0000 7fe65ca00640 1 -- start start 2024-03-07T06:52:12.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.622+0000 7fe65ca00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe6580b4530 0x7fe6580b5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.622+0000 7fe65ca00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe658147540 0x7fe658071a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.622+0000 7fe65ca00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe658147cb0 con 0x7fe658146fe0 2024-03-07T06:52:12.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.622+0000 7fe65ca00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe658161ef0 con 0x7fe6580b4530 2024-03-07T06:52:12.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.622+0000 7fe65ca00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6581621d0 con 0x7fe658147540 2024-03-07T06:52:12.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.622+0000 7fe655400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe6580b4530 0x7fe6580b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:12.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.622+0000 7fe655400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe6580b4530 0x7fe6580b5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38698/0 (socket says 172.21.15.133:38698) 2024-03-07T06:52:12.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.622+0000 7fe655400640 1 -- 172.21.15.133:0/1210569668 learned_addr learned my addr 172.21.15.133:0/1210569668 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.623+0000 7fe656800640 1 --2- 172.21.15.133:0/1210569668 >> v2:172.21.15.161:3300/0 conn(0x7fe658147540 0x7fe658071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.623+0000 7fe655400640 1 -- 172.21.15.133:0/1210569668 >> v2:172.21.15.161:3300/0 conn(0x7fe658147540 msgr2=0x7fe658071a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.623+0000 7fe655400640 1 --2- 172.21.15.133:0/1210569668 >> v2:172.21.15.161:3300/0 conn(0x7fe658147540 0x7fe658071a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.623+0000 7fe655400640 1 -- 172.21.15.133:0/1210569668 >> v1:172.21.15.161:6789/0 conn(0x7fe658146fe0 legacy=0x7fe6580b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.623+0000 7fe655400640 1 -- 172.21.15.133:0/1210569668 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6581624b0 con 0x7fe6580b4530 2024-03-07T06:52:12.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.623+0000 7fe656800640 1 --2- 172.21.15.133:0/1210569668 >> v2:172.21.15.161:3300/0 conn(0x7fe658147540 0x7fe658071a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.624+0000 7fe655400640 1 --2- 172.21.15.133:0/1210569668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6580b4530 0x7fe6580b5830 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe658073130 tx=0x7fe64002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94650b901a226b62 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.624+0000 7fe654a00640 1 -- 172.21.15.133:0/1210569668 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640035020 con 0x7fe6580b4530 2024-03-07T06:52:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.624+0000 7fe654a00640 1 -- 172.21.15.133:0/1210569668 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe640031870 con 0x7fe6580b4530 2024-03-07T06:52:12.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.624+0000 7fe654a00640 1 -- 172.21.15.133:0/1210569668 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640031c40 con 0x7fe6580b4530 2024-03-07T06:52:12.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.625+0000 7fe65ca00640 1 -- 172.21.15.133:0/1210569668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6580b4530 msgr2=0x7fe6580b5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:12.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.625+0000 7fe65ca00640 1 --2- 172.21.15.133:0/1210569668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6580b4530 0x7fe6580b5830 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe658073130 tx=0x7fe64002b910 comp rx=0 tx=0).stop 2024-03-07T06:52:12.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.625+0000 7fe65ca00640 1 -- 172.21.15.133:0/1210569668 shutdown_connections 2024-03-07T06:52:12.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.625+0000 7fe65ca00640 1 --2- 172.21.15.133:0/1210569668 >> v2:172.21.15.161:3300/0 conn(0x7fe658147540 0x7fe658071a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.625+0000 7fe65ca00640 1 --2- 172.21.15.133:0/1210569668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6580b4530 0x7fe6580b5830 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.625+0000 7fe65ca00640 1 -- 172.21.15.133:0/1210569668 >> 172.21.15.133:0/1210569668 conn(0x7fe6580705e0 msgr2=0x7fe6580709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:12.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.625+0000 7fe65ca00640 1 -- 172.21.15.133:0/1210569668 shutdown_connections 2024-03-07T06:52:12.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.625+0000 7fe65ca00640 1 -- 172.21.15.133:0/1210569668 wait complete. 2024-03-07T06:52:12.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.627+0000 7fe65ca00640 1 Processor -- start 2024-03-07T06:52:12.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.627+0000 7fe65ca00640 1 -- start start 2024-03-07T06:52:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.628+0000 7fe65ca00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6580b4530 0x7fe6581e9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.628+0000 7fe65ca00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe658146fe0 0x7fe658159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.628+0000 7fe65ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe658147540 0x7fe65815ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.628+0000 7fe65ca00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe658162cb0 con 0x7fe658147540 2024-03-07T06:52:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.628+0000 7fe65ca00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe658147970 con 0x7fe658146fe0 2024-03-07T06:52:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.628+0000 7fe65ca00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65815f800 con 0x7fe6580b4530 2024-03-07T06:52:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.628+0000 7fe656800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe658147540 0x7fe65815ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.628+0000 7fe656800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe658147540 0x7fe65815ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38700/0 (socket says 172.21.15.133:38700) 2024-03-07T06:52:12.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.628+0000 7fe656800640 1 -- 172.21.15.133:0/3210579266 learned_addr learned my addr 172.21.15.133:0/3210579266 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:12.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe655e00640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6580b4530 0x7fe6581e9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe655400640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe658146fe0 0x7fe658159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe656800640 1 -- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6580b4530 msgr2=0x7fe6581e9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe656800640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6580b4530 0x7fe6581e9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe656800640 1 -- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe658146fe0 msgr2=0x7fe658159430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe656800640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe658146fe0 0x7fe658159430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe656800640 1 -- 172.21.15.133:0/3210579266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6581e89c0 con 0x7fe658147540 2024-03-07T06:52:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe655e00640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6580b4530 0x7fe6581e9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe655400640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe658146fe0 0x7fe658159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe656800640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe658147540 0x7fe65815ecf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe65000c7f0 tx=0x7fe65000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe64e000640 1 -- 172.21.15.133:0/3210579266 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650015070 con 0x7fe658147540 2024-03-07T06:52:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe64e000640 1 -- 172.21.15.133:0/3210579266 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe650007820 con 0x7fe658147540 2024-03-07T06:52:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe64e000640 1 -- 172.21.15.133:0/3210579266 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650007c50 con 0x7fe658147540 2024-03-07T06:52:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe65ca00640 1 -- 172.21.15.133:0/3210579266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6581e8ca0 con 0x7fe658147540 2024-03-07T06:52:12.629 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.629+0000 7fe65ca00640 1 -- 172.21.15.133:0/3210579266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6581e9260 con 0x7fe658147540 2024-03-07T06:52:12.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.630+0000 7fe65ca00640 1 -- 172.21.15.133:0/3210579266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe658062c90 con 0x7fe658147540 2024-03-07T06:52:12.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.631+0000 7fe64e000640 1 -- 172.21.15.133:0/3210579266 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe650018070 con 0x7fe658147540 2024-03-07T06:52:12.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.632+0000 7fe64e000640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe63406b500 0x7fe63406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:12.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.632+0000 7fe64e000640 1 -- 172.21.15.133:0/3210579266 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe65008a3f0 con 0x7fe658147540 2024-03-07T06:52:12.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.633+0000 7fe655e00640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe63406b500 0x7fe63406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:12.632 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.634+0000 7fe655e00640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe63406b500 0x7fe63406d9c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe648004520 tx=0x7fe648009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:12.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.637+0000 7fe64e000640 1 -- 172.21.15.133:0/3210579266 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe6500591e0 con 0x7fe658147540 2024-03-07T06:52:12.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.830+0000 7fe65ca00640 1 -- 172.21.15.133:0/3210579266 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe65806dc20 con 0x7fe658147540 2024-03-07T06:52:12.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.831+0000 7fe64e000640 1 -- 172.21.15.133:0/3210579266 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe650058ae0 con 0x7fe658147540 2024-03-07T06:52:12.829 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:52:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.833+0000 7fe65ca00640 1 -- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe63406b500 msgr2=0x7fe63406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.833+0000 7fe65ca00640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe63406b500 0x7fe63406d9c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe648004520 tx=0x7fe648009040 comp rx=0 tx=0).stop 2024-03-07T06:52:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.833+0000 7fe65ca00640 1 -- 172.21.15.133:0/3210579266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe658147540 msgr2=0x7fe65815ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.833+0000 7fe65ca00640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe658147540 0x7fe65815ecf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe65000c7f0 tx=0x7fe65000ccb0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.834+0000 7fe65ca00640 1 -- 172.21.15.133:0/3210579266 shutdown_connections 2024-03-07T06:52:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.834+0000 7fe65ca00640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe658147540 0x7fe65815ecf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.834+0000 7fe65ca00640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe63406b500 0x7fe63406d9c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.834+0000 7fe65ca00640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe658146fe0 0x7fe658159430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.834+0000 7fe65ca00640 1 --2- 172.21.15.133:0/3210579266 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe6580b4530 0x7fe6581e9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.834+0000 7fe65ca00640 1 -- 172.21.15.133:0/3210579266 >> 172.21.15.133:0/3210579266 conn(0x7fe6580705e0 msgr2=0x7fe658071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:12.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.834+0000 7fe65ca00640 1 -- 172.21.15.133:0/3210579266 shutdown_connections 2024-03-07T06:52:12.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:12.834+0000 7fe65ca00640 1 -- 172.21.15.133:0/3210579266 wait complete. 2024-03-07T06:52:12.851 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:52:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.069+0000 7f0793000640 1 Processor -- start 2024-03-07T06:52:13.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.069+0000 7f0793000640 1 -- start start 2024-03-07T06:52:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.070+0000 7f0793000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c0b3ed0 0x7f078c15b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.070+0000 7f0793000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f078c06d180 0x7f078c15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.070+0000 7f0793000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078c06d740 con 0x7f078c15b370 2024-03-07T06:52:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.070+0000 7f0793000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f078c15c1d0 con 0x7f078c06d180 2024-03-07T06:52:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.070+0000 7f0793000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078c06da20 con 0x7f078c0b3ed0 2024-03-07T06:52:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.071+0000 7f0791000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f078c06d180 0x7f078c15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.071+0000 7f078be00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f078c15b370 0x7f078c0cef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33338/0 (socket says 172.21.15.133:33338) 2024-03-07T06:52:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.071+0000 7f0791000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f078c06d180 0x7f078c15bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38710/0 (socket says 172.21.15.133:38710) 2024-03-07T06:52:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.071+0000 7f078be00640 1 -- 172.21.15.133:0/3971624333 learned_addr learned my addr 172.21.15.133:0/3971624333 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:13.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.071+0000 7f0783400640 1 --2- 172.21.15.133:0/3971624333 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c0b3ed0 0x7f078c15b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.071+0000 7f0791000640 1 -- 172.21.15.133:0/3971624333 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c0b3ed0 msgr2=0x7f078c15b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.071+0000 7f0791000640 1 --2- 172.21.15.133:0/3971624333 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c0b3ed0 0x7f078c15b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.071+0000 7f0791000640 1 -- 172.21.15.133:0/3971624333 >> v1:172.21.15.133:6789/0 conn(0x7f078c15b370 legacy=0x7f078c0cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.071+0000 7f0791000640 1 -- 172.21.15.133:0/3971624333 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c06dd00 con 0x7f078c06d180 2024-03-07T06:52:13.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.071+0000 7f078b400640 1 -- 172.21.15.133:0/3971624333 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034425305 0 0) 0x7f078c06d740 con 0x7f078c15b370 2024-03-07T06:52:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.072+0000 7f0791000640 1 --2- 172.21.15.133:0/3971624333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078c06d180 0x7f078c15bc90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f078400bfa0 tx=0x7f078402e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314f6218e025dc1b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.073+0000 7f078b400640 1 -- 172.21.15.133:0/3971624333 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078402ea40 con 0x7f078c06d180 2024-03-07T06:52:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.073+0000 7f078b400640 1 -- 172.21.15.133:0/3971624333 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078402ebe0 con 0x7f078c06d180 2024-03-07T06:52:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.073+0000 7f078b400640 1 -- 172.21.15.133:0/3971624333 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784009540 con 0x7f078c06d180 2024-03-07T06:52:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.073+0000 7f0793000640 1 -- 172.21.15.133:0/3971624333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078c06d180 msgr2=0x7f078c15bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.073+0000 7f0793000640 1 --2- 172.21.15.133:0/3971624333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078c06d180 0x7f078c15bc90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f078400bfa0 tx=0x7f078402e0a0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.073+0000 7f0793000640 1 -- 172.21.15.133:0/3971624333 shutdown_connections 2024-03-07T06:52:13.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.073+0000 7f0793000640 1 --2- 172.21.15.133:0/3971624333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078c06d180 0x7f078c15bc90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.073+0000 7f0793000640 1 --2- 172.21.15.133:0/3971624333 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c0b3ed0 0x7f078c15b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.073+0000 7f0793000640 1 -- 172.21.15.133:0/3971624333 >> 172.21.15.133:0/3971624333 conn(0x7f078c0b1d30 msgr2=0x7f078c0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:13.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.074+0000 7f0793000640 1 -- 172.21.15.133:0/3971624333 shutdown_connections 2024-03-07T06:52:13.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.074+0000 7f0793000640 1 -- 172.21.15.133:0/3971624333 wait complete. 2024-03-07T06:52:13.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.075+0000 7f0793000640 1 Processor -- start 2024-03-07T06:52:13.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.075+0000 7f0793000640 1 -- start start 2024-03-07T06:52:13.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.076+0000 7f0793000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c06d180 0x7f078c143040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.076+0000 7f0793000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f078c0b3ed0 0x7f078c159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.076+0000 7f0793000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078c15b370 0x7f078c143990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.076+0000 7f0793000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c0cf660 con 0x7f078c15b370 2024-03-07T06:52:13.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.076+0000 7f0793000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c06e2c0 con 0x7f078c0b3ed0 2024-03-07T06:52:13.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.076+0000 7f0793000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078c0cf7e0 con 0x7f078c06d180 2024-03-07T06:52:13.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.077+0000 7f0791000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078c15b370 0x7f078c143990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.077+0000 7f0791000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078c15b370 0x7f078c143990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38716/0 (socket says 172.21.15.133:38716) 2024-03-07T06:52:13.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.077+0000 7f0791000640 1 -- 172.21.15.133:0/1469767418 learned_addr learned my addr 172.21.15.133:0/1469767418 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:13.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.077+0000 7f0783400640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f078c0b3ed0 0x7f078c159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.077+0000 7f078be00640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c06d180 0x7f078c143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.077+0000 7f0791000640 1 -- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c06d180 msgr2=0x7f078c143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.077+0000 7f0791000640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c06d180 0x7f078c143040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.077+0000 7f0791000640 1 -- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f078c0b3ed0 msgr2=0x7f078c159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.077+0000 7f0791000640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f078c0b3ed0 0x7f078c159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.077+0000 7f0791000640 1 -- 172.21.15.133:0/1469767418 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c1468c0 con 0x7f078c15b370 2024-03-07T06:52:13.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.077+0000 7f0783400640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f078c0b3ed0 0x7f078c159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:13.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.077+0000 7f0791000640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078c15b370 0x7f078c143990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0784038040 tx=0x7f0784004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:13.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.078+0000 7f0788c00640 1 -- 172.21.15.133:0/1469767418 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784040620 con 0x7f078c15b370 2024-03-07T06:52:13.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.078+0000 7f0788c00640 1 -- 172.21.15.133:0/1469767418 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0784045070 con 0x7f078c15b370 2024-03-07T06:52:13.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.078+0000 7f0788c00640 1 -- 172.21.15.133:0/1469767418 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784040a20 con 0x7f078c15b370 2024-03-07T06:52:13.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.078+0000 7f0793000640 1 -- 172.21.15.133:0/1469767418 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c158ad0 con 0x7f078c15b370 2024-03-07T06:52:13.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.078+0000 7f0793000640 1 -- 172.21.15.133:0/1469767418 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c159090 con 0x7f078c15b370 2024-03-07T06:52:13.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.079+0000 7f0793000640 1 -- 172.21.15.133:0/1469767418 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078c062c90 con 0x7f078c15b370 2024-03-07T06:52:13.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.080+0000 7f0788c00640 1 -- 172.21.15.133:0/1469767418 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0784040bc0 con 0x7f078c15b370 2024-03-07T06:52:13.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.081+0000 7f0788c00640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f076806b500 0x7f076806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.081+0000 7f0788c00640 1 -- 172.21.15.133:0/1469767418 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f07840ad1e0 con 0x7f078c15b370 2024-03-07T06:52:13.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.082+0000 7f078be00640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f076806b500 0x7f076806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.083+0000 7f078be00640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f076806b500 0x7f076806d9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f077c000f60 tx=0x7f077c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:13.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.085+0000 7f0788c00640 1 -- 172.21.15.133:0/1469767418 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f078407c060 con 0x7f078c15b370 2024-03-07T06:52:13.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.279+0000 7f0793000640 1 -- 172.21.15.133:0/1469767418 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f078c072690 con 0x7f078c15b370 2024-03-07T06:52:13.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.280+0000 7f0788c00640 1 -- 172.21.15.133:0/1469767418 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f078407b960 con 0x7f078c15b370 2024-03-07T06:52:13.279 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:52:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.283+0000 7f0793000640 1 -- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f076806b500 msgr2=0x7f076806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.283+0000 7f0793000640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f076806b500 0x7f076806d9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f077c000f60 tx=0x7f077c009040 comp rx=0 tx=0).stop 2024-03-07T06:52:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.283+0000 7f0793000640 1 -- 172.21.15.133:0/1469767418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078c15b370 msgr2=0x7f078c143990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.283+0000 7f0793000640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078c15b370 0x7f078c143990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0784038040 tx=0x7f0784004a40 comp rx=0 tx=0).stop 2024-03-07T06:52:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.283+0000 7f0793000640 1 -- 172.21.15.133:0/1469767418 shutdown_connections 2024-03-07T06:52:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.283+0000 7f0793000640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f078c15b370 0x7f078c143990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.283+0000 7f0793000640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f076806b500 0x7f076806d9c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.283+0000 7f0793000640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f078c0b3ed0 0x7f078c159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.283+0000 7f0793000640 1 --2- 172.21.15.133:0/1469767418 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f078c06d180 0x7f078c143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.283+0000 7f0793000640 1 -- 172.21.15.133:0/1469767418 >> 172.21.15.133:0/1469767418 conn(0x7f078c0b1d30 msgr2=0x7f078c0728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.283+0000 7f0793000640 1 -- 172.21.15.133:0/1469767418 shutdown_connections 2024-03-07T06:52:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.283+0000 7f0793000640 1 -- 172.21.15.133:0/1469767418 wait complete. 2024-03-07T06:52:13.301 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:52:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.517+0000 7f3cf4400640 1 Processor -- start 2024-03-07T06:52:13.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.517+0000 7f3cf4400640 1 -- start start 2024-03-07T06:52:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.519+0000 7f3cf4400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3cec159ab0 0x7f3cec15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.519+0000 7f3cf4400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cec0b3eb0 0x7f3cec0d2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.519+0000 7f3cf4400640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cec15d430 con 0x7f3cec0b3a20 2024-03-07T06:52:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.519+0000 7f3cf4400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cec159e90 con 0x7f3cec159ab0 2024-03-07T06:52:13.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.519+0000 7f3cf4400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec0b6080 con 0x7f3cec0b3eb0 2024-03-07T06:52:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.519+0000 7f3cf1000640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3cec0b3a20 0x7f3cec0d1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33340/0 (socket says 172.21.15.133:33340) 2024-03-07T06:52:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.519+0000 7f3cf1000640 1 -- 172.21.15.133:0/918946159 learned_addr learned my addr 172.21.15.133:0/918946159 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.520+0000 7f3cf2400640 1 --2- 172.21.15.133:0/918946159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cec0b3eb0 0x7f3cec0d2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.520+0000 7f3cf1a00640 1 --2- 172.21.15.133:0/918946159 >> v2:172.21.15.161:3300/0 conn(0x7f3cec159ab0 0x7f3cec15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.520+0000 7f3cebe00640 1 -- 172.21.15.133:0/918946159 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937743461 0 0) 0x7f3cdc001110 con 0x7f3cec0b3a20 2024-03-07T06:52:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.520+0000 7f3cebe00640 1 -- 172.21.15.133:0/918946159 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cd4003520 con 0x7f3cec0b3a20 2024-03-07T06:52:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.520+0000 7f3cf2400640 1 -- 172.21.15.133:0/918946159 >> v2:172.21.15.161:3300/0 conn(0x7f3cec159ab0 msgr2=0x7f3cec15cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:13.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.520+0000 7f3cf2400640 1 --2- 172.21.15.133:0/918946159 >> v2:172.21.15.161:3300/0 conn(0x7f3cec159ab0 0x7f3cec15cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.520+0000 7f3cf2400640 1 -- 172.21.15.133:0/918946159 >> v1:172.21.15.133:6789/0 conn(0x7f3cec0b3a20 legacy=0x7f3cec0d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.520+0000 7f3cf2400640 1 -- 172.21.15.133:0/918946159 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec15fde0 con 0x7f3cec0b3eb0 2024-03-07T06:52:13.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.522+0000 7f3cf2400640 1 --2- 172.21.15.133:0/918946159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cec0b3eb0 0x7f3cec0d2440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009d90 tx=0x7f3ce402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d82eafe6624861 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:13.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.522+0000 7f3cebe00640 1 -- 172.21.15.133:0/918946159 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4039070 con 0x7f3cec0b3eb0 2024-03-07T06:52:13.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.522+0000 7f3cebe00640 1 -- 172.21.15.133:0/918946159 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce402cbc0 con 0x7f3cec0b3eb0 2024-03-07T06:52:13.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.522+0000 7f3cebe00640 1 -- 172.21.15.133:0/918946159 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4034440 con 0x7f3cec0b3eb0 2024-03-07T06:52:13.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.523+0000 7f3cf4400640 1 -- 172.21.15.133:0/918946159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cec0b3eb0 msgr2=0x7f3cec0d2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.523+0000 7f3cf4400640 1 --2- 172.21.15.133:0/918946159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cec0b3eb0 0x7f3cec0d2440 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009d90 tx=0x7f3ce402be10 comp rx=0 tx=0).stop 2024-03-07T06:52:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.523+0000 7f3cf4400640 1 -- 172.21.15.133:0/918946159 shutdown_connections 2024-03-07T06:52:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.523+0000 7f3cf4400640 1 --2- 172.21.15.133:0/918946159 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cec0b3eb0 0x7f3cec0d2440 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.523+0000 7f3cf4400640 1 --2- 172.21.15.133:0/918946159 >> v2:172.21.15.161:3300/0 conn(0x7f3cec159ab0 0x7f3cec15cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.523+0000 7f3cf4400640 1 -- 172.21.15.133:0/918946159 >> 172.21.15.133:0/918946159 conn(0x7f3cec070400 msgr2=0x7f3cec0707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.523+0000 7f3cf4400640 1 -- 172.21.15.133:0/918946159 shutdown_connections 2024-03-07T06:52:13.522 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.523+0000 7f3cf4400640 1 -- 172.21.15.133:0/918946159 wait complete. 2024-03-07T06:52:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.525+0000 7f3cf4400640 1 Processor -- start 2024-03-07T06:52:13.523 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.525+0000 7f3cf4400640 1 -- start start 2024-03-07T06:52:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.526+0000 7f3cf4400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cec0b3a20 0x7f3cec1491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.526+0000 7f3cf4400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cec0b3eb0 0x7f3cec15b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.526+0000 7f3cf4400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cec159ab0 0x7f3cec072e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.526+0000 7f3cf4400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec160680 con 0x7f3cec159ab0 2024-03-07T06:52:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.526+0000 7f3cf4400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec1603a0 con 0x7f3cec0b3a20 2024-03-07T06:52:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.526+0000 7f3cf4400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec0b42e0 con 0x7f3cec0b3eb0 2024-03-07T06:52:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.526+0000 7f3cf2400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cec159ab0 0x7f3cec072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.526+0000 7f3cf2400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cec159ab0 0x7f3cec072e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38732/0 (socket says 172.21.15.133:38732) 2024-03-07T06:52:13.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.526+0000 7f3cf2400640 1 -- 172.21.15.133:0/37476391 learned_addr learned my addr 172.21.15.133:0/37476391 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.526+0000 7f3cf1000640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cec0b3eb0 0x7f3cec15b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.526+0000 7f3cf1a00640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cec0b3a20 0x7f3cec1491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.527+0000 7f3cf2400640 1 -- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cec0b3eb0 msgr2=0x7f3cec15b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.527+0000 7f3cf2400640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cec0b3eb0 0x7f3cec15b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.527+0000 7f3cf2400640 1 -- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cec0b3a20 msgr2=0x7f3cec1491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.527+0000 7f3cf2400640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cec0b3a20 0x7f3cec1491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.527+0000 7f3cf2400640 1 -- 172.21.15.133:0/37476391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec1f47b0 con 0x7f3cec159ab0 2024-03-07T06:52:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.527+0000 7f3cf1a00640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cec0b3a20 0x7f3cec1491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.527+0000 7f3cf2400640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cec159ab0 0x7f3cec072e10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4037040 tx=0x7f3ce4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.527+0000 7f3ce9600640 1 -- 172.21.15.133:0/37476391 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4039050 con 0x7f3cec159ab0 2024-03-07T06:52:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.527+0000 7f3ce9600640 1 -- 172.21.15.133:0/37476391 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce403a040 con 0x7f3cec159ab0 2024-03-07T06:52:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.527+0000 7f3cf4400640 1 -- 172.21.15.133:0/37476391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cec1f4a90 con 0x7f3cec159ab0 2024-03-07T06:52:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.527+0000 7f3ce9600640 1 -- 172.21.15.133:0/37476391 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4009450 con 0x7f3cec159ab0 2024-03-07T06:52:13.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.527+0000 7f3cf4400640 1 -- 172.21.15.133:0/37476391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cec1f5050 con 0x7f3cec159ab0 2024-03-07T06:52:13.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.529+0000 7f3cf4400640 1 -- 172.21.15.133:0/37476391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cec06cdd0 con 0x7f3cec159ab0 2024-03-07T06:52:13.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.529+0000 7f3ce9600640 1 -- 172.21.15.133:0/37476391 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3ce4047020 con 0x7f3cec159ab0 2024-03-07T06:52:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.531+0000 7f3ce9600640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3cd006b4d0 0x7f3cd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.531+0000 7f3ce9600640 1 -- 172.21.15.133:0/37476391 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3ce40abfe0 con 0x7f3cec159ab0 2024-03-07T06:52:13.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.531+0000 7f3cf1a00640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3cd006b4d0 0x7f3cd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.532+0000 7f3cf1a00640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3cd006b4d0 0x7f3cd006d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009a20 tx=0x7f3ce0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:13.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.534+0000 7f3ce9600640 1 -- 172.21.15.133:0/37476391 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3cec06cdd0 con 0x7f3cec159ab0 2024-03-07T06:52:13.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.729+0000 7f3cf4400640 1 -- 172.21.15.133:0/37476391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3cec067c80 con 0x7f3cec159ab0 2024-03-07T06:52:13.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.730+0000 7f3ce9600640 1 -- 172.21.15.133:0/37476391 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ce407aef0 con 0x7f3cec159ab0 2024-03-07T06:52:13.729 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:52:13.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.733+0000 7f3cf4400640 1 -- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3cd006b4d0 msgr2=0x7f3cd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.733+0000 7f3cf4400640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3cd006b4d0 0x7f3cd006d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0009a20 tx=0x7f3ce0008040 comp rx=0 tx=0).stop 2024-03-07T06:52:13.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.733+0000 7f3cf4400640 1 -- 172.21.15.133:0/37476391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cec159ab0 msgr2=0x7f3cec072e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.733+0000 7f3cf4400640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cec159ab0 0x7f3cec072e10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4037040 tx=0x7f3ce4004a40 comp rx=0 tx=0).stop 2024-03-07T06:52:13.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.733+0000 7f3cf4400640 1 -- 172.21.15.133:0/37476391 shutdown_connections 2024-03-07T06:52:13.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.733+0000 7f3cf4400640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cec159ab0 0x7f3cec072e10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.733+0000 7f3cf4400640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cec0b3eb0 0x7f3cec15b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.733+0000 7f3cf4400640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3cd006b4d0 0x7f3cd006d990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.733+0000 7f3cf4400640 1 --2- 172.21.15.133:0/37476391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cec0b3a20 0x7f3cec1491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.733+0000 7f3cf4400640 1 -- 172.21.15.133:0/37476391 >> 172.21.15.133:0/37476391 conn(0x7f3cec070400 msgr2=0x7f3cec0b4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:13.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.733+0000 7f3cf4400640 1 -- 172.21.15.133:0/37476391 shutdown_connections 2024-03-07T06:52:13.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.733+0000 7f3cf4400640 1 -- 172.21.15.133:0/37476391 wait complete. 2024-03-07T06:52:13.749 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:52:13.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.945+0000 7f873a000640 1 Processor -- start 2024-03-07T06:52:13.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.945+0000 7f873a000640 1 -- start start 2024-03-07T06:52:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.946+0000 7f873a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f873415fbb0 0x7f87340cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.947+0000 7f873a000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87340731b0 con 0x7f8734072950 2024-03-07T06:52:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.947+0000 7f873a000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8734072150 con 0x7f87340cdee0 2024-03-07T06:52:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.947+0000 7f873a000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8734072f60 con 0x7f873415fbb0 2024-03-07T06:52:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.947+0000 7f872be00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8734072950 0x7f873415ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33350/0 (socket says 172.21.15.133:33350) 2024-03-07T06:52:13.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.947+0000 7f872be00640 1 -- 172.21.15.133:0/1201922931 learned_addr learned my addr 172.21.15.133:0/1201922931 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.947+0000 7f8733400640 1 --2- 172.21.15.133:0/1201922931 >> v2:172.21.15.161:3300/0 conn(0x7f873415fbb0 0x7f87340cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.947+0000 7f8732a00640 1 -- 172.21.15.133:0/1201922931 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2683799784 0 0) 0x7f87340731b0 con 0x7f8734072950 2024-03-07T06:52:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.947+0000 7f8732a00640 1 -- 172.21.15.133:0/1201922931 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8718003520 con 0x7f8734072950 2024-03-07T06:52:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8732a00640 1 -- 172.21.15.133:0/1201922931 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 223678850 0 0) 0x7f8718003520 con 0x7f8734072950 2024-03-07T06:52:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8732a00640 1 -- 172.21.15.133:0/1201922931 >> v2:172.21.15.161:3300/0 conn(0x7f873415fbb0 msgr2=0x7f87340cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8732a00640 1 --2- 172.21.15.133:0/1201922931 >> v2:172.21.15.161:3300/0 conn(0x7f873415fbb0 0x7f87340cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8732a00640 1 -- 172.21.15.133:0/1201922931 >> v1:172.21.15.161:6789/0 conn(0x7f87340cdee0 legacy=0x7f873415c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8732a00640 1 -- 172.21.15.133:0/1201922931 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8734072430 con 0x7f8734072950 2024-03-07T06:52:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8732a00640 1 -- 172.21.15.133:0/1201922931 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f871c001670 con 0x7f8734072950 2024-03-07T06:52:13.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8732a00640 1 -- 172.21.15.133:0/1201922931 >> v1:172.21.15.133:6789/0 conn(0x7f8734072950 legacy=0x7f873415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8732a00640 1 --2- 172.21.15.133:0/1201922931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87180039b0 0x7f8718003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8732a00640 1 --2- 172.21.15.133:0/1201922931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8718004800 0x7f8718004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8732a00640 1 --2- 172.21.15.133:0/1201922931 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87180052e0 0x7f871800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8732a00640 1 -- 172.21.15.133:0/1201922931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8734072f60 con 0x7f87180039b0 2024-03-07T06:52:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8732a00640 1 -- 172.21.15.133:0/1201922931 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871800c190 con 0x7f8718004800 2024-03-07T06:52:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8732a00640 1 -- 172.21.15.133:0/1201922931 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871800c390 con 0x7f87180052e0 2024-03-07T06:52:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8733400640 1 --2- 172.21.15.133:0/1201922931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87180039b0 0x7f8718003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f872be00640 1 --2- 172.21.15.133:0/1201922931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8718004800 0x7f8718004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8733e00640 1 --2- 172.21.15.133:0/1201922931 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87180052e0 0x7f871800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8733400640 1 -- 172.21.15.133:0/1201922931 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87180052e0 msgr2=0x7f871800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8733400640 1 --2- 172.21.15.133:0/1201922931 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87180052e0 0x7f871800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8733400640 1 -- 172.21.15.133:0/1201922931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8718004800 msgr2=0x7f8718004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8733400640 1 --2- 172.21.15.133:0/1201922931 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8718004800 0x7f8718004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8733400640 1 -- 172.21.15.133:0/1201922931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8718003520 con 0x7f87180039b0 2024-03-07T06:52:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.948+0000 7f8733e00640 1 --2- 172.21.15.133:0/1201922931 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87180052e0 0x7f871800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.949+0000 7f8733e00640 1 -- 172.21.15.133:0/1201922931 reap_dead start 2024-03-07T06:52:13.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.949+0000 7f8733400640 1 --2- 172.21.15.133:0/1201922931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87180039b0 0x7f8718003db0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f872400ba00 tx=0x7f8718005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5029ec089fe7ecc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.950+0000 7f8732a00640 1 -- 172.21.15.133:0/1201922931 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872400b600 con 0x7f87180039b0 2024-03-07T06:52:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.950+0000 7f8732a00640 1 -- 172.21.15.133:0/1201922931 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8724038070 con 0x7f87180039b0 2024-03-07T06:52:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.950+0000 7f8732a00640 1 -- 172.21.15.133:0/1201922931 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87240338a0 con 0x7f87180039b0 2024-03-07T06:52:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.950+0000 7f873a000640 1 -- 172.21.15.133:0/1201922931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87180039b0 msgr2=0x7f8718003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.950+0000 7f873a000640 1 --2- 172.21.15.133:0/1201922931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87180039b0 0x7f8718003db0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f872400ba00 tx=0x7f8718005830 comp rx=0 tx=0).stop 2024-03-07T06:52:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.950+0000 7f873a000640 1 -- 172.21.15.133:0/1201922931 shutdown_connections 2024-03-07T06:52:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.950+0000 7f873a000640 1 --2- 172.21.15.133:0/1201922931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87180039b0 0x7f8718003db0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.950+0000 7f873a000640 1 -- 172.21.15.133:0/1201922931 >> 172.21.15.133:0/1201922931 conn(0x7f87340b3120 msgr2=0x7f87340b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.950+0000 7f873a000640 1 -- 172.21.15.133:0/1201922931 shutdown_connections 2024-03-07T06:52:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.950+0000 7f873a000640 1 -- 172.21.15.133:0/1201922931 wait complete. 2024-03-07T06:52:13.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.952+0000 7f873a000640 1 Processor -- start 2024-03-07T06:52:13.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.952+0000 7f873a000640 1 -- start start 2024-03-07T06:52:13.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.953+0000 7f873a000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87180039b0 0x7f873406e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.953+0000 7f873a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f873415b410 0x7f87340b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.953+0000 7f873a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87340b5830 0x7f87341f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.953+0000 7f873a000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87340ce2c0 con 0x7f87340b5830 2024-03-07T06:52:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.953+0000 7f873a000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8734073490 con 0x7f873415b410 2024-03-07T06:52:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.953+0000 7f873a000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87340b1a00 con 0x7f87180039b0 2024-03-07T06:52:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.953+0000 7f8733e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87340b5830 0x7f87341f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.953+0000 7f8733e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87340b5830 0x7f87341f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38746/0 (socket says 172.21.15.133:38746) 2024-03-07T06:52:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.953+0000 7f8733e00640 1 -- 172.21.15.133:0/1666980679 learned_addr learned my addr 172.21.15.133:0/1666980679 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.954+0000 7f8733400640 1 --2- 172.21.15.133:0/1666980679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87180039b0 0x7f873406e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.954+0000 7f8733e00640 1 -- 172.21.15.133:0/1666980679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87180039b0 msgr2=0x7f873406e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.954+0000 7f8733e00640 1 --2- 172.21.15.133:0/1666980679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87180039b0 0x7f873406e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.954+0000 7f8733e00640 1 -- 172.21.15.133:0/1666980679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f873415b410 msgr2=0x7f87340b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:13.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.954+0000 7f872be00640 1 --2- 172.21.15.133:0/1666980679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f873415b410 0x7f87340b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.954+0000 7f8733e00640 1 --2- 172.21.15.133:0/1666980679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f873415b410 0x7f87340b52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.954+0000 7f8733e00640 1 -- 172.21.15.133:0/1666980679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873406d2a0 con 0x7f87340b5830 2024-03-07T06:52:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.954+0000 7f8733e00640 1 --2- 172.21.15.133:0/1666980679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87340b5830 0x7f87341f2b80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f873404faa0 tx=0x7f872c004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.954+0000 7f872b400640 1 -- 172.21.15.133:0/1666980679 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c006b50 con 0x7f87340b5830 2024-03-07T06:52:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.954+0000 7f872b400640 1 -- 172.21.15.133:0/1666980679 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c006d10 con 0x7f87340b5830 2024-03-07T06:52:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.954+0000 7f873a000640 1 -- 172.21.15.133:0/1666980679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873406d580 con 0x7f87340b5830 2024-03-07T06:52:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.954+0000 7f872b400640 1 -- 172.21.15.133:0/1666980679 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c011ae0 con 0x7f87340b5830 2024-03-07T06:52:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.954+0000 7f873a000640 1 -- 172.21.15.133:0/1666980679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873406db40 con 0x7f87340b5830 2024-03-07T06:52:13.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.955+0000 7f873a000640 1 -- 172.21.15.133:0/1666980679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8734062c90 con 0x7f87340b5830 2024-03-07T06:52:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.956+0000 7f872b400640 1 -- 172.21.15.133:0/1666980679 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f872c01e020 con 0x7f87340b5830 2024-03-07T06:52:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.958+0000 7f872b400640 1 --2- 172.21.15.133:0/1666980679 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f871006b4d0 0x7f871006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.958+0000 7f872b400640 1 -- 172.21.15.133:0/1666980679 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f872c08dcd0 con 0x7f87340b5830 2024-03-07T06:52:13.956 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.958+0000 7f8733400640 1 --2- 172.21.15.133:0/1666980679 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f871006b4d0 0x7f871006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:13.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.959+0000 7f8733400640 1 --2- 172.21.15.133:0/1666980679 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f871006b4d0 0x7f871006d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f87180057e0 tx=0x7f8724030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:13.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:13.962+0000 7f872b400640 1 -- 172.21.15.133:0/1666980679 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f872c05cc20 con 0x7f87340b5830 2024-03-07T06:52:14.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.198+0000 7f873a000640 1 -- 172.21.15.133:0/1666980679 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8734072d40 con 0x7f87340b5830 2024-03-07T06:52:14.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.200+0000 7f872b400640 1 -- 172.21.15.133:0/1666980679 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4983 (secure 0 0 0) 0x7f872c05c520 con 0x7f87340b5830 2024-03-07T06:52:14.199 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:14.199 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1116649196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1116649196},{"type":"v1","addr":"172.21.15.161:6835","nonce":1116649196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3485625105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3485625105},{"type":"v1","addr":"172.21.15.133:6835","nonce":3485625105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3866978630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3866978630},{"type":"v1","addr":"172.21.15.133:6833","nonce":3866978630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:51:14.594513+0000","modified":"2024-03-07T06:51:17.675044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2885273912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2885273912},{"type":"v1","addr":"172.21.15.161:6837","nonce":2885273912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-03-07T06:52:14.199 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 8 2024-03-07T06:52:14.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.202+0000 7f873a000640 1 -- 172.21.15.133:0/1666980679 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f871006b4d0 msgr2=0x7f871006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:14.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.202+0000 7f873a000640 1 --2- 172.21.15.133:0/1666980679 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f871006b4d0 0x7f871006d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f87180057e0 tx=0x7f8724030000 comp rx=0 tx=0).stop 2024-03-07T06:52:14.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.202+0000 7f873a000640 1 -- 172.21.15.133:0/1666980679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87340b5830 msgr2=0x7f87341f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:14.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.202+0000 7f873a000640 1 --2- 172.21.15.133:0/1666980679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87340b5830 0x7f87341f2b80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f873404faa0 tx=0x7f872c004690 comp rx=0 tx=0).stop 2024-03-07T06:52:14.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.203+0000 7f873a000640 1 -- 172.21.15.133:0/1666980679 shutdown_connections 2024-03-07T06:52:14.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.203+0000 7f873a000640 1 --2- 172.21.15.133:0/1666980679 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87340b5830 0x7f87341f2b80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.203+0000 7f873a000640 1 --2- 172.21.15.133:0/1666980679 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f871006b4d0 0x7f871006d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.203+0000 7f873a000640 1 --2- 172.21.15.133:0/1666980679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f873415b410 0x7f87340b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.203+0000 7f873a000640 1 --2- 172.21.15.133:0/1666980679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87180039b0 0x7f873406e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.203+0000 7f873a000640 1 -- 172.21.15.133:0/1666980679 >> 172.21.15.133:0/1666980679 conn(0x7f87340b3120 msgr2=0x7f873415ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:14.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.203+0000 7f873a000640 1 -- 172.21.15.133:0/1666980679 shutdown_connections 2024-03-07T06:52:14.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.203+0000 7f873a000640 1 -- 172.21.15.133:0/1666980679 wait complete. 2024-03-07T06:52:14.221 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:52:14.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.418+0000 7f3b07200640 1 Processor -- start 2024-03-07T06:52:14.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.418+0000 7f3b07200640 1 -- start start 2024-03-07T06:52:14.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.420+0000 7f3b07200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b0815eea0 0x7f3b08072830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:14.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.420+0000 7f3b07200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3b08072d70 0x7f3b080ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:14.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.420+0000 7f3b07200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0806e3f0 con 0x7f3b0815eac0 2024-03-07T06:52:14.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.420+0000 7f3b07200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0806e1d0 con 0x7f3b08072d70 2024-03-07T06:52:14.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.420+0000 7f3b07200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0806e6d0 con 0x7f3b0815eea0 2024-03-07T06:52:14.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.420+0000 7f3b06800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3b08072d70 0x7f3b080ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:14.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.420+0000 7f3b06800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3b08072d70 0x7f3b080ae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38756/0 (socket says 172.21.15.133:38756) 2024-03-07T06:52:14.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.420+0000 7f3b06800640 1 -- 172.21.15.133:0/428724425 learned_addr learned my addr 172.21.15.133:0/428724425 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:14.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.420+0000 7f3affe00640 1 --2- 172.21.15.133:0/428724425 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b0815eea0 0x7f3b08072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:14.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.421+0000 7f3b06800640 1 -- 172.21.15.133:0/428724425 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b0815eea0 msgr2=0x7f3b08072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:14.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.421+0000 7f3b06800640 1 --2- 172.21.15.133:0/428724425 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b0815eea0 0x7f3b08072830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.421+0000 7f3b06800640 1 -- 172.21.15.133:0/428724425 >> v1:172.21.15.161:6789/0 conn(0x7f3b0815eac0 legacy=0x7f3b0806dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:14.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.421+0000 7f3b06800640 1 -- 172.21.15.133:0/428724425 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b080ad9c0 con 0x7f3b08072d70 2024-03-07T06:52:14.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.422+0000 7f3b06800640 1 --2- 172.21.15.133:0/428724425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b08072d70 0x7f3b080ae4a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009e50 tx=0x7f3b0002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2340b6fece59c886 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:14.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.422+0000 7f3b05400640 1 -- 172.21.15.133:0/428724425 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0002ca70 con 0x7f3b08072d70 2024-03-07T06:52:14.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.422+0000 7f3b05400640 1 -- 172.21.15.133:0/428724425 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0002cc10 con 0x7f3b08072d70 2024-03-07T06:52:14.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.422+0000 7f3b05400640 1 -- 172.21.15.133:0/428724425 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b000343f0 con 0x7f3b08072d70 2024-03-07T06:52:14.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.422+0000 7f3b07200640 1 -- 172.21.15.133:0/428724425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b08072d70 msgr2=0x7f3b080ae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:14.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.422+0000 7f3b07200640 1 --2- 172.21.15.133:0/428724425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b08072d70 0x7f3b080ae4a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3b00009e50 tx=0x7f3b0002bed0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.423+0000 7f3b07200640 1 -- 172.21.15.133:0/428724425 shutdown_connections 2024-03-07T06:52:14.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.423+0000 7f3b07200640 1 --2- 172.21.15.133:0/428724425 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b08072d70 0x7f3b080ae4a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.423+0000 7f3b07200640 1 --2- 172.21.15.133:0/428724425 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b0815eea0 0x7f3b08072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.423+0000 7f3b07200640 1 -- 172.21.15.133:0/428724425 >> 172.21.15.133:0/428724425 conn(0x7f3b080b2e90 msgr2=0x7f3b080b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:14.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.423+0000 7f3b07200640 1 -- 172.21.15.133:0/428724425 shutdown_connections 2024-03-07T06:52:14.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.423+0000 7f3b07200640 1 -- 172.21.15.133:0/428724425 wait complete. 2024-03-07T06:52:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.424+0000 7f3b07200640 1 Processor -- start 2024-03-07T06:52:14.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.425+0000 7f3b07200640 1 -- start start 2024-03-07T06:52:14.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.425+0000 7f3b07200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b08072d70 0x7f3b08141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:14.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.425+0000 7f3b07200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0815eac0 0x7f3b08071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:14.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.425+0000 7f3b07200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b0815eea0 0x7f3b08071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:14.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.425+0000 7f3b07200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0815f300 con 0x7f3b08072d70 2024-03-07T06:52:14.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.425+0000 7f3b07200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b080adf80 con 0x7f3b0815eac0 2024-03-07T06:52:14.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.425+0000 7f3b07200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b080ae9e0 con 0x7f3b0815eea0 2024-03-07T06:52:14.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.426+0000 7f3b05e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b08072d70 0x7f3b08141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:14.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.426+0000 7f3b05e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b08072d70 0x7f3b08141bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38770/0 (socket says 172.21.15.133:38770) 2024-03-07T06:52:14.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.426+0000 7f3b05e00640 1 -- 172.21.15.133:0/3761045804 learned_addr learned my addr 172.21.15.133:0/3761045804 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:14.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.426+0000 7f3affe00640 1 --2- 172.21.15.133:0/3761045804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0815eac0 0x7f3b08071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:14.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.426+0000 7f3b05e00640 1 -- 172.21.15.133:0/3761045804 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b0815eea0 msgr2=0x7f3b08071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:14.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.426+0000 7f3b05e00640 1 --2- 172.21.15.133:0/3761045804 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b0815eea0 0x7f3b08071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.426+0000 7f3b05e00640 1 -- 172.21.15.133:0/3761045804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0815eac0 msgr2=0x7f3b08071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:14.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.426+0000 7f3b05e00640 1 --2- 172.21.15.133:0/3761045804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0815eac0 0x7f3b08071720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.426+0000 7f3b05e00640 1 -- 172.21.15.133:0/3761045804 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0815af50 con 0x7f3b08072d70 2024-03-07T06:52:14.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.426+0000 7f3b05e00640 1 --2- 172.21.15.133:0/3761045804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b08072d70 0x7f3b08141bb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3af800c430 tx=0x7f3af800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.427+0000 7f3afe000640 1 -- 172.21.15.133:0/3761045804 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8011790 con 0x7f3b08072d70 2024-03-07T06:52:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.427+0000 7f3afe000640 1 -- 172.21.15.133:0/3761045804 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af8017070 con 0x7f3b08072d70 2024-03-07T06:52:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.427+0000 7f3afe000640 1 -- 172.21.15.133:0/3761045804 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af8011bb0 con 0x7f3b08072d70 2024-03-07T06:52:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.427+0000 7f3b07200640 1 -- 172.21.15.133:0/3761045804 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0815b230 con 0x7f3b08072d70 2024-03-07T06:52:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.427+0000 7f3b07200640 1 -- 172.21.15.133:0/3761045804 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0815b7f0 con 0x7f3b08072d70 2024-03-07T06:52:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.428+0000 7f3b07200640 1 -- 172.21.15.133:0/3761045804 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0815d940 con 0x7f3b08072d70 2024-03-07T06:52:14.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.429+0000 7f3afe000640 1 -- 172.21.15.133:0/3761045804 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3af801c460 con 0x7f3b08072d70 2024-03-07T06:52:14.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.430+0000 7f3afe000640 1 --2- 172.21.15.133:0/3761045804 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3ae406b4d0 0x7f3ae406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:14.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.430+0000 7f3afe000640 1 -- 172.21.15.133:0/3761045804 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f3af808a250 con 0x7f3b08072d70 2024-03-07T06:52:14.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.430+0000 7f3affe00640 1 --2- 172.21.15.133:0/3761045804 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3ae406b4d0 0x7f3ae406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:14.430 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.431+0000 7f3affe00640 1 --2- 172.21.15.133:0/3761045804 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3ae406b4d0 0x7f3ae406d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3af0006fd0 tx=0x7f3af0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:14.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.435+0000 7f3afe000640 1 -- 172.21.15.133:0/3761045804 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3b0815d940 con 0x7f3b08072d70 2024-03-07T06:52:14.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.627+0000 7f3b07200640 1 -- 172.21.15.133:0/3761045804 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b08062c90 con 0x7f3b08072d70 2024-03-07T06:52:14.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.629+0000 7f3afe000640 1 -- 172.21.15.133:0/3761045804 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16901 (secure 0 0 0) 0x7f3af8058fb0 con 0x7f3b08072d70 2024-03-07T06:52:14.628 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:14.628 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:56.413096+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:52:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.632+0000 7f3b07200640 1 -- 172.21.15.133:0/3761045804 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3ae406b4d0 msgr2=0x7f3ae406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.632+0000 7f3b07200640 1 --2- 172.21.15.133:0/3761045804 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3ae406b4d0 0x7f3ae406d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3af0006fd0 tx=0x7f3af0008040 comp rx=0 tx=0).stop 2024-03-07T06:52:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.632+0000 7f3b07200640 1 -- 172.21.15.133:0/3761045804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b08072d70 msgr2=0x7f3b08141bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.632+0000 7f3b07200640 1 --2- 172.21.15.133:0/3761045804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b08072d70 0x7f3b08141bb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3af800c430 tx=0x7f3af800c8f0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.632+0000 7f3b07200640 1 -- 172.21.15.133:0/3761045804 shutdown_connections 2024-03-07T06:52:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.632+0000 7f3b07200640 1 --2- 172.21.15.133:0/3761045804 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b0815eea0 0x7f3b08071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.632+0000 7f3b07200640 1 --2- 172.21.15.133:0/3761045804 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3ae406b4d0 0x7f3ae406d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.632+0000 7f3b07200640 1 --2- 172.21.15.133:0/3761045804 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b0815eac0 0x7f3b08071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.632+0000 7f3b07200640 1 --2- 172.21.15.133:0/3761045804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b08072d70 0x7f3b08141bb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.632+0000 7f3b07200640 1 -- 172.21.15.133:0/3761045804 >> 172.21.15.133:0/3761045804 conn(0x7f3b080b2e90 msgr2=0x7f3b08072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.632+0000 7f3b07200640 1 -- 172.21.15.133:0/3761045804 shutdown_connections 2024-03-07T06:52:14.631 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.632+0000 7f3b07200640 1 -- 172.21.15.133:0/3761045804 wait complete. 2024-03-07T06:52:14.648 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-07T06:52:14.651 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:52:14.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.846+0000 7f3815a00640 1 Processor -- start 2024-03-07T06:52:14.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.846+0000 7f3815a00640 1 -- start start 2024-03-07T06:52:14.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.848+0000 7f3815a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f381015aaf0 0x7f381006e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:14.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.848+0000 7f3815a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f38100cd4c0 0x7f381015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:14.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.848+0000 7f3815a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3810072830 0x7f3810072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:14.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.848+0000 7f3815a00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38100731e0 con 0x7f3810072830 2024-03-07T06:52:14.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.848+0000 7f3815a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381006e850 con 0x7f38100cd4c0 2024-03-07T06:52:14.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.848+0000 7f3815a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381006eb30 con 0x7f381015aaf0 2024-03-07T06:52:14.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.848+0000 7f3815000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3810072830 0x7f3810072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:14.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.848+0000 7f3815000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3810072830 0x7f3810072ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38784/0 (socket says 172.21.15.133:38784) 2024-03-07T06:52:14.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.848+0000 7f3815000640 1 -- 172.21.15.133:0/2214226814 learned_addr learned my addr 172.21.15.133:0/2214226814 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:14.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.848+0000 7f380fe00640 1 --2- 172.21.15.133:0/2214226814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f381015aaf0 0x7f381006e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:14.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.848+0000 7f380f400640 1 --2- 172.21.15.133:0/2214226814 >> v2:172.21.15.161:3300/0 conn(0x7f38100cd4c0 0x7f381015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:14.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.848+0000 7f3815000640 1 -- 172.21.15.133:0/2214226814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f381015aaf0 msgr2=0x7f381006e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:14.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.848+0000 7f3815000640 1 --2- 172.21.15.133:0/2214226814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f381015aaf0 0x7f381006e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.849+0000 7f3815000640 1 -- 172.21.15.133:0/2214226814 >> v2:172.21.15.161:3300/0 conn(0x7f38100cd4c0 msgr2=0x7f381015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:14.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.849+0000 7f3815000640 1 --2- 172.21.15.133:0/2214226814 >> v2:172.21.15.161:3300/0 conn(0x7f38100cd4c0 0x7f381015aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.849+0000 7f3815000640 1 -- 172.21.15.133:0/2214226814 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381006ee10 con 0x7f3810072830 2024-03-07T06:52:14.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.849+0000 7f380f400640 1 --2- 172.21.15.133:0/2214226814 >> v2:172.21.15.161:3300/0 conn(0x7f38100cd4c0 0x7f381015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:14.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.850+0000 7f3815000640 1 --2- 172.21.15.133:0/2214226814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3810072830 0x7f3810072ca0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3810069140 tx=0x7f380802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b20ed6bb10342b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:14.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.850+0000 7f380ea00640 1 -- 172.21.15.133:0/2214226814 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380802ca70 con 0x7f3810072830 2024-03-07T06:52:14.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.850+0000 7f380ea00640 1 -- 172.21.15.133:0/2214226814 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380802cc10 con 0x7f3810072830 2024-03-07T06:52:14.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.850+0000 7f380ea00640 1 -- 172.21.15.133:0/2214226814 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38080343f0 con 0x7f3810072830 2024-03-07T06:52:14.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.851+0000 7f3815a00640 1 -- 172.21.15.133:0/2214226814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3810072830 msgr2=0x7f3810072ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:14.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.851+0000 7f3815a00640 1 --2- 172.21.15.133:0/2214226814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3810072830 0x7f3810072ca0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3810069140 tx=0x7f380802bed0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.851+0000 7f3815a00640 1 -- 172.21.15.133:0/2214226814 shutdown_connections 2024-03-07T06:52:14.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.851+0000 7f3815a00640 1 --2- 172.21.15.133:0/2214226814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3810072830 0x7f3810072ca0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.851+0000 7f3815a00640 1 --2- 172.21.15.133:0/2214226814 >> v2:172.21.15.161:3300/0 conn(0x7f38100cd4c0 0x7f381015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.851+0000 7f3815a00640 1 --2- 172.21.15.133:0/2214226814 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f381015aaf0 0x7f381006e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.851+0000 7f3815a00640 1 -- 172.21.15.133:0/2214226814 >> 172.21.15.133:0/2214226814 conn(0x7f38100b2e90 msgr2=0x7f38100b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:14.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.851+0000 7f3815a00640 1 -- 172.21.15.133:0/2214226814 shutdown_connections 2024-03-07T06:52:14.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.851+0000 7f3815a00640 1 -- 172.21.15.133:0/2214226814 wait complete. 2024-03-07T06:52:14.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.853+0000 7f3815a00640 1 Processor -- start 2024-03-07T06:52:14.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.853+0000 7f3815a00640 1 -- start start 2024-03-07T06:52:14.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f3815a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3810072830 0x7f381015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:14.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f3815a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f38100cd4c0 0x7f381013e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:14.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f3815a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381015aaf0 0x7f3810154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:14.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f3815a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381006f0f0 con 0x7f381015aaf0 2024-03-07T06:52:14.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f3815a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381006e560 con 0x7f3810072830 2024-03-07T06:52:14.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f3815a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381015b410 con 0x7f38100cd4c0 2024-03-07T06:52:14.853 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f3815000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381015aaf0 0x7f3810154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:14.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f3815000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381015aaf0 0x7f3810154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38800/0 (socket says 172.21.15.133:38800) 2024-03-07T06:52:14.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f3815000640 1 -- 172.21.15.133:0/1346208972 learned_addr learned my addr 172.21.15.133:0/1346208972 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:14.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f380f400640 1 --2- 172.21.15.133:0/1346208972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f38100cd4c0 0x7f381013e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:14.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f3815000640 1 -- 172.21.15.133:0/1346208972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f38100cd4c0 msgr2=0x7f381013e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:14.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f380fe00640 1 --2- 172.21.15.133:0/1346208972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3810072830 0x7f381015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:14.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f3815000640 1 --2- 172.21.15.133:0/1346208972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f38100cd4c0 0x7f381013e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f3815000640 1 -- 172.21.15.133:0/1346208972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3810072830 msgr2=0x7f381015c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:14.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f3815000640 1 --2- 172.21.15.133:0/1346208972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3810072830 0x7f381015c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:14.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.854+0000 7f3815000640 1 -- 172.21.15.133:0/1346208972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38100af290 con 0x7f381015aaf0 2024-03-07T06:52:14.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.855+0000 7f3815000640 1 --2- 172.21.15.133:0/1346208972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381015aaf0 0x7f3810154f30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3808036040 tx=0x7f3808004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:14.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.855+0000 7f3807e00640 1 -- 172.21.15.133:0/1346208972 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808043020 con 0x7f381015aaf0 2024-03-07T06:52:14.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.855+0000 7f3807e00640 1 -- 172.21.15.133:0/1346208972 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380803e450 con 0x7f381015aaf0 2024-03-07T06:52:14.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.855+0000 7f3807e00640 1 -- 172.21.15.133:0/1346208972 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808009510 con 0x7f381015aaf0 2024-03-07T06:52:14.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.855+0000 7f3815a00640 1 -- 172.21.15.133:0/1346208972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381015c9f0 con 0x7f381015aaf0 2024-03-07T06:52:14.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.855+0000 7f3815a00640 1 -- 172.21.15.133:0/1346208972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381015cfb0 con 0x7f381015aaf0 2024-03-07T06:52:14.855 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.856+0000 7f3815a00640 1 -- 172.21.15.133:0/1346208972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3810062c90 con 0x7f381015aaf0 2024-03-07T06:52:14.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.857+0000 7f3807e00640 1 -- 172.21.15.133:0/1346208972 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3808046020 con 0x7f381015aaf0 2024-03-07T06:52:14.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.859+0000 7f3807e00640 1 --2- 172.21.15.133:0/1346208972 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f37f006b4d0 0x7f37f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:14.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.859+0000 7f3807e00640 1 -- 172.21.15.133:0/1346208972 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f38080acc00 con 0x7f381015aaf0 2024-03-07T06:52:14.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.859+0000 7f380fe00640 1 --2- 172.21.15.133:0/1346208972 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f37f006b4d0 0x7f37f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:14.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.860+0000 7f380fe00640 1 --2- 172.21.15.133:0/1346208972 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f37f006b4d0 0x7f37f006d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3800006fd0 tx=0x7f3800008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:14.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:14.863+0000 7f3807e00640 1 -- 172.21.15.133:0/1346208972 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f380804d030 con 0x7f381015aaf0 2024-03-07T06:52:15.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.080+0000 7f3815a00640 1 -- 172.21.15.133:0/1346208972 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38100cdfd0 con 0x7f381015aaf0 2024-03-07T06:52:15.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.081+0000 7f3807e00640 1 -- 172.21.15.133:0/1346208972 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4983 (secure 0 0 0) 0x7f380807b8d0 con 0x7f381015aaf0 2024-03-07T06:52:15.080 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:15.080 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1116649196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1116649196},{"type":"v1","addr":"172.21.15.161:6835","nonce":1116649196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3485625105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3485625105},{"type":"v1","addr":"172.21.15.133:6835","nonce":3485625105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3866978630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3866978630},{"type":"v1","addr":"172.21.15.133:6833","nonce":3866978630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:51:14.594513+0000","modified":"2024-03-07T06:51:17.675044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2885273912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2885273912},{"type":"v1","addr":"172.21.15.161:6837","nonce":2885273912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-03-07T06:52:15.080 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 8 2024-03-07T06:52:15.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.083+0000 7f3815a00640 1 -- 172.21.15.133:0/1346208972 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f37f006b4d0 msgr2=0x7f37f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:15.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.083+0000 7f3815a00640 1 --2- 172.21.15.133:0/1346208972 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f37f006b4d0 0x7f37f006d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3800006fd0 tx=0x7f3800008040 comp rx=0 tx=0).stop 2024-03-07T06:52:15.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.083+0000 7f3815a00640 1 -- 172.21.15.133:0/1346208972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381015aaf0 msgr2=0x7f3810154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:15.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.083+0000 7f3815a00640 1 --2- 172.21.15.133:0/1346208972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381015aaf0 0x7f3810154f30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3808036040 tx=0x7f3808004a40 comp rx=0 tx=0).stop 2024-03-07T06:52:15.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.083+0000 7f3815a00640 1 -- 172.21.15.133:0/1346208972 shutdown_connections 2024-03-07T06:52:15.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.083+0000 7f3815a00640 1 --2- 172.21.15.133:0/1346208972 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381015aaf0 0x7f3810154f30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.083+0000 7f3815a00640 1 --2- 172.21.15.133:0/1346208972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f38100cd4c0 0x7f381013e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.083+0000 7f3815a00640 1 --2- 172.21.15.133:0/1346208972 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f37f006b4d0 0x7f37f006d990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.083+0000 7f3815a00640 1 --2- 172.21.15.133:0/1346208972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3810072830 0x7f381015c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.083+0000 7f3815a00640 1 -- 172.21.15.133:0/1346208972 >> 172.21.15.133:0/1346208972 conn(0x7f38100b2e90 msgr2=0x7f381006d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:15.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.084+0000 7f3815a00640 1 -- 172.21.15.133:0/1346208972 shutdown_connections 2024-03-07T06:52:15.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.084+0000 7f3815a00640 1 -- 172.21.15.133:0/1346208972 wait complete. 2024-03-07T06:52:15.099 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:52:15.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.300+0000 7fe244400640 1 Processor -- start 2024-03-07T06:52:15.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.300+0000 7fe244400640 1 -- start start 2024-03-07T06:52:15.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.302+0000 7fe244400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe23c15e390 0x7fe23c0724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.302+0000 7fe244400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe23c071d20 0x7fe23c0ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.302+0000 7fe244400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe23c0cdf80 con 0x7fe23c073050 2024-03-07T06:52:15.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.302+0000 7fe244400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23c0ce4d0 con 0x7fe23c15e390 2024-03-07T06:52:15.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.302+0000 7fe244400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c15e770 con 0x7fe23c071d20 2024-03-07T06:52:15.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.303+0000 7fe241a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe23c15e390 0x7fe23c0724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.303+0000 7fe242400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe23c071d20 0x7fe23c0ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.303+0000 7fe242400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe23c071d20 0x7fe23c0ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:37438/0 (socket says 172.21.15.133:37438) 2024-03-07T06:52:15.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.303+0000 7fe241a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe23c15e390 0x7fe23c0724b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:35046/0 (socket says 172.21.15.133:35046) 2024-03-07T06:52:15.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.303+0000 7fe242400640 1 -- 172.21.15.133:0/1635567476 learned_addr learned my addr 172.21.15.133:0/1635567476 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:15.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.304+0000 7fe242400640 1 -- 172.21.15.133:0/1635567476 >> v2:172.21.15.161:3300/0 conn(0x7fe23c15e390 msgr2=0x7fe23c0724b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:15.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.304+0000 7fe242400640 1 --2- 172.21.15.133:0/1635567476 >> v2:172.21.15.161:3300/0 conn(0x7fe23c15e390 0x7fe23c0724b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.304+0000 7fe242400640 1 -- 172.21.15.133:0/1635567476 >> v1:172.21.15.161:6789/0 conn(0x7fe23c073050 legacy=0x7fe23c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:15.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.304+0000 7fe242400640 1 -- 172.21.15.133:0/1635567476 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c0b3d90 con 0x7fe23c071d20 2024-03-07T06:52:15.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.304+0000 7fe241a00640 1 --2- 172.21.15.133:0/1635567476 >> v2:172.21.15.161:3300/0 conn(0x7fe23c15e390 0x7fe23c0724b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T06:52:15.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.305+0000 7fe242400640 1 --2- 172.21.15.133:0/1635567476 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe23c071d20 0x7fe23c0ae720 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0ce650 tx=0x7fe23402bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d939421c9765210b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.305+0000 7fe23be00640 1 -- 172.21.15.133:0/1635567476 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23402cae0 con 0x7fe23c071d20 2024-03-07T06:52:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.305+0000 7fe23be00640 1 -- 172.21.15.133:0/1635567476 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23402cc80 con 0x7fe23c071d20 2024-03-07T06:52:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.305+0000 7fe23be00640 1 -- 172.21.15.133:0/1635567476 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234034610 con 0x7fe23c071d20 2024-03-07T06:52:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.306+0000 7fe244400640 1 -- 172.21.15.133:0/1635567476 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe23c071d20 msgr2=0x7fe23c0ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.306+0000 7fe244400640 1 --2- 172.21.15.133:0/1635567476 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe23c071d20 0x7fe23c0ae720 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0ce650 tx=0x7fe23402bf40 comp rx=0 tx=0).stop 2024-03-07T06:52:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.306+0000 7fe244400640 1 -- 172.21.15.133:0/1635567476 shutdown_connections 2024-03-07T06:52:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.306+0000 7fe244400640 1 --2- 172.21.15.133:0/1635567476 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe23c071d20 0x7fe23c0ae720 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.306+0000 7fe244400640 1 --2- 172.21.15.133:0/1635567476 >> v2:172.21.15.161:3300/0 conn(0x7fe23c15e390 0x7fe23c0724b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.306+0000 7fe244400640 1 -- 172.21.15.133:0/1635567476 >> 172.21.15.133:0/1635567476 conn(0x7fe23c0b3120 msgr2=0x7fe23c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.306+0000 7fe244400640 1 -- 172.21.15.133:0/1635567476 shutdown_connections 2024-03-07T06:52:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.306+0000 7fe244400640 1 -- 172.21.15.133:0/1635567476 wait complete. 2024-03-07T06:52:15.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.308+0000 7fe244400640 1 Processor -- start 2024-03-07T06:52:15.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.308+0000 7fe244400640 1 -- start start 2024-03-07T06:52:15.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.309+0000 7fe244400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe23c071d20 0x7fe23c14cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.309+0000 7fe244400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe23c073050 0x7fe23c09b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.309+0000 7fe244400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe23c15e390 0x7fe23c15edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.309+0000 7fe244400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c06d2a0 con 0x7fe23c15e390 2024-03-07T06:52:15.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.309+0000 7fe244400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0b4070 con 0x7fe23c073050 2024-03-07T06:52:15.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.309+0000 7fe244400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c06d420 con 0x7fe23c071d20 2024-03-07T06:52:15.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.309+0000 7fe242400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe23c15e390 0x7fe23c15edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.309+0000 7fe242400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe23c15e390 0x7fe23c15edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38808/0 (socket says 172.21.15.133:38808) 2024-03-07T06:52:15.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.309+0000 7fe242400640 1 -- 172.21.15.133:0/2276723298 learned_addr learned my addr 172.21.15.133:0/2276723298 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:15.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.310+0000 7fe241a00640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe23c071d20 0x7fe23c14cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.310+0000 7fe241000640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe23c073050 0x7fe23c09b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.310+0000 7fe242400640 1 -- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe23c071d20 msgr2=0x7fe23c14cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:15.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.310+0000 7fe242400640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe23c071d20 0x7fe23c14cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.310+0000 7fe242400640 1 -- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe23c073050 msgr2=0x7fe23c09b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:15.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.310+0000 7fe242400640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe23c073050 0x7fe23c09b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.310+0000 7fe242400640 1 -- 172.21.15.133:0/2276723298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c15f2f0 con 0x7fe23c15e390 2024-03-07T06:52:15.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.310+0000 7fe241000640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe23c073050 0x7fe23c09b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:15.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.310+0000 7fe242400640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe23c15e390 0x7fe23c15edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe23402c550 tx=0x7fe234004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:15.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.310+0000 7fe239600640 1 -- 172.21.15.133:0/2276723298 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234043020 con 0x7fe23c15e390 2024-03-07T06:52:15.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.310+0000 7fe239600640 1 -- 172.21.15.133:0/2276723298 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23403e450 con 0x7fe23c15e390 2024-03-07T06:52:15.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.310+0000 7fe239600640 1 -- 172.21.15.133:0/2276723298 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234009510 con 0x7fe23c15e390 2024-03-07T06:52:15.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.310+0000 7fe244400640 1 -- 172.21.15.133:0/2276723298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23c159410 con 0x7fe23c15e390 2024-03-07T06:52:15.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.311+0000 7fe244400640 1 -- 172.21.15.133:0/2276723298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23c1599d0 con 0x7fe23c15e390 2024-03-07T06:52:15.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.312+0000 7fe244400640 1 -- 172.21.15.133:0/2276723298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23c062c90 con 0x7fe23c15e390 2024-03-07T06:52:15.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.313+0000 7fe239600640 1 -- 172.21.15.133:0/2276723298 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe234046020 con 0x7fe23c15e390 2024-03-07T06:52:15.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.313+0000 7fe239600640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe21c06b4d0 0x7fe21c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.314+0000 7fe239600640 1 -- 172.21.15.133:0/2276723298 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe2340acab0 con 0x7fe23c15e390 2024-03-07T06:52:15.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.314+0000 7fe241a00640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe21c06b4d0 0x7fe21c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.313 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.315+0000 7fe241a00640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe21c06b4d0 0x7fe21c06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe230009950 tx=0x7fe230008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:15.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.318+0000 7fe239600640 1 -- 172.21.15.133:0/2276723298 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe23404d030 con 0x7fe23c15e390 2024-03-07T06:52:15.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.551+0000 7fe244400640 1 -- 172.21.15.133:0/2276723298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe23c06e7b0 con 0x7fe23c15e390 2024-03-07T06:52:15.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.552+0000 7fe239600640 1 -- 172.21.15.133:0/2276723298 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4983 (secure 0 0 0) 0x7fe23407b810 con 0x7fe23c15e390 2024-03-07T06:52:15.551 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:15.551 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/1116649196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":1116649196},{"type":"v1","addr":"172.21.15.161:6835","nonce":1116649196}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4272,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3485625105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3485625105},{"type":"v1","addr":"172.21.15.133:6835","nonce":3485625105}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/3866978630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3866978630},{"type":"v1","addr":"172.21.15.133:6833","nonce":3866978630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:51:14.594513+0000","modified":"2024-03-07T06:51:17.675044+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6837/2885273912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":2885273912},{"type":"v1","addr":"172.21.15.161:6837","nonce":2885273912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4288,"qdb_cluster":[4288]},"id":1}]} 2024-03-07T06:52:15.551 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 8 2024-03-07T06:52:15.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.555+0000 7fe244400640 1 -- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe21c06b4d0 msgr2=0x7fe21c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:15.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.555+0000 7fe244400640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe21c06b4d0 0x7fe21c06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe230009950 tx=0x7fe230008040 comp rx=0 tx=0).stop 2024-03-07T06:52:15.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.555+0000 7fe244400640 1 -- 172.21.15.133:0/2276723298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe23c15e390 msgr2=0x7fe23c15edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:15.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.555+0000 7fe244400640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe23c15e390 0x7fe23c15edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe23402c550 tx=0x7fe234004a40 comp rx=0 tx=0).stop 2024-03-07T06:52:15.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.555+0000 7fe244400640 1 -- 172.21.15.133:0/2276723298 shutdown_connections 2024-03-07T06:52:15.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.555+0000 7fe244400640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe23c15e390 0x7fe23c15edb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.555+0000 7fe244400640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe21c06b4d0 0x7fe21c06d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.555+0000 7fe244400640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe23c073050 0x7fe23c09b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.555+0000 7fe244400640 1 --2- 172.21.15.133:0/2276723298 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe23c071d20 0x7fe23c14cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.555+0000 7fe244400640 1 -- 172.21.15.133:0/2276723298 >> 172.21.15.133:0/2276723298 conn(0x7fe23c0b3120 msgr2=0x7fe23c0728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:15.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.555+0000 7fe244400640 1 -- 172.21.15.133:0/2276723298 shutdown_connections 2024-03-07T06:52:15.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.556+0000 7fe244400640 1 -- 172.21.15.133:0/2276723298 wait complete. 2024-03-07T06:52:15.574 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:52:15.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.774+0000 7ff65aa00640 1 Processor -- start 2024-03-07T06:52:15.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.774+0000 7ff65aa00640 1 -- start start 2024-03-07T06:52:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.776+0000 7ff65aa00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6540ce5f0 0x7ff6540b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.776+0000 7ff65aa00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff6540af950 0x7ff654071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.776+0000 7ff65aa00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff65406f140 con 0x7ff65415b1b0 2024-03-07T06:52:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.776+0000 7ff65aa00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff654071610 con 0x7ff6540af950 2024-03-07T06:52:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.776+0000 7ff65aa00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65406f420 con 0x7ff6540ce5f0 2024-03-07T06:52:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.776+0000 7ff653e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff65415b1b0 0x7ff6540cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33358/0 (socket says 172.21.15.133:33358) 2024-03-07T06:52:15.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.776+0000 7ff653e00640 1 -- 172.21.15.133:0/1571673423 learned_addr learned my addr 172.21.15.133:0/1571673423 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:15.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.777+0000 7ff658a00640 1 --2- 172.21.15.133:0/1571673423 >> v2:172.21.15.161:3300/0 conn(0x7ff6540af950 0x7ff654071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.777+0000 7ff653400640 1 --2- 172.21.15.133:0/1571673423 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6540ce5f0 0x7ff6540b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.777+0000 7ff652a00640 1 -- 172.21.15.133:0/1571673423 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374262075 0 0) 0x7ff65406f140 con 0x7ff65415b1b0 2024-03-07T06:52:15.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.777+0000 7ff652a00640 1 -- 172.21.15.133:0/1571673423 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff638003520 con 0x7ff65415b1b0 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.777+0000 7ff652a00640 1 -- 172.21.15.133:0/1571673423 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2013785652 0 0) 0x7ff638003520 con 0x7ff65415b1b0 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.777+0000 7ff652a00640 1 -- 172.21.15.133:0/1571673423 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6540ce5f0 msgr2=0x7ff6540b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.777+0000 7ff652a00640 1 --2- 172.21.15.133:0/1571673423 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6540ce5f0 0x7ff6540b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.777+0000 7ff652a00640 1 -- 172.21.15.133:0/1571673423 >> v2:172.21.15.161:3300/0 conn(0x7ff6540af950 msgr2=0x7ff654071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.777+0000 7ff652a00640 1 --2- 172.21.15.133:0/1571673423 >> v2:172.21.15.161:3300/0 conn(0x7ff6540af950 0x7ff654071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.777+0000 7ff652a00640 1 -- 172.21.15.133:0/1571673423 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff654071860 con 0x7ff65415b1b0 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.777+0000 7ff652a00640 1 -- 172.21.15.133:0/1571673423 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7ff644001670 con 0x7ff65415b1b0 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.777+0000 7ff652a00640 1 -- 172.21.15.133:0/1571673423 >> v1:172.21.15.133:6789/0 conn(0x7ff65415b1b0 legacy=0x7ff6540cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.778+0000 7ff652a00640 1 --2- 172.21.15.133:0/1571673423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6380039b0 0x7ff638003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.778+0000 7ff652a00640 1 --2- 172.21.15.133:0/1571673423 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6540ce5f0 0x7ff638004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.778+0000 7ff652a00640 1 --2- 172.21.15.133:0/1571673423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff638004d40 0x7ff6380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.778+0000 7ff652a00640 1 -- 172.21.15.133:0/1571673423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff654071610 con 0x7ff638004d40 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.778+0000 7ff652a00640 1 -- 172.21.15.133:0/1571673423 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65406f420 con 0x7ff6380039b0 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.778+0000 7ff652a00640 1 -- 172.21.15.133:0/1571673423 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63800bb20 con 0x7ff6540ce5f0 2024-03-07T06:52:15.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.778+0000 7ff658a00640 1 --2- 172.21.15.133:0/1571673423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff638004d40 0x7ff6380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.778+0000 7ff653e00640 1 --2- 172.21.15.133:0/1571673423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6380039b0 0x7ff638003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.778+0000 7ff653400640 1 --2- 172.21.15.133:0/1571673423 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6540ce5f0 0x7ff638004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.778+0000 7ff658a00640 1 -- 172.21.15.133:0/1571673423 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6540ce5f0 msgr2=0x7ff638004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.778+0000 7ff658a00640 1 --2- 172.21.15.133:0/1571673423 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6540ce5f0 0x7ff638004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.778+0000 7ff658a00640 1 -- 172.21.15.133:0/1571673423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6380039b0 msgr2=0x7ff638003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.778+0000 7ff658a00640 1 --2- 172.21.15.133:0/1571673423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6380039b0 0x7ff638003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.778+0000 7ff658a00640 1 -- 172.21.15.133:0/1571673423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff638003520 con 0x7ff638004d40 2024-03-07T06:52:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.779+0000 7ff658a00640 1 --2- 172.21.15.133:0/1571673423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff638004d40 0x7ff6380051c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009fc0 tx=0x7ff638005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471679c38ccf8c87 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.780+0000 7ff652a00640 1 -- 172.21.15.133:0/1571673423 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c039070 con 0x7ff638004d40 2024-03-07T06:52:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.780+0000 7ff652a00640 1 -- 172.21.15.133:0/1571673423 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64c02cc60 con 0x7ff638004d40 2024-03-07T06:52:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.780+0000 7ff652a00640 1 -- 172.21.15.133:0/1571673423 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64c009c10 con 0x7ff638004d40 2024-03-07T06:52:15.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.780+0000 7ff65aa00640 1 -- 172.21.15.133:0/1571673423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff638004d40 msgr2=0x7ff6380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.780+0000 7ff65aa00640 1 --2- 172.21.15.133:0/1571673423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff638004d40 0x7ff6380051c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009fc0 tx=0x7ff638005830 comp rx=0 tx=0).stop 2024-03-07T06:52:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.780+0000 7ff658a00640 1 -- 172.21.15.133:0/1571673423 reap_dead start 2024-03-07T06:52:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.780+0000 7ff65aa00640 1 -- 172.21.15.133:0/1571673423 shutdown_connections 2024-03-07T06:52:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.780+0000 7ff65aa00640 1 -- 172.21.15.133:0/1571673423 >> 172.21.15.133:0/1571673423 conn(0x7ff6540b3120 msgr2=0x7ff6540b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.780+0000 7ff65aa00640 1 -- 172.21.15.133:0/1571673423 shutdown_connections 2024-03-07T06:52:15.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.780+0000 7ff65aa00640 1 -- 172.21.15.133:0/1571673423 wait complete. 2024-03-07T06:52:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.782+0000 7ff65aa00640 1 Processor -- start 2024-03-07T06:52:15.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.782+0000 7ff65aa00640 1 -- start start 2024-03-07T06:52:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.783+0000 7ff65aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff638004d40 0x7ff65414f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.783+0000 7ff65aa00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff654155a20 0x7ff6541595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.783+0000 7ff65aa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6540b5710 0x7ff6540b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.783+0000 7ff65aa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65415b590 con 0x7ff638004d40 2024-03-07T06:52:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.783+0000 7ff65aa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65406ee30 con 0x7ff6540b5710 2024-03-07T06:52:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.783+0000 7ff65aa00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65415b710 con 0x7ff654155a20 2024-03-07T06:52:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.783+0000 7ff653e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff638004d40 0x7ff65414f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.783+0000 7ff653e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff638004d40 0x7ff65414f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38830/0 (socket says 172.21.15.133:38830) 2024-03-07T06:52:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.783+0000 7ff653e00640 1 -- 172.21.15.133:0/3420536087 learned_addr learned my addr 172.21.15.133:0/3420536087 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.784+0000 7ff658a00640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6540b5710 0x7ff6540b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.784+0000 7ff653e00640 1 -- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff654155a20 msgr2=0x7ff6541595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.784+0000 7ff653400640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff654155a20 0x7ff6541595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.784+0000 7ff653e00640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff654155a20 0x7ff6541595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.784+0000 7ff653e00640 1 -- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6540b5710 msgr2=0x7ff6540b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:15.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.784+0000 7ff653e00640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6540b5710 0x7ff6540b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.784+0000 7ff653e00640 1 -- 172.21.15.133:0/3420536087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65414fef0 con 0x7ff638004d40 2024-03-07T06:52:15.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.784+0000 7ff658a00640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6540b5710 0x7ff6540b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:15.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.784+0000 7ff653e00640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff638004d40 0x7ff65414f9b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff644000fb0 tx=0x7ff64400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.784+0000 7ff64be00640 1 -- 172.21.15.133:0/3420536087 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64400ee60 con 0x7ff638004d40 2024-03-07T06:52:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.784+0000 7ff64be00640 1 -- 172.21.15.133:0/3420536087 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff644018070 con 0x7ff638004d40 2024-03-07T06:52:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.784+0000 7ff64be00640 1 -- 172.21.15.133:0/3420536087 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64400ca40 con 0x7ff638004d40 2024-03-07T06:52:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.785+0000 7ff65aa00640 1 -- 172.21.15.133:0/3420536087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff654159af0 con 0x7ff638004d40 2024-03-07T06:52:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.785+0000 7ff65aa00640 1 -- 172.21.15.133:0/3420536087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65415bc10 con 0x7ff638004d40 2024-03-07T06:52:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.786+0000 7ff65aa00640 1 -- 172.21.15.133:0/3420536087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff654062c90 con 0x7ff638004d40 2024-03-07T06:52:15.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.787+0000 7ff64be00640 1 -- 172.21.15.133:0/3420536087 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff64400cbe0 con 0x7ff638004d40 2024-03-07T06:52:15.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.787+0000 7ff64be00640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff63406b500 0x7ff63406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:15.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.787+0000 7ff64be00640 1 -- 172.21.15.133:0/3420536087 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff64408a1a0 con 0x7ff638004d40 2024-03-07T06:52:15.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.788+0000 7ff653400640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff63406b500 0x7ff63406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:15.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.789+0000 7ff653400640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff63406b500 0x7ff63406d9c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff654158480 tx=0x7ff640008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:15.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.792+0000 7ff64be00640 1 -- 172.21.15.133:0/3420536087 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff65414fef0 con 0x7ff638004d40 2024-03-07T06:52:15.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.987+0000 7ff65aa00640 1 -- 172.21.15.133:0/3420536087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6540cc1f0 con 0x7ff638004d40 2024-03-07T06:52:15.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.989+0000 7ff64be00640 1 -- 172.21.15.133:0/3420536087 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16901 (secure 0 0 0) 0x7ff644058f00 con 0x7ff638004d40 2024-03-07T06:52:15.989 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:15.989 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:51:56.413096+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:51:12.858333+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-07T06:51:13.615348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:52:15.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.992+0000 7ff65aa00640 1 -- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff63406b500 msgr2=0x7ff63406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:15.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.992+0000 7ff65aa00640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff63406b500 0x7ff63406d9c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff654158480 tx=0x7ff640008040 comp rx=0 tx=0).stop 2024-03-07T06:52:15.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.992+0000 7ff65aa00640 1 -- 172.21.15.133:0/3420536087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff638004d40 msgr2=0x7ff65414f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:15.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.992+0000 7ff65aa00640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff638004d40 0x7ff65414f9b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff644000fb0 tx=0x7ff64400ec10 comp rx=0 tx=0).stop 2024-03-07T06:52:15.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.992+0000 7ff65aa00640 1 -- 172.21.15.133:0/3420536087 shutdown_connections 2024-03-07T06:52:15.991 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.992+0000 7ff65aa00640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff63406b500 0x7ff63406d9c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.992+0000 7ff65aa00640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6540b5710 0x7ff6540b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.992+0000 7ff65aa00640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff654155a20 0x7ff6541595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.992+0000 7ff65aa00640 1 --2- 172.21.15.133:0/3420536087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff638004d40 0x7ff65414f9b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:15.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.992+0000 7ff65aa00640 1 -- 172.21.15.133:0/3420536087 >> 172.21.15.133:0/3420536087 conn(0x7ff6540b3120 msgr2=0x7ff6540cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:15.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.993+0000 7ff65aa00640 1 -- 172.21.15.133:0/3420536087 shutdown_connections 2024-03-07T06:52:15.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:15.993+0000 7ff65aa00640 1 -- 172.21.15.133:0/3420536087 wait complete. 2024-03-07T06:52:16.010 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-07T06:52:16.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.210+0000 7ff8b2600640 1 Processor -- start 2024-03-07T06:52:16.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.210+0000 7ff8b2600640 1 -- start start 2024-03-07T06:52:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.212+0000 7ff8b2600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff8ac0b5090 0x7ff8ac072910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.212+0000 7ff8b2600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ac15fc30 con 0x7ff8ac147bb0 2024-03-07T06:52:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.212+0000 7ff8b2600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ac15ff10 con 0x7ff8ac0d1260 2024-03-07T06:52:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.212+0000 7ff8b2600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8ac1601f0 con 0x7ff8ac0b5090 2024-03-07T06:52:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.212+0000 7ff8a3e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff8ac0b5090 0x7ff8ac072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.212+0000 7ff8a3e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff8ac0b5090 0x7ff8ac072910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38832/0 (socket says 172.21.15.133:38832) 2024-03-07T06:52:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.212+0000 7ff8a3e00640 1 -- 172.21.15.133:0/3310925303 learned_addr learned my addr 172.21.15.133:0/3310925303 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:16.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.212+0000 7ff8ab400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff8ac147bb0 0x7ff8ac0d0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33364/0 (socket says 172.21.15.133:33364) 2024-03-07T06:52:16.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.213+0000 7ff8a3e00640 1 -- 172.21.15.133:0/3310925303 >> v1:172.21.15.161:6789/0 conn(0x7ff8ac0d1260 legacy=0x7ff8ac160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:16.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.213+0000 7ff8a3e00640 1 -- 172.21.15.133:0/3310925303 >> v1:172.21.15.133:6789/0 conn(0x7ff8ac147bb0 legacy=0x7ff8ac0d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:16.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.213+0000 7ff8aaa00640 1 -- 172.21.15.133:0/3310925303 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 843343130 0 0) 0x7ff8ac15fc30 con 0x7ff8ac147bb0 2024-03-07T06:52:16.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.213+0000 7ff8a3e00640 1 -- 172.21.15.133:0/3310925303 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac1604d0 con 0x7ff8ac0b5090 2024-03-07T06:52:16.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.214+0000 7ff8a3e00640 1 --2- 172.21.15.133:0/3310925303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8ac0b5090 0x7ff8ac072910 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff894009870 tx=0x7ff89402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c03080c2627dda server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:16.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.214+0000 7ff8aaa00640 1 -- 172.21.15.133:0/3310925303 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894035020 con 0x7ff8ac0b5090 2024-03-07T06:52:16.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.215+0000 7ff8aaa00640 1 -- 172.21.15.133:0/3310925303 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff894031870 con 0x7ff8ac0b5090 2024-03-07T06:52:16.213 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.215+0000 7ff8aaa00640 1 -- 172.21.15.133:0/3310925303 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894031c40 con 0x7ff8ac0b5090 2024-03-07T06:52:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.215+0000 7ff8b2600640 1 -- 172.21.15.133:0/3310925303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8ac0b5090 msgr2=0x7ff8ac072910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.215+0000 7ff8b2600640 1 --2- 172.21.15.133:0/3310925303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8ac0b5090 0x7ff8ac072910 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff894009870 tx=0x7ff89402b910 comp rx=0 tx=0).stop 2024-03-07T06:52:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.215+0000 7ff8b2600640 1 -- 172.21.15.133:0/3310925303 shutdown_connections 2024-03-07T06:52:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.215+0000 7ff8b2600640 1 --2- 172.21.15.133:0/3310925303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8ac0b5090 0x7ff8ac072910 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.215+0000 7ff8b2600640 1 -- 172.21.15.133:0/3310925303 >> 172.21.15.133:0/3310925303 conn(0x7ff8ac06b480 msgr2=0x7ff8ac06b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.215+0000 7ff8b2600640 1 -- 172.21.15.133:0/3310925303 shutdown_connections 2024-03-07T06:52:16.214 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.215+0000 7ff8b2600640 1 -- 172.21.15.133:0/3310925303 wait complete. 2024-03-07T06:52:16.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.217+0000 7ff8b2600640 1 Processor -- start 2024-03-07T06:52:16.215 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.217+0000 7ff8b2600640 1 -- start start 2024-03-07T06:52:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.218+0000 7ff8b2600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8ac0b5090 0x7ff8ac155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.218+0000 7ff8b2600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8ac0d1260 0x7ff8ac1f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.218+0000 7ff8b2600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8ac147bb0 0x7ff8ac157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.218+0000 7ff8b2600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac072e50 con 0x7ff8ac0b5090 2024-03-07T06:52:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.218+0000 7ff8b2600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac147f90 con 0x7ff8ac0d1260 2024-03-07T06:52:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.218+0000 7ff8b2600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ac072fd0 con 0x7ff8ac147bb0 2024-03-07T06:52:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.218+0000 7ff8ab400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8ac0b5090 0x7ff8ac155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:16.218 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.218+0000 7ff8ab400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8ac0b5090 0x7ff8ac155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38844/0 (socket says 172.21.15.133:38844) 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.218+0000 7ff8ab400640 1 -- 172.21.15.133:0/4221632356 learned_addr learned my addr 172.21.15.133:0/4221632356 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.219+0000 7ff8a3e00640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8ac0d1260 0x7ff8ac1f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.219+0000 7ff8ab400640 1 -- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8ac147bb0 msgr2=0x7ff8ac157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.219+0000 7ff8abe00640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8ac147bb0 0x7ff8ac157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.219+0000 7ff8ab400640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8ac147bb0 0x7ff8ac157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.219+0000 7ff8ab400640 1 -- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8ac0d1260 msgr2=0x7ff8ac1f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.219+0000 7ff8ab400640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8ac0d1260 0x7ff8ac1f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.219+0000 7ff8ab400640 1 -- 172.21.15.133:0/4221632356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ac156130 con 0x7ff8ac0b5090 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.219+0000 7ff8a3e00640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8ac0d1260 0x7ff8ac1f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.219+0000 7ff8ab400640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8ac0b5090 0x7ff8ac155a50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00c430 tx=0x7ff89c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.219+0000 7ff8a3400640 1 -- 172.21.15.133:0/4221632356 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0074e0 con 0x7ff8ac0b5090 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.219+0000 7ff8a3400640 1 -- 172.21.15.133:0/4221632356 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c0047e0 con 0x7ff8ac0b5090 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.219+0000 7ff8a3400640 1 -- 172.21.15.133:0/4221632356 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c004bd0 con 0x7ff8ac0b5090 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.219+0000 7ff8b2600640 1 -- 172.21.15.133:0/4221632356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ac156410 con 0x7ff8ac0b5090 2024-03-07T06:52:16.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.220+0000 7ff8b2600640 1 -- 172.21.15.133:0/4221632356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ac1569d0 con 0x7ff8ac0b5090 2024-03-07T06:52:16.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.221+0000 7ff8b2600640 1 -- 172.21.15.133:0/4221632356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ac062c90 con 0x7ff8ac0b5090 2024-03-07T06:52:16.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.221+0000 7ff8a3400640 1 -- 172.21.15.133:0/4221632356 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff89c011070 con 0x7ff8ac0b5090 2024-03-07T06:52:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.222+0000 7ff8a3400640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff88806b4d0 0x7ff88806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.222+0000 7ff8a3400640 1 -- 172.21.15.133:0/4221632356 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7ff89c089fe0 con 0x7ff8ac0b5090 2024-03-07T06:52:16.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.223+0000 7ff8a3e00640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff88806b4d0 0x7ff88806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:16.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.224+0000 7ff8a3e00640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff88806b4d0 0x7ff88806d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff894036000 tx=0x7ff894008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:16.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.227+0000 7ff8a3400640 1 -- 172.21.15.133:0/4221632356 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff89c058d40 con 0x7ff8ac0b5090 2024-03-07T06:52:16.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.457+0000 7ff8b2600640 1 -- 172.21.15.133:0/4221632356 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff8ac15cfa0 con 0x7ff8ac0b5090 2024-03-07T06:52:16.456 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-03-07T06:52:16.458+0000 7fb4e8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-07T06:52:16.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.461+0000 7ff8a3400640 1 -- 172.21.15.133:0/4221632356 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff89c058640 con 0x7ff8ac0b5090 2024-03-07T06:52:16.459 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-07T06:52:16.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.464+0000 7ff8b2600640 1 -- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff88806b4d0 msgr2=0x7ff88806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:16.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.464+0000 7ff8b2600640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff88806b4d0 0x7ff88806d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff894036000 tx=0x7ff894008030 comp rx=0 tx=0).stop 2024-03-07T06:52:16.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.464+0000 7ff8b2600640 1 -- 172.21.15.133:0/4221632356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8ac0b5090 msgr2=0x7ff8ac155a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:16.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.464+0000 7ff8b2600640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8ac0b5090 0x7ff8ac155a50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00c430 tx=0x7ff89c00c8f0 comp rx=0 tx=0).stop 2024-03-07T06:52:16.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.464+0000 7ff8b2600640 1 -- 172.21.15.133:0/4221632356 shutdown_connections 2024-03-07T06:52:16.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.464+0000 7ff8b2600640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8ac147bb0 0x7ff8ac157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:16.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.464+0000 7ff8b2600640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff88806b4d0 0x7ff88806d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:16.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.464+0000 7ff8b2600640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8ac0d1260 0x7ff8ac1f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:16.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.464+0000 7ff8b2600640 1 --2- 172.21.15.133:0/4221632356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8ac0b5090 0x7ff8ac155a50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:16.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.465+0000 7ff8b2600640 1 -- 172.21.15.133:0/4221632356 >> 172.21.15.133:0/4221632356 conn(0x7ff8ac06b480 msgr2=0x7ff8ac146ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:16.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.465+0000 7ff8b2600640 1 -- 172.21.15.133:0/4221632356 shutdown_connections 2024-03-07T06:52:16.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.465+0000 7ff8b2600640 1 -- 172.21.15.133:0/4221632356 wait complete. 2024-03-07T06:52:16.481 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-07T06:52:16.508 INFO:tasks.ceph.mds.d.smithi161.stderr: -7820> 2024-03-07T06:51:20.913+0000 7f32ca200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-07T06:52:16.583 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2024-03-07T06:52:16.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.686+0000 7f0d22a00640 1 Processor -- start 2024-03-07T06:52:16.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.686+0000 7f0d22a00640 1 -- start start 2024-03-07T06:52:16.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.687+0000 7f0d22a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0d1c15ada0 0x7f0d1c0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:16.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.687+0000 7f0d22a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0d1c0ae460 0x7f0d1c0ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:16.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.687+0000 7f0d22a00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1c071310 con 0x7f0d1c06e260 2024-03-07T06:52:16.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.688+0000 7f0d22a00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1c0aee00 con 0x7f0d1c15ada0 2024-03-07T06:52:16.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.688+0000 7f0d22a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1c15f840 con 0x7f0d1c0ae460 2024-03-07T06:52:16.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.688+0000 7f0d1be00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0d1c15ada0 0x7f0d1c0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:16.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.688+0000 7f0d20a00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0d1c06e260 0x7f0d1c06e6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33380/0 (socket says 172.21.15.133:33380) 2024-03-07T06:52:16.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.688+0000 7f0d20a00640 1 -- 172.21.15.133:0/3640775329 learned_addr learned my addr 172.21.15.133:0/3640775329 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:16.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.688+0000 7f0d13e00640 1 --2- 172.21.15.133:0/3640775329 >> v2:172.21.15.161:3300/0 conn(0x7f0d1c0ae460 0x7f0d1c0ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:16.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.688+0000 7f0d1be00640 1 -- 172.21.15.133:0/3640775329 >> v2:172.21.15.161:3300/0 conn(0x7f0d1c0ae460 msgr2=0x7f0d1c0ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:16.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.688+0000 7f0d1be00640 1 --2- 172.21.15.133:0/3640775329 >> v2:172.21.15.161:3300/0 conn(0x7f0d1c0ae460 0x7f0d1c0ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:16.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.688+0000 7f0d1be00640 1 -- 172.21.15.133:0/3640775329 >> v1:172.21.15.133:6789/0 conn(0x7f0d1c06e260 legacy=0x7f0d1c06e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:16.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.688+0000 7f0d1be00640 1 -- 172.21.15.133:0/3640775329 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c15e950 con 0x7f0d1c15ada0 2024-03-07T06:52:16.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.688+0000 7f0d1b400640 1 -- 172.21.15.133:0/3640775329 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622731620 0 0) 0x7f0d14001490 con 0x7f0d1c06e260 2024-03-07T06:52:16.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.690+0000 7f0d1be00640 1 --2- 172.21.15.133:0/3640775329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d1c15ada0 0x7f0d1c0b46d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0cf9e0 tx=0x7f0d0c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c479b58313b3e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:16.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.690+0000 7f0d1b400640 1 -- 172.21.15.133:0/3640775329 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c035020 con 0x7f0d1c15ada0 2024-03-07T06:52:16.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.690+0000 7f0d1b400640 1 -- 172.21.15.133:0/3640775329 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c031830 con 0x7f0d1c15ada0 2024-03-07T06:52:16.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.690+0000 7f0d1b400640 1 -- 172.21.15.133:0/3640775329 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c031c00 con 0x7f0d1c15ada0 2024-03-07T06:52:16.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.690+0000 7f0d22a00640 1 -- 172.21.15.133:0/3640775329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d1c15ada0 msgr2=0x7f0d1c0b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:16.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.690+0000 7f0d22a00640 1 --2- 172.21.15.133:0/3640775329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d1c15ada0 0x7f0d1c0b46d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0cf9e0 tx=0x7f0d0c02b8d0 comp rx=0 tx=0).stop 2024-03-07T06:52:16.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.690+0000 7f0d22a00640 1 -- 172.21.15.133:0/3640775329 shutdown_connections 2024-03-07T06:52:16.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.690+0000 7f0d22a00640 1 --2- 172.21.15.133:0/3640775329 >> v2:172.21.15.161:3300/0 conn(0x7f0d1c0ae460 0x7f0d1c0ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:16.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.691+0000 7f0d22a00640 1 --2- 172.21.15.133:0/3640775329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d1c15ada0 0x7f0d1c0b46d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:16.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.691+0000 7f0d22a00640 1 -- 172.21.15.133:0/3640775329 >> 172.21.15.133:0/3640775329 conn(0x7f0d1c0cfb00 msgr2=0x7f0d1c0cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:16.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.691+0000 7f0d22a00640 1 -- 172.21.15.133:0/3640775329 shutdown_connections 2024-03-07T06:52:16.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.691+0000 7f0d22a00640 1 -- 172.21.15.133:0/3640775329 wait complete. 2024-03-07T06:52:16.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.692+0000 7f0d22a00640 1 Processor -- start 2024-03-07T06:52:16.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.692+0000 7f0d22a00640 1 -- start start 2024-03-07T06:52:16.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.693+0000 7f0d22a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d1c06e260 0x7f0d1c15c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:16.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.693+0000 7f0d22a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d1c0ae460 0x7f0d1c14cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:16.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.693+0000 7f0d22a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d1c15ada0 0x7f0d1c1f8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:16.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.694+0000 7f0d22a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c15f1f0 con 0x7f0d1c06e260 2024-03-07T06:52:16.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.694+0000 7f0d22a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c15ef10 con 0x7f0d1c0ae460 2024-03-07T06:52:16.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.694+0000 7f0d22a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c15f4d0 con 0x7f0d1c15ada0 2024-03-07T06:52:16.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.694+0000 7f0d1be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d1c06e260 0x7f0d1c15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:16.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.694+0000 7f0d1be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d1c06e260 0x7f0d1c15c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38858/0 (socket says 172.21.15.133:38858) 2024-03-07T06:52:16.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.694+0000 7f0d1be00640 1 -- 172.21.15.133:0/2667082637 learned_addr learned my addr 172.21.15.133:0/2667082637 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:16.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.694+0000 7f0d13e00640 1 --2- 172.21.15.133:0/2667082637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d1c0ae460 0x7f0d1c14cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:16.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.694+0000 7f0d1be00640 1 -- 172.21.15.133:0/2667082637 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d1c15ada0 msgr2=0x7f0d1c1f8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:16.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.694+0000 7f0d1be00640 1 --2- 172.21.15.133:0/2667082637 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d1c15ada0 0x7f0d1c1f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:16.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.694+0000 7f0d1be00640 1 -- 172.21.15.133:0/2667082637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d1c0ae460 msgr2=0x7f0d1c14cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:16.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.694+0000 7f0d1be00640 1 --2- 172.21.15.133:0/2667082637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d1c0ae460 0x7f0d1c14cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:16.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.694+0000 7f0d1be00640 1 -- 172.21.15.133:0/2667082637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c14d170 con 0x7f0d1c06e260 2024-03-07T06:52:16.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.694+0000 7f0d13e00640 1 --2- 172.21.15.133:0/2667082637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d1c0ae460 0x7f0d1c14cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:16.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.695+0000 7f0d1be00640 1 --2- 172.21.15.133:0/2667082637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d1c06e260 0x7f0d1c15c060 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000c00 tx=0x7f0d0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:16.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.695+0000 7f0d18c00640 1 -- 172.21.15.133:0/2667082637 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c035040 con 0x7f0d1c06e260 2024-03-07T06:52:16.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.695+0000 7f0d18c00640 1 -- 172.21.15.133:0/2667082637 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c004230 con 0x7f0d1c06e260 2024-03-07T06:52:16.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.695+0000 7f0d22a00640 1 -- 172.21.15.133:0/2667082637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1c15dc30 con 0x7f0d1c06e260 2024-03-07T06:52:16.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.695+0000 7f0d18c00640 1 -- 172.21.15.133:0/2667082637 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c03a700 con 0x7f0d1c06e260 2024-03-07T06:52:16.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.695+0000 7f0d22a00640 1 -- 172.21.15.133:0/2667082637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1c15e1f0 con 0x7f0d1c06e260 2024-03-07T06:52:16.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.696+0000 7f0d22a00640 1 -- 172.21.15.133:0/2667082637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1c062c90 con 0x7f0d1c06e260 2024-03-07T06:52:16.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.697+0000 7f0d18c00640 1 -- 172.21.15.133:0/2667082637 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0d0c038020 con 0x7f0d1c06e260 2024-03-07T06:52:16.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.698+0000 7f0d18c00640 1 --2- 172.21.15.133:0/2667082637 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0cf806b4d0 0x7f0cf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:16.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.698+0000 7f0d18c00640 1 -- 172.21.15.133:0/2667082637 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f0d0c0abfc0 con 0x7f0d1c06e260 2024-03-07T06:52:16.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.699+0000 7f0d13e00640 1 --2- 172.21.15.133:0/2667082637 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0cf806b4d0 0x7f0cf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:16.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.700+0000 7f0d13e00640 1 --2- 172.21.15.133:0/2667082637 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0cf806b4d0 0x7f0cf806d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0b3c90 tx=0x7f0d04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:16.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.703+0000 7f0d18c00640 1 -- 172.21.15.133:0/2667082637 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0d1c14d170 con 0x7f0d1c06e260 2024-03-07T06:52:16.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:16.919+0000 7f0d22a00640 1 -- 172.21.15.133:0/2667082637 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0d1c15b430 con 0x7f0d1c06e260 2024-03-07T06:52:17.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.567+0000 7f0d18c00640 1 -- 172.21.15.133:0/2667082637 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0d0c07ae00 con 0x7f0d1c06e260 2024-03-07T06:52:17.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.570+0000 7f0d22a00640 1 -- 172.21.15.133:0/2667082637 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0cf806b4d0 msgr2=0x7f0cf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.570+0000 7f0d22a00640 1 --2- 172.21.15.133:0/2667082637 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0cf806b4d0 0x7f0cf806d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0b3c90 tx=0x7f0d04008040 comp rx=0 tx=0).stop 2024-03-07T06:52:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.570+0000 7f0d22a00640 1 -- 172.21.15.133:0/2667082637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d1c06e260 msgr2=0x7f0d1c15c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.570+0000 7f0d22a00640 1 --2- 172.21.15.133:0/2667082637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d1c06e260 0x7f0d1c15c060 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c000c00 tx=0x7f0d0c0046c0 comp rx=0 tx=0).stop 2024-03-07T06:52:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.570+0000 7f0d22a00640 1 -- 172.21.15.133:0/2667082637 shutdown_connections 2024-03-07T06:52:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.570+0000 7f0d22a00640 1 --2- 172.21.15.133:0/2667082637 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d1c15ada0 0x7f0d1c1f8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.571+0000 7f0d22a00640 1 --2- 172.21.15.133:0/2667082637 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0cf806b4d0 0x7f0cf806d990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.571+0000 7f0d22a00640 1 --2- 172.21.15.133:0/2667082637 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d1c0ae460 0x7f0d1c14cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.571+0000 7f0d22a00640 1 --2- 172.21.15.133:0/2667082637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d1c06e260 0x7f0d1c15c060 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.571+0000 7f0d22a00640 1 -- 172.21.15.133:0/2667082637 >> 172.21.15.133:0/2667082637 conn(0x7f0d1c0cfb00 msgr2=0x7f0d1c0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.571+0000 7f0d22a00640 1 -- 172.21.15.133:0/2667082637 shutdown_connections 2024-03-07T06:52:17.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.571+0000 7f0d22a00640 1 -- 172.21.15.133:0/2667082637 wait complete. 2024-03-07T06:52:17.590 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-07T06:52:17.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.788+0000 7f5da2800640 1 Processor -- start 2024-03-07T06:52:17.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.788+0000 7f5da2800640 1 -- start start 2024-03-07T06:52:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.790+0000 7f5da2800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5d9c1600a0 0x7f5d9c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.790+0000 7f5da2800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c06e710 con 0x7f5d9c15be20 2024-03-07T06:52:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.790+0000 7f5da2800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c160580 con 0x7f5d9c0af6b0 2024-03-07T06:52:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.790+0000 7f5da2800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c06e9b0 con 0x7f5d9c1600a0 2024-03-07T06:52:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.790+0000 7f5d9b400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5d9c15be20 0x7f5d9c0aefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33394/0 (socket says 172.21.15.133:33394) 2024-03-07T06:52:17.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.790+0000 7f5d9b400640 1 -- 172.21.15.133:0/142748394 learned_addr learned my addr 172.21.15.133:0/142748394 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:17.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9be00640 1 --2- 172.21.15.133:0/142748394 >> v2:172.21.15.161:3300/0 conn(0x7f5d9c1600a0 0x7f5d9c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:17.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 -- 172.21.15.133:0/142748394 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562826521 0 0) 0x7f5d9c06e710 con 0x7f5d9c15be20 2024-03-07T06:52:17.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 -- 172.21.15.133:0/142748394 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d80003520 con 0x7f5d9c15be20 2024-03-07T06:52:17.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 -- 172.21.15.133:0/142748394 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1470616686 0 0) 0x7f5d80003520 con 0x7f5d9c15be20 2024-03-07T06:52:17.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 -- 172.21.15.133:0/142748394 >> v2:172.21.15.161:3300/0 conn(0x7f5d9c1600a0 msgr2=0x7f5d9c06f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:17.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 --2- 172.21.15.133:0/142748394 >> v2:172.21.15.161:3300/0 conn(0x7f5d9c1600a0 0x7f5d9c06f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:17.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 -- 172.21.15.133:0/142748394 >> v1:172.21.15.161:6789/0 conn(0x7f5d9c0af6b0 legacy=0x7f5d9c15f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:17.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 -- 172.21.15.133:0/142748394 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c06ec90 con 0x7f5d9c15be20 2024-03-07T06:52:17.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 -- 172.21.15.133:0/142748394 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f5d8c001690 con 0x7f5d9c15be20 2024-03-07T06:52:17.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 -- 172.21.15.133:0/142748394 >> v1:172.21.15.133:6789/0 conn(0x7f5d9c15be20 legacy=0x7f5d9c0aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:17.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 --2- 172.21.15.133:0/142748394 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d80003a00 0x7f5d80003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:17.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 --2- 172.21.15.133:0/142748394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80004800 0x7f5d80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:17.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 --2- 172.21.15.133:0/142748394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d800051c0 0x7f5d8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:17.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 -- 172.21.15.133:0/142748394 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c06e9b0 con 0x7f5d80004800 2024-03-07T06:52:17.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 -- 172.21.15.133:0/142748394 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8000c0c0 con 0x7f5d800051c0 2024-03-07T06:52:17.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.791+0000 7f5d9a000640 1 -- 172.21.15.133:0/142748394 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8000c2c0 con 0x7f5d80003a00 2024-03-07T06:52:17.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.792+0000 7f5d9aa00640 1 --2- 172.21.15.133:0/142748394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80004800 0x7f5d80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:17.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.792+0000 7f5d9be00640 1 --2- 172.21.15.133:0/142748394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d800051c0 0x7f5d8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:17.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.792+0000 7f5d9b400640 1 --2- 172.21.15.133:0/142748394 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d80003a00 0x7f5d80003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:17.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.792+0000 7f5d9aa00640 1 -- 172.21.15.133:0/142748394 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d80003a00 msgr2=0x7f5d80003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:17.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.792+0000 7f5d9aa00640 1 --2- 172.21.15.133:0/142748394 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d80003a00 0x7f5d80003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:17.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.792+0000 7f5d9aa00640 1 -- 172.21.15.133:0/142748394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d800051c0 msgr2=0x7f5d8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:17.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.792+0000 7f5d9aa00640 1 --2- 172.21.15.133:0/142748394 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d800051c0 0x7f5d8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:17.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.792+0000 7f5d9aa00640 1 -- 172.21.15.133:0/142748394 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d80003520 con 0x7f5d80004800 2024-03-07T06:52:17.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.792+0000 7f5d9be00640 1 -- 172.21.15.133:0/142748394 reap_dead start 2024-03-07T06:52:17.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.793+0000 7f5d9aa00640 1 --2- 172.21.15.133:0/142748394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80004800 0x7f5d80004c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5d88001750 tx=0x7f5d80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397d12de1ed00e16 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:17.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.793+0000 7f5d9a000640 1 -- 172.21.15.133:0/142748394 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88001370 con 0x7f5d80004800 2024-03-07T06:52:17.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.793+0000 7f5d9a000640 1 -- 172.21.15.133:0/142748394 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d880052d0 con 0x7f5d80004800 2024-03-07T06:52:17.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.793+0000 7f5d9a000640 1 -- 172.21.15.133:0/142748394 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d88003830 con 0x7f5d80004800 2024-03-07T06:52:17.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.794+0000 7f5da2800640 1 -- 172.21.15.133:0/142748394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80004800 msgr2=0x7f5d80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:17.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.794+0000 7f5da2800640 1 --2- 172.21.15.133:0/142748394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80004800 0x7f5d80004c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5d88001750 tx=0x7f5d80005830 comp rx=0 tx=0).stop 2024-03-07T06:52:17.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.794+0000 7f5da2800640 1 -- 172.21.15.133:0/142748394 shutdown_connections 2024-03-07T06:52:17.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.794+0000 7f5da2800640 1 --2- 172.21.15.133:0/142748394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80004800 0x7f5d80004c80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:17.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.794+0000 7f5da2800640 1 -- 172.21.15.133:0/142748394 >> 172.21.15.133:0/142748394 conn(0x7f5d9c0b43f0 msgr2=0x7f5d9c0b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.794+0000 7f5da2800640 1 -- 172.21.15.133:0/142748394 shutdown_connections 2024-03-07T06:52:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.794+0000 7f5da2800640 1 -- 172.21.15.133:0/142748394 wait complete. 2024-03-07T06:52:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.796+0000 7f5da2800640 1 Processor -- start 2024-03-07T06:52:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.796+0000 7f5da2800640 1 -- start start 2024-03-07T06:52:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.797+0000 7f5da2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80004800 0x7f5d9c15a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.797+0000 7f5da2800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d9c1452e0 0x7f5d9c145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.797+0000 7f5da2800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d9c145fa0 0x7f5d9c1463e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.797+0000 7f5da2800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c06f330 con 0x7f5d80004800 2024-03-07T06:52:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.797+0000 7f5da2800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c06f1b0 con 0x7f5d9c145fa0 2024-03-07T06:52:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.797+0000 7f5da2800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c06f4b0 con 0x7f5d9c1452e0 2024-03-07T06:52:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.797+0000 7f5d9b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80004800 0x7f5d9c15a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.797+0000 7f5d9b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80004800 0x7f5d9c15a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38882/0 (socket says 172.21.15.133:38882) 2024-03-07T06:52:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.797+0000 7f5d9b400640 1 -- 172.21.15.133:0/1042792788 learned_addr learned my addr 172.21.15.133:0/1042792788 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:17.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.798+0000 7f5d9b400640 1 -- 172.21.15.133:0/1042792788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d9c1452e0 msgr2=0x7f5d9c145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.798+0000 7f5d9b400640 1 --2- 172.21.15.133:0/1042792788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d9c1452e0 0x7f5d9c145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.798+0000 7f5d9b400640 1 -- 172.21.15.133:0/1042792788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d9c145fa0 msgr2=0x7f5d9c1463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.798+0000 7f5d9be00640 1 --2- 172.21.15.133:0/1042792788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d9c145fa0 0x7f5d9c1463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.798+0000 7f5d9b400640 1 --2- 172.21.15.133:0/1042792788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d9c145fa0 0x7f5d9c1463e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.798+0000 7f5d9b400640 1 -- 172.21.15.133:0/1042792788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c145ca0 con 0x7f5d80004800 2024-03-07T06:52:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.798+0000 7f5d9b400640 1 --2- 172.21.15.133:0/1042792788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80004800 0x7f5d9c15a6d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00d7a0 tx=0x7f5d8c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.798+0000 7f5d93400640 1 -- 172.21.15.133:0/1042792788 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c019070 con 0x7f5d80004800 2024-03-07T06:52:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.798+0000 7f5d93400640 1 -- 172.21.15.133:0/1042792788 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c0072d0 con 0x7f5d80004800 2024-03-07T06:52:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.798+0000 7f5d93400640 1 -- 172.21.15.133:0/1042792788 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c0047e0 con 0x7f5d80004800 2024-03-07T06:52:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.798+0000 7f5da2800640 1 -- 172.21.15.133:0/1042792788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c146c60 con 0x7f5d80004800 2024-03-07T06:52:17.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.799+0000 7f5da2800640 1 -- 172.21.15.133:0/1042792788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c147220 con 0x7f5d80004800 2024-03-07T06:52:17.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.800+0000 7f5da2800640 1 -- 172.21.15.133:0/1042792788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c062c90 con 0x7f5d80004800 2024-03-07T06:52:17.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.800+0000 7f5d93400640 1 -- 172.21.15.133:0/1042792788 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5d8c01c070 con 0x7f5d80004800 2024-03-07T06:52:17.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.801+0000 7f5d93400640 1 --2- 172.21.15.133:0/1042792788 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5d7c06b4d0 0x7f5d7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:17.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.801+0000 7f5d93400640 1 -- 172.21.15.133:0/1042792788 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f5d8c089c60 con 0x7f5d80004800 2024-03-07T06:52:17.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.802+0000 7f5d9aa00640 1 --2- 172.21.15.133:0/1042792788 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5d7c06b4d0 0x7f5d7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:17.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.803+0000 7f5d9aa00640 1 --2- 172.21.15.133:0/1042792788 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5d7c06b4d0 0x7f5d7c06d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5d88033000 tx=0x7f5d8802f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:17.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.805+0000 7f5d93400640 1 -- 172.21.15.133:0/1042792788 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5d8c058a10 con 0x7f5d80004800 2024-03-07T06:52:17.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:17.990+0000 7f5da2800640 1 -- 172.21.15.133:0/1042792788 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5d9c06b2f0 con 0x7f5d80004800 2024-03-07T06:52:18.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.578+0000 7f5d93400640 1 -- 172.21.15.133:0/1042792788 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5d8c058310 con 0x7f5d80004800 2024-03-07T06:52:18.578 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-03-07T06:52:18.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.584+0000 7f5da2800640 1 -- 172.21.15.133:0/1042792788 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5d7c06b4d0 msgr2=0x7f5d7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:18.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.584+0000 7f5da2800640 1 --2- 172.21.15.133:0/1042792788 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5d7c06b4d0 0x7f5d7c06d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5d88033000 tx=0x7f5d8802f000 comp rx=0 tx=0).stop 2024-03-07T06:52:18.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.584+0000 7f5da2800640 1 -- 172.21.15.133:0/1042792788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80004800 msgr2=0x7f5d9c15a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:18.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.584+0000 7f5da2800640 1 --2- 172.21.15.133:0/1042792788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80004800 0x7f5d9c15a6d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00d7a0 tx=0x7f5d8c00dc60 comp rx=0 tx=0).stop 2024-03-07T06:52:18.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.585+0000 7f5da2800640 1 -- 172.21.15.133:0/1042792788 shutdown_connections 2024-03-07T06:52:18.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.585+0000 7f5da2800640 1 --2- 172.21.15.133:0/1042792788 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5d7c06b4d0 0x7f5d7c06d990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:18.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.585+0000 7f5da2800640 1 --2- 172.21.15.133:0/1042792788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5d9c145fa0 0x7f5d9c1463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:18.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.585+0000 7f5da2800640 1 --2- 172.21.15.133:0/1042792788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5d9c1452e0 0x7f5d9c145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:18.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.585+0000 7f5da2800640 1 --2- 172.21.15.133:0/1042792788 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d80004800 0x7f5d9c15a6d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:18.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.585+0000 7f5da2800640 1 -- 172.21.15.133:0/1042792788 >> 172.21.15.133:0/1042792788 conn(0x7f5d9c0b43f0 msgr2=0x7f5d9c157e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:18.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.585+0000 7f5da2800640 1 -- 172.21.15.133:0/1042792788 shutdown_connections 2024-03-07T06:52:18.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.585+0000 7f5da2800640 1 -- 172.21.15.133:0/1042792788 wait complete. 2024-03-07T06:52:18.608 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-07T06:52:18.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.818+0000 7fb04dc00640 1 Processor -- start 2024-03-07T06:52:18.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.818+0000 7fb04dc00640 1 -- start start 2024-03-07T06:52:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.820+0000 7fb04dc00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb0480b1c50 0x7fb0480b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.820+0000 7fb04dc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0480b52a0 0x7fb04815bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.820+0000 7fb04dc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb04815e8f0 0x7fb0480b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.820+0000 7fb04dc00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0480b2570 con 0x7fb0480b1c50 2024-03-07T06:52:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.820+0000 7fb04dc00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04815f800 con 0x7fb04815e8f0 2024-03-07T06:52:18.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.820+0000 7fb04dc00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04815f240 con 0x7fb0480b52a0 2024-03-07T06:52:18.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.820+0000 7fb046a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb0480b1c50 0x7fb0480b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:18.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.820+0000 7fb046a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb0480b1c50 0x7fb0480b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38898/0 (socket says 172.21.15.133:38898) 2024-03-07T06:52:18.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.820+0000 7fb046a00640 1 -- 172.21.15.133:0/3606246468 learned_addr learned my addr 172.21.15.133:0/3606246468 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:18.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.821+0000 7fb046a00640 1 -- 172.21.15.133:0/3606246468 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0480b52a0 msgr2=0x7fb04815bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:18.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.821+0000 7fb046a00640 1 --2- 172.21.15.133:0/3606246468 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0480b52a0 0x7fb04815bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:18.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.821+0000 7fb046a00640 1 -- 172.21.15.133:0/3606246468 >> v2:172.21.15.161:3300/0 conn(0x7fb04815e8f0 msgr2=0x7fb0480b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:18.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.821+0000 7fb046a00640 1 --2- 172.21.15.133:0/3606246468 >> v2:172.21.15.161:3300/0 conn(0x7fb04815e8f0 0x7fb0480b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:18.820 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.821+0000 7fb046a00640 1 -- 172.21.15.133:0/3606246468 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04815f520 con 0x7fb0480b1c50 2024-03-07T06:52:18.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.822+0000 7fb046a00640 1 --2- 172.21.15.133:0/3606246468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0480b1c50 0x7fb0480b2030 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb0480cd3d0 tx=0x7fb03002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155d866008cc4f51 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:18.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.822+0000 7fb045600640 1 -- 172.21.15.133:0/3606246468 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030035020 con 0x7fb0480b1c50 2024-03-07T06:52:18.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.822+0000 7fb045600640 1 -- 172.21.15.133:0/3606246468 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb030031870 con 0x7fb0480b1c50 2024-03-07T06:52:18.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.822+0000 7fb045600640 1 -- 172.21.15.133:0/3606246468 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb030031c40 con 0x7fb0480b1c50 2024-03-07T06:52:18.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.823+0000 7fb04dc00640 1 -- 172.21.15.133:0/3606246468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0480b1c50 msgr2=0x7fb0480b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:18.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.823+0000 7fb04dc00640 1 --2- 172.21.15.133:0/3606246468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0480b1c50 0x7fb0480b2030 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb0480cd3d0 tx=0x7fb03002b910 comp rx=0 tx=0).stop 2024-03-07T06:52:18.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.823+0000 7fb04dc00640 1 -- 172.21.15.133:0/3606246468 shutdown_connections 2024-03-07T06:52:18.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.823+0000 7fb04dc00640 1 --2- 172.21.15.133:0/3606246468 >> v2:172.21.15.161:3300/0 conn(0x7fb04815e8f0 0x7fb0480b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:18.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.823+0000 7fb04dc00640 1 --2- 172.21.15.133:0/3606246468 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0480b52a0 0x7fb04815bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:18.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.823+0000 7fb04dc00640 1 --2- 172.21.15.133:0/3606246468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0480b1c50 0x7fb0480b2030 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:18.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.823+0000 7fb04dc00640 1 -- 172.21.15.133:0/3606246468 >> 172.21.15.133:0/3606246468 conn(0x7fb048066eb0 msgr2=0x7fb048067290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:18.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.823+0000 7fb04dc00640 1 -- 172.21.15.133:0/3606246468 shutdown_connections 2024-03-07T06:52:18.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.823+0000 7fb04dc00640 1 -- 172.21.15.133:0/3606246468 wait complete. 2024-03-07T06:52:18.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.825+0000 7fb04dc00640 1 Processor -- start 2024-03-07T06:52:18.823 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.825+0000 7fb04dc00640 1 -- start start 2024-03-07T06:52:18.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.826+0000 7fb04dc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0480b1c50 0x7fb048155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:18.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.826+0000 7fb04dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0480b52a0 0x7fb0480c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:18.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.826+0000 7fb04dc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb04815e8f0 0x7fb0480c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:18.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.826+0000 7fb04dc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04815c150 con 0x7fb0480b52a0 2024-03-07T06:52:18.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.826+0000 7fb04dc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04815b6f0 con 0x7fb04815e8f0 2024-03-07T06:52:18.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.826+0000 7fb04dc00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04815d990 con 0x7fb0480b1c50 2024-03-07T06:52:18.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.826+0000 7fb046000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0480b52a0 0x7fb0480c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:18.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.826+0000 7fb046000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0480b52a0 0x7fb0480c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:38914/0 (socket says 172.21.15.133:38914) 2024-03-07T06:52:18.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.826+0000 7fb046000640 1 -- 172.21.15.133:0/2268133659 learned_addr learned my addr 172.21.15.133:0/2268133659 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:18.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.827+0000 7fb046000640 1 -- 172.21.15.133:0/2268133659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0480b1c50 msgr2=0x7fb048155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:18.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.827+0000 7fb046000640 1 --2- 172.21.15.133:0/2268133659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0480b1c50 0x7fb048155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:18.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.827+0000 7fb046000640 1 -- 172.21.15.133:0/2268133659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb04815e8f0 msgr2=0x7fb0480c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:18.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.827+0000 7fb046000640 1 --2- 172.21.15.133:0/2268133659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb04815e8f0 0x7fb0480c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:18.826 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.827+0000 7fb046000640 1 -- 172.21.15.133:0/2268133659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04815dc70 con 0x7fb0480b52a0 2024-03-07T06:52:18.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.827+0000 7fb046000640 1 --2- 172.21.15.133:0/2268133659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0480b52a0 0x7fb0480c5fd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb03800ba80 tx=0x7fb03800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:18.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.827+0000 7fb03ea00640 1 -- 172.21.15.133:0/2268133659 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038019070 con 0x7fb0480b52a0 2024-03-07T06:52:18.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.827+0000 7fb03ea00640 1 -- 172.21.15.133:0/2268133659 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03800cc20 con 0x7fb0480b52a0 2024-03-07T06:52:18.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.827+0000 7fb03ea00640 1 -- 172.21.15.133:0/2268133659 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038014440 con 0x7fb0480b52a0 2024-03-07T06:52:18.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.827+0000 7fb04dc00640 1 -- 172.21.15.133:0/2268133659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04815df50 con 0x7fb0480b52a0 2024-03-07T06:52:18.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.827+0000 7fb04dc00640 1 -- 172.21.15.133:0/2268133659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04815d580 con 0x7fb0480b52a0 2024-03-07T06:52:18.827 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.829+0000 7fb04dc00640 1 -- 172.21.15.133:0/2268133659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04815e470 con 0x7fb0480b52a0 2024-03-07T06:52:18.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.830+0000 7fb03ea00640 1 -- 172.21.15.133:0/2268133659 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb03801c070 con 0x7fb0480b52a0 2024-03-07T06:52:18.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.830+0000 7fb03ea00640 1 --2- 172.21.15.133:0/2268133659 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb02406b4d0 0x7fb02406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:18.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.830+0000 7fb03ea00640 1 -- 172.21.15.133:0/2268133659 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5473+0+0 (secure 0 0 0) 0x7fb038089dc0 con 0x7fb0480b52a0 2024-03-07T06:52:18.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.831+0000 7fb046a00640 1 --2- 172.21.15.133:0/2268133659 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb02406b4d0 0x7fb02406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:18.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.832+0000 7fb046a00640 1 --2- 172.21.15.133:0/2268133659 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb02406b4d0 0x7fb02406d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb030036000 tx=0x7fb030030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:18.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:18.835+0000 7fb03ea00640 1 -- 172.21.15.133:0/2268133659 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb0380586d0 con 0x7fb0480b52a0 2024-03-07T06:52:19.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:19.038+0000 7fb04dc00640 1 -- 172.21.15.133:0/2268133659 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb048062c90 con 0x7fb0480b52a0 2024-03-07T06:52:19.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:19.582+0000 7fb03ea00640 1 -- 172.21.15.133:0/2268133659 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb03805c770 con 0x7fb0480b52a0 2024-03-07T06:52:19.581 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-03-07T06:52:19.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:19.586+0000 7fb04dc00640 1 -- 172.21.15.133:0/2268133659 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb02406b4d0 msgr2=0x7fb02406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:19.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:19.586+0000 7fb04dc00640 1 --2- 172.21.15.133:0/2268133659 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb02406b4d0 0x7fb02406d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb030036000 tx=0x7fb030030000 comp rx=0 tx=0).stop 2024-03-07T06:52:19.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:19.586+0000 7fb04dc00640 1 -- 172.21.15.133:0/2268133659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0480b52a0 msgr2=0x7fb0480c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:19.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:19.586+0000 7fb04dc00640 1 --2- 172.21.15.133:0/2268133659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0480b52a0 0x7fb0480c5fd0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb03800ba80 tx=0x7fb03800bf40 comp rx=0 tx=0).stop 2024-03-07T06:52:19.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:19.587+0000 7fb04dc00640 1 -- 172.21.15.133:0/2268133659 shutdown_connections 2024-03-07T06:52:19.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:19.587+0000 7fb04dc00640 1 --2- 172.21.15.133:0/2268133659 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb02406b4d0 0x7fb02406d990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:19.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:19.587+0000 7fb04dc00640 1 --2- 172.21.15.133:0/2268133659 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb04815e8f0 0x7fb0480c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:19.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:19.587+0000 7fb04dc00640 1 --2- 172.21.15.133:0/2268133659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0480b52a0 0x7fb0480c5fd0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:19.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:19.587+0000 7fb04dc00640 1 --2- 172.21.15.133:0/2268133659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb0480b1c50 0x7fb048155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:19.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:19.587+0000 7fb04dc00640 1 -- 172.21.15.133:0/2268133659 >> 172.21.15.133:0/2268133659 conn(0x7fb048066eb0 msgr2=0x7fb0480b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:19.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:19.588+0000 7fb04dc00640 1 -- 172.21.15.133:0/2268133659 shutdown_connections 2024-03-07T06:52:19.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:19.588+0000 7fb04dc00640 1 -- 172.21.15.133:0/2268133659 wait complete. 2024-03-07T06:52:19.613 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-07T06:52:19.613 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-07T06:52:19.613 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-07T06:52:19.613 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:52:19.614 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-07T06:52:19.614 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-07T06:52:19.614 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-07T06:52:19.614 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:52:19.614 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-07T06:52:19.614 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-07T06:52:19.614 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-07T06:52:19.614 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:52:19.615 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-07T06:52:19.615 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-07T06:52:19.615 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-07T06:52:19.615 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:52:19.633 INFO:tasks.ceph.mds.c:Stopped 2024-03-07T06:52:19.633 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-07T06:52:19.635 INFO:tasks.ceph.mds.b:Stopped 2024-03-07T06:52:19.635 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-07T06:52:19.637 INFO:tasks.ceph.mds.a:Stopped 2024-03-07T06:52:19.637 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-07T06:52:19.683 INFO:tasks.ceph.mds.b:Started 2024-03-07T06:52:19.683 DEBUG:teuthology.parallel:result is None 2024-03-07T06:52:19.684 INFO:tasks.ceph.mds.c:Started 2024-03-07T06:52:19.684 DEBUG:teuthology.parallel:result is None 2024-03-07T06:52:19.690 INFO:tasks.ceph.mds.a:Started 2024-03-07T06:52:19.690 DEBUG:teuthology.parallel:result is None 2024-03-07T06:52:19.843 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-03-07T06:52:19.845 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-03-07T06:52:19.864 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2024-03-07T06:52:20.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-03-07T06:52:25.637 INFO:tasks.ceph.mds.d:Stopped 2024-03-07T06:52:25.637 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-07T06:52:25.645 INFO:tasks.ceph.mds.d:Started 2024-03-07T06:52:25.646 DEBUG:teuthology.parallel:result is None 2024-03-07T06:52:25.646 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-07T06:52:25.823 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2024-03-07T06:52:25.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.870+0000 7fd104400640 1 Processor -- start 2024-03-07T06:52:25.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.870+0000 7fd104400640 1 -- start start 2024-03-07T06:52:25.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.872+0000 7fd104400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0fc15fbb0 0x7fd0fc0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:25.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.872+0000 7fd104400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd0fc072950 0x7fd0fc0cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:25.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.872+0000 7fd104400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd0fc15ff90 0x7fd0fc0acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:25.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.872+0000 7fd104400640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0fc072f60 con 0x7fd0fc15ff90 2024-03-07T06:52:25.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.872+0000 7fd104400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0fc073490 con 0x7fd0fc072950 2024-03-07T06:52:25.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.872+0000 7fd104400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc0731b0 con 0x7fd0fc15fbb0 2024-03-07T06:52:25.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.872+0000 7fd102400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd0fc15ff90 0x7fd0fc0acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:25.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.872+0000 7fd102400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd0fc15ff90 0x7fd0fc0acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36546/0 (socket says 172.21.15.133:36546) 2024-03-07T06:52:25.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.872+0000 7fd102400640 1 -- 172.21.15.133:0/2481384799 learned_addr learned my addr 172.21.15.133:0/2481384799 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:25.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.872+0000 7fd101000640 1 --2- 172.21.15.133:0/2481384799 >> v2:172.21.15.161:3300/0 conn(0x7fd0fc072950 0x7fd0fc0cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:25.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.872+0000 7fd101a00640 1 --2- 172.21.15.133:0/2481384799 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0fc15fbb0 0x7fd0fc0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:25.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.873+0000 7fd102400640 1 -- 172.21.15.133:0/2481384799 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0fc15fbb0 msgr2=0x7fd0fc0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:25.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.873+0000 7fd102400640 1 --2- 172.21.15.133:0/2481384799 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0fc15fbb0 0x7fd0fc0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:25.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.873+0000 7fd102400640 1 -- 172.21.15.133:0/2481384799 >> v2:172.21.15.161:3300/0 conn(0x7fd0fc072950 msgr2=0x7fd0fc0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:25.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.873+0000 7fd102400640 1 --2- 172.21.15.133:0/2481384799 >> v2:172.21.15.161:3300/0 conn(0x7fd0fc072950 0x7fd0fc0cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:25.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.873+0000 7fd102400640 1 -- 172.21.15.133:0/2481384799 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc072150 con 0x7fd0fc15ff90 2024-03-07T06:52:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.874+0000 7fd102400640 1 --2- 172.21.15.133:0/2481384799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0fc15ff90 0x7fd0fc0acd00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc067e20 tx=0x7fd0f402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b2874d16da49cc server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.874+0000 7fd0fbe00640 1 -- 172.21.15.133:0/2481384799 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f402e070 con 0x7fd0fc15ff90 2024-03-07T06:52:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.874+0000 7fd0fbe00640 1 -- 172.21.15.133:0/2481384799 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f4037070 con 0x7fd0fc15ff90 2024-03-07T06:52:25.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.874+0000 7fd0fbe00640 1 -- 172.21.15.133:0/2481384799 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4031a00 con 0x7fd0fc15ff90 2024-03-07T06:52:25.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.875+0000 7fd104400640 1 -- 172.21.15.133:0/2481384799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0fc15ff90 msgr2=0x7fd0fc0acd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:25.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.875+0000 7fd104400640 1 --2- 172.21.15.133:0/2481384799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0fc15ff90 0x7fd0fc0acd00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc067e20 tx=0x7fd0f402b880 comp rx=0 tx=0).stop 2024-03-07T06:52:25.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.875+0000 7fd104400640 1 -- 172.21.15.133:0/2481384799 shutdown_connections 2024-03-07T06:52:25.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.875+0000 7fd104400640 1 --2- 172.21.15.133:0/2481384799 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0fc15ff90 0x7fd0fc0acd00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:25.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.875+0000 7fd104400640 1 --2- 172.21.15.133:0/2481384799 >> v2:172.21.15.161:3300/0 conn(0x7fd0fc072950 0x7fd0fc0cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:25.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.875+0000 7fd104400640 1 --2- 172.21.15.133:0/2481384799 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0fc15fbb0 0x7fd0fc0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:25.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.875+0000 7fd104400640 1 -- 172.21.15.133:0/2481384799 >> 172.21.15.133:0/2481384799 conn(0x7fd0fc0b3120 msgr2=0x7fd0fc0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:25.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.875+0000 7fd104400640 1 -- 172.21.15.133:0/2481384799 shutdown_connections 2024-03-07T06:52:25.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.875+0000 7fd104400640 1 -- 172.21.15.133:0/2481384799 wait complete. 2024-03-07T06:52:25.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.877+0000 7fd104400640 1 Processor -- start 2024-03-07T06:52:25.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.877+0000 7fd104400640 1 -- start start 2024-03-07T06:52:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.878+0000 7fd104400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0fc15fbb0 0x7fd0fc152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.878+0000 7fd104400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0fc15ff90 0x7fd0fc15eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.878+0000 7fd104400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0fc09c5b0 0x7fd0fc09d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.878+0000 7fd104400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc0b1a00 con 0x7fd0fc09c5b0 2024-03-07T06:52:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.878+0000 7fd104400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc160980 con 0x7fd0fc15ff90 2024-03-07T06:52:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.878+0000 7fd104400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc1603a0 con 0x7fd0fc15fbb0 2024-03-07T06:52:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.879+0000 7fd102400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0fc09c5b0 0x7fd0fc09d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.879+0000 7fd102400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0fc09c5b0 0x7fd0fc09d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36552/0 (socket says 172.21.15.133:36552) 2024-03-07T06:52:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.879+0000 7fd102400640 1 -- 172.21.15.133:0/2205556169 learned_addr learned my addr 172.21.15.133:0/2205556169 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.879+0000 7fd101a00640 1 --2- 172.21.15.133:0/2205556169 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0fc15fbb0 0x7fd0fc152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:25.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.879+0000 7fd101000640 1 --2- 172.21.15.133:0/2205556169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0fc15ff90 0x7fd0fc15eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.879+0000 7fd102400640 1 -- 172.21.15.133:0/2205556169 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0fc15fbb0 msgr2=0x7fd0fc152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.879+0000 7fd102400640 1 --2- 172.21.15.133:0/2205556169 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0fc15fbb0 0x7fd0fc152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.879+0000 7fd102400640 1 -- 172.21.15.133:0/2205556169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0fc15ff90 msgr2=0x7fd0fc15eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.879+0000 7fd102400640 1 --2- 172.21.15.133:0/2205556169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0fc15ff90 0x7fd0fc15eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.879+0000 7fd102400640 1 -- 172.21.15.133:0/2205556169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc152dc0 con 0x7fd0fc09c5b0 2024-03-07T06:52:25.880 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.879+0000 7fd102400640 1 --2- 172.21.15.133:0/2205556169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0fc09c5b0 0x7fd0fc09d110 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40090b0 tx=0x7fd0f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.880+0000 7fd0f9600640 1 -- 172.21.15.133:0/2205556169 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4032670 con 0x7fd0fc09c5b0 2024-03-07T06:52:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.880+0000 7fd0f9600640 1 -- 172.21.15.133:0/2205556169 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f4032810 con 0x7fd0fc09c5b0 2024-03-07T06:52:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.880+0000 7fd0f9600640 1 -- 172.21.15.133:0/2205556169 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4032c00 con 0x7fd0fc09c5b0 2024-03-07T06:52:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.880+0000 7fd104400640 1 -- 172.21.15.133:0/2205556169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0fc0ca760 con 0x7fd0fc09c5b0 2024-03-07T06:52:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.880+0000 7fd104400640 1 -- 172.21.15.133:0/2205556169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0fc09bdb0 con 0x7fd0fc09c5b0 2024-03-07T06:52:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.881+0000 7fd104400640 1 -- 172.21.15.133:0/2205556169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0fc062c90 con 0x7fd0fc09c5b0 2024-03-07T06:52:25.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.882+0000 7fd0f9600640 1 -- 172.21.15.133:0/2205556169 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd0f404c060 con 0x7fd0fc09c5b0 2024-03-07T06:52:25.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.883+0000 7fd0f9600640 1 --2- 172.21.15.133:0/2205556169 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd0dc06b4d0 0x7fd0dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:25.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.883+0000 7fd0f9600640 1 -- 172.21.15.133:0/2205556169 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd0f40ab950 con 0x7fd0fc09c5b0 2024-03-07T06:52:25.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.883+0000 7fd101a00640 1 --2- 172.21.15.133:0/2205556169 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd0dc06b4d0 0x7fd0dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:25.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.884+0000 7fd101a00640 1 --2- 172.21.15.133:0/2205556169 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd0dc06b4d0 0x7fd0dc06d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec006fd0 tx=0x7fd0ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:25.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:25.888+0000 7fd0f9600640 1 -- 172.21.15.133:0/2205556169 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd0f4040070 con 0x7fd0fc09c5b0 2024-03-07T06:52:26.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:26.097+0000 7fd104400640 1 -- 172.21.15.133:0/2205556169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd0fc1528a0 con 0x7fd0fc09c5b0 2024-03-07T06:52:27.165 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-03-07T06:52:27.654 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.654+0000 7fd0f9600640 1 -- 172.21.15.133:0/2205556169 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd0f407aa30 con 0x7fd0fc09c5b0 2024-03-07T06:52:27.655 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-03-07T06:52:27.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.659+0000 7fd104400640 1 -- 172.21.15.133:0/2205556169 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd0dc06b4d0 msgr2=0x7fd0dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:27.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.659+0000 7fd104400640 1 --2- 172.21.15.133:0/2205556169 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd0dc06b4d0 0x7fd0dc06d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec006fd0 tx=0x7fd0ec008040 comp rx=0 tx=0).stop 2024-03-07T06:52:27.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.659+0000 7fd104400640 1 -- 172.21.15.133:0/2205556169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0fc09c5b0 msgr2=0x7fd0fc09d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:27.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.659+0000 7fd104400640 1 --2- 172.21.15.133:0/2205556169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0fc09c5b0 0x7fd0fc09d110 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40090b0 tx=0x7fd0f40046c0 comp rx=0 tx=0).stop 2024-03-07T06:52:27.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.659+0000 7fd104400640 1 -- 172.21.15.133:0/2205556169 shutdown_connections 2024-03-07T06:52:27.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.659+0000 7fd104400640 1 --2- 172.21.15.133:0/2205556169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0fc09c5b0 0x7fd0fc09d110 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:27.659 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.659+0000 7fd104400640 1 --2- 172.21.15.133:0/2205556169 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd0dc06b4d0 0x7fd0dc06d990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:27.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.659+0000 7fd104400640 1 --2- 172.21.15.133:0/2205556169 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0fc15ff90 0x7fd0fc15eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:27.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.659+0000 7fd104400640 1 --2- 172.21.15.133:0/2205556169 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0fc15fbb0 0x7fd0fc152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:27.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.659+0000 7fd104400640 1 -- 172.21.15.133:0/2205556169 >> 172.21.15.133:0/2205556169 conn(0x7fd0fc0b3120 msgr2=0x7fd0fc0acd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:27.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.660+0000 7fd104400640 1 -- 172.21.15.133:0/2205556169 shutdown_connections 2024-03-07T06:52:27.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.660+0000 7fd104400640 1 -- 172.21.15.133:0/2205556169 wait complete. 2024-03-07T06:52:27.681 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-07T06:52:27.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.891+0000 7fb6eba00640 1 Processor -- start 2024-03-07T06:52:27.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.891+0000 7fb6eba00640 1 -- start start 2024-03-07T06:52:27.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.893+0000 7fb6eba00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e40ce5f0 0x7fb6e40b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.893+0000 7fb6eba00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb6e40af950 0x7fb6e4071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.893+0000 7fb6eba00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e406f140 con 0x7fb6e415b1b0 2024-03-07T06:52:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.893+0000 7fb6eba00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e4071610 con 0x7fb6e40af950 2024-03-07T06:52:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.893+0000 7fb6eba00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e406f420 con 0x7fb6e40ce5f0 2024-03-07T06:52:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.893+0000 7fb6e9a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb6e40af950 0x7fb6e4071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.893+0000 7fb6e9a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb6e40af950 0x7fb6e4071d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36568/0 (socket says 172.21.15.133:36568) 2024-03-07T06:52:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.893+0000 7fb6e9a00640 1 -- 172.21.15.133:0/2243233249 learned_addr learned my addr 172.21.15.133:0/2243233249 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:27.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.893+0000 7fb6e3e00640 1 --2- 172.21.15.133:0/2243233249 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e40ce5f0 0x7fb6e40b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:27.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.894+0000 7fb6e9a00640 1 -- 172.21.15.133:0/2243233249 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e40ce5f0 msgr2=0x7fb6e40b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:27.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.894+0000 7fb6e9a00640 1 --2- 172.21.15.133:0/2243233249 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e40ce5f0 0x7fb6e40b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:27.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.894+0000 7fb6e9a00640 1 -- 172.21.15.133:0/2243233249 >> v1:172.21.15.161:6789/0 conn(0x7fb6e415b1b0 legacy=0x7fb6e40cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:27.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.894+0000 7fb6e9a00640 1 -- 172.21.15.133:0/2243233249 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e4071860 con 0x7fb6e40af950 2024-03-07T06:52:27.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.894+0000 7fb6e3e00640 1 --2- 172.21.15.133:0/2243233249 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e40ce5f0 0x7fb6e40b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:27.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.895+0000 7fb6e9a00640 1 --2- 172.21.15.133:0/2243233249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e40af950 0x7fb6e4071d80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4071790 tx=0x7fb6dc02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b449a1a5cf002 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:27.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.895+0000 7fb6e3400640 1 -- 172.21.15.133:0/2243233249 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc02c930 con 0x7fb6e40af950 2024-03-07T06:52:27.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.895+0000 7fb6e3400640 1 -- 172.21.15.133:0/2243233249 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc02cad0 con 0x7fb6e40af950 2024-03-07T06:52:27.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.895+0000 7fb6e3400640 1 -- 172.21.15.133:0/2243233249 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc0343f0 con 0x7fb6e40af950 2024-03-07T06:52:27.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.896+0000 7fb6eba00640 1 -- 172.21.15.133:0/2243233249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e40af950 msgr2=0x7fb6e4071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:27.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.896+0000 7fb6eba00640 1 --2- 172.21.15.133:0/2243233249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e40af950 0x7fb6e4071d80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4071790 tx=0x7fb6dc02bd90 comp rx=0 tx=0).stop 2024-03-07T06:52:27.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.896+0000 7fb6eba00640 1 -- 172.21.15.133:0/2243233249 shutdown_connections 2024-03-07T06:52:27.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.896+0000 7fb6eba00640 1 --2- 172.21.15.133:0/2243233249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e40af950 0x7fb6e4071d80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:27.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.896+0000 7fb6eba00640 1 --2- 172.21.15.133:0/2243233249 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e40ce5f0 0x7fb6e40b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:27.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.896+0000 7fb6eba00640 1 -- 172.21.15.133:0/2243233249 >> 172.21.15.133:0/2243233249 conn(0x7fb6e40b3120 msgr2=0x7fb6e40b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:27.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.896+0000 7fb6eba00640 1 -- 172.21.15.133:0/2243233249 shutdown_connections 2024-03-07T06:52:27.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.896+0000 7fb6eba00640 1 -- 172.21.15.133:0/2243233249 wait complete. 2024-03-07T06:52:27.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.898+0000 7fb6eba00640 1 Processor -- start 2024-03-07T06:52:27.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.898+0000 7fb6eba00640 1 -- start start 2024-03-07T06:52:27.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.899+0000 7fb6eba00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e40af950 0x7fb6e41388d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:27.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.899+0000 7fb6eba00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e40ce5f0 0x7fb6e414ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:27.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.899+0000 7fb6eba00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e415b1b0 0x7fb6e4072740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:27.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.899+0000 7fb6eba00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e415b590 con 0x7fb6e40af950 2024-03-07T06:52:27.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.899+0000 7fb6eba00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e406ee30 con 0x7fb6e40ce5f0 2024-03-07T06:52:27.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.899+0000 7fb6eba00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e415b710 con 0x7fb6e415b1b0 2024-03-07T06:52:27.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.899+0000 7fb6e9000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e40af950 0x7fb6e41388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:27.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.899+0000 7fb6e9000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e40af950 0x7fb6e41388d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36576/0 (socket says 172.21.15.133:36576) 2024-03-07T06:52:27.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.899+0000 7fb6e9000640 1 -- 172.21.15.133:0/1293730552 learned_addr learned my addr 172.21.15.133:0/1293730552 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:27.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.900+0000 7fb6e9000640 1 -- 172.21.15.133:0/1293730552 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e415b1b0 msgr2=0x7fb6e4072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:27.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.900+0000 7fb6e9000640 1 --2- 172.21.15.133:0/1293730552 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e415b1b0 0x7fb6e4072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:27.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.900+0000 7fb6e9000640 1 -- 172.21.15.133:0/1293730552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e40ce5f0 msgr2=0x7fb6e414ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:27.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.900+0000 7fb6e9000640 1 --2- 172.21.15.133:0/1293730552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e40ce5f0 0x7fb6e414ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:27.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.900+0000 7fb6e9000640 1 -- 172.21.15.133:0/1293730552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e41566e0 con 0x7fb6e40af950 2024-03-07T06:52:27.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.900+0000 7fb6e9000640 1 --2- 172.21.15.133:0/1293730552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e40af950 0x7fb6e41388d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800b3f0 tx=0x7fb6d800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.900+0000 7fb6e0c00640 1 -- 172.21.15.133:0/1293730552 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8015020 con 0x7fb6e40af950 2024-03-07T06:52:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.900+0000 7fb6e0c00640 1 -- 172.21.15.133:0/1293730552 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d8011830 con 0x7fb6e40af950 2024-03-07T06:52:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.900+0000 7fb6e0c00640 1 -- 172.21.15.133:0/1293730552 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8011c20 con 0x7fb6e40af950 2024-03-07T06:52:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.900+0000 7fb6eba00640 1 -- 172.21.15.133:0/1293730552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e41569c0 con 0x7fb6e40af950 2024-03-07T06:52:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.900+0000 7fb6eba00640 1 -- 172.21.15.133:0/1293730552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e4156f80 con 0x7fb6e40af950 2024-03-07T06:52:27.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.902+0000 7fb6eba00640 1 -- 172.21.15.133:0/1293730552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e4062c90 con 0x7fb6e40af950 2024-03-07T06:52:27.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.902+0000 7fb6e0c00640 1 -- 172.21.15.133:0/1293730552 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb6d8018020 con 0x7fb6e40af950 2024-03-07T06:52:27.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.904+0000 7fb6e0c00640 1 --2- 172.21.15.133:0/1293730552 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb6c406b4d0 0x7fb6c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:27.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.904+0000 7fb6e0c00640 1 -- 172.21.15.133:0/1293730552 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb6d8089ab0 con 0x7fb6e40af950 2024-03-07T06:52:27.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.904+0000 7fb6e3e00640 1 --2- 172.21.15.133:0/1293730552 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb6c406b4d0 0x7fb6c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:27.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.906+0000 7fb6e3e00640 1 --2- 172.21.15.133:0/1293730552 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb6c406b4d0 0x7fb6c406d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0006fd0 tx=0x7fb6d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:27.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:27.908+0000 7fb6e0c00640 1 -- 172.21.15.133:0/1293730552 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb6d801f030 con 0x7fb6e40af950 2024-03-07T06:52:28.167 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.168+0000 7fb6eba00640 1 -- 172.21.15.133:0/1293730552 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb6e40cc660 con 0x7fb6e40af950 2024-03-07T06:52:28.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.169+0000 7fb6e0c00640 1 -- 172.21.15.133:0/1293730552 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fb6d80172c0 con 0x7fb6e40af950 2024-03-07T06:52:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:28.169 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:52:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-03-07T06:52:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-03-07T06:52:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCaY+llrqSIMBAAy1d9K8CCdERld7xCmKOutQ==", 2024-03-07T06:52:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:52:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-03-07T06:52:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-03-07T06:52:28.169 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCbY+llx7hIFxAAaEpBpdI7eaGAU9GuxPPfhg==", 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-03-07T06:52:28.170 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCbY+ll0XgoBhAAXJiu001WO6mHv/UaVuHoJA==", 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCbY+llrDLKJxAAASZPgaXrsJYHB3pzksPcSA==", 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-03-07T06:52:28.171 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCgY+llgbaKFxAAUNMCRhWDof+qhuedrCY57Q==", 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCjY+llQx/1IRAA5yMfKhH4CxK5751ymJYSzQ==", 2024-03-07T06:52:28.172 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCmY+lluw4uOhAAvy/SoCA7TWJGid53bj1fhA==", 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.173 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-03-07T06:52:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCqY+llf5L0FRAA/WlR9r+5bSmu+Y4ggoobdw==", 2024-03-07T06:52:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:52:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:52:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-03-07T06:52:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCxY+llYmATHxAA+eGqKm4rlXZtcYgV3bZDDg==", 2024-03-07T06:52:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:52:28.174 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQC0Y+llQJ+EOhAAs7gC9YWEap3xhX/Qiyt0lg==", 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQC4Y+llpADECBAA42RfVF05S45UiaHJyPCRzA==", 2024-03-07T06:52:28.175 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQC7Y+llC4N/IRAAcnaB9/yZj2fz9nL6hS5mSA==", 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.176 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCbY+ll6xWENRAArKbw8ID7PLMFVCO3UlKDOg==", 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCcY+llCvgxEBAA+PC1o+JD9S0G7IUJVLVh+g==", 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-03-07T06:52:28.177 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCcY+llvYWxBBAA32ACXxhIhSDI+wI6zZiafA==", 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.178 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCcY+llwgZYGxAAtbCo73WlwL2ftJ4yyFEzXA==", 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCZY+lluFVeJBAA30ba1sDHilXgfvKFTiofwg==", 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-03-07T06:52:28.179 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.bootstrap-mds", 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDJY+ll9JJbExAAiAIIh9PJhrQPh1uc+UH8wQ==", 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile bootstrap-mds" 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.bootstrap-mgr", 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDJY+lll7FbExAADp9S4whxCk711LN0Da45sg==", 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-07T06:52:28.180 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.181 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.181 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.181 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.bootstrap-osd", 2024-03-07T06:52:28.181 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDJY+llW8lbExAAUeEd4qXuOvy4d/ticNS0yA==", 2024-03-07T06:52:28.181 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.181 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile bootstrap-osd" 2024-03-07T06:52:28.181 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.181 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.181 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.181 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.bootstrap-rbd", 2024-03-07T06:52:28.181 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDJY+llJeFbExAAapopyGpvy8vfdVfSyy9MHA==", 2024-03-07T06:52:28.181 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.181 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-07T06:52:28.181 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDJY+llpvlbExAAH2NyArFDab4iX55R9Z5qeQ==", 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.bootstrap-rgw", 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQDJY+llnxJcExAAtPYdRGCiPW3/95NztMMgaA==", 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.182 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCaY+llqK68JRAAuPudks7z1/uL593N8/VyGA==", 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCaY+llosjXGBAAjyuWL3GnYWP6nHe9jnyUNg==", 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:52:28.183 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.184 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:28.184 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-03-07T06:52:28.184 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:52:28.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.171+0000 7fb6eba00640 1 -- 172.21.15.133:0/1293730552 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb6c406b4d0 msgr2=0x7fb6c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:28.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.171+0000 7fb6eba00640 1 --2- 172.21.15.133:0/1293730552 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb6c406b4d0 0x7fb6c406d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0006fd0 tx=0x7fb6d0008040 comp rx=0 tx=0).stop 2024-03-07T06:52:28.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.171+0000 7fb6eba00640 1 -- 172.21.15.133:0/1293730552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e40af950 msgr2=0x7fb6e41388d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:28.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.171+0000 7fb6eba00640 1 --2- 172.21.15.133:0/1293730552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e40af950 0x7fb6e41388d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800b3f0 tx=0x7fb6d800b8b0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.172+0000 7fb6eba00640 1 -- 172.21.15.133:0/1293730552 shutdown_connections 2024-03-07T06:52:28.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.172+0000 7fb6eba00640 1 --2- 172.21.15.133:0/1293730552 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6e415b1b0 0x7fb6e4072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.172+0000 7fb6eba00640 1 --2- 172.21.15.133:0/1293730552 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb6c406b4d0 0x7fb6c406d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.172+0000 7fb6eba00640 1 --2- 172.21.15.133:0/1293730552 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6e40ce5f0 0x7fb6e414ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.172+0000 7fb6eba00640 1 --2- 172.21.15.133:0/1293730552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6e40af950 0x7fb6e41388d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.172+0000 7fb6eba00640 1 -- 172.21.15.133:0/1293730552 >> 172.21.15.133:0/1293730552 conn(0x7fb6e40b3120 msgr2=0x7fb6e40b5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:28.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.172+0000 7fb6eba00640 1 -- 172.21.15.133:0/1293730552 shutdown_connections 2024-03-07T06:52:28.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.172+0000 7fb6eba00640 1 -- 172.21.15.133:0/1293730552 wait complete. 2024-03-07T06:52:28.191 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-07T06:52:28.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.399+0000 7f83ff200640 1 Processor -- start 2024-03-07T06:52:28.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.399+0000 7f83ff200640 1 -- start start 2024-03-07T06:52:28.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.400+0000 7f83ff200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f83f80ce5f0 0x7f83f80b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:28.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.400+0000 7f83ff200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83f80af950 0x7f83f8071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:28.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.400+0000 7f83ff200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f806f140 con 0x7f83f815b1b0 2024-03-07T06:52:28.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.400+0000 7f83ff200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f8071610 con 0x7f83f80ce5f0 2024-03-07T06:52:28.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.400+0000 7f83ff200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f806f420 con 0x7f83f80af950 2024-03-07T06:52:28.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.401+0000 7f83efe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f83f80ce5f0 0x7f83f80b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:28.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.401+0000 7f83efe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f83f80ce5f0 0x7f83f80b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36578/0 (socket says 172.21.15.133:36578) 2024-03-07T06:52:28.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.401+0000 7f83efe00640 1 -- 172.21.15.133:0/4098675391 learned_addr learned my addr 172.21.15.133:0/4098675391 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:28.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.401+0000 7f83fd200640 1 --2- 172.21.15.133:0/4098675391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83f80af950 0x7f83f8071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:28.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.401+0000 7f83efe00640 1 -- 172.21.15.133:0/4098675391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83f80af950 msgr2=0x7f83f8071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:28.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.401+0000 7f83efe00640 1 --2- 172.21.15.133:0/4098675391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83f80af950 0x7f83f8071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.401+0000 7f83efe00640 1 -- 172.21.15.133:0/4098675391 >> v1:172.21.15.161:6789/0 conn(0x7f83f815b1b0 legacy=0x7f83f80cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:28.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.401+0000 7f83efe00640 1 -- 172.21.15.133:0/4098675391 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f8071860 con 0x7f83f80ce5f0 2024-03-07T06:52:28.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.402+0000 7f83efe00640 1 --2- 172.21.15.133:0/4098675391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80ce5f0 0x7f83f80b3c10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009870 tx=0x7f83e002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c67c469af84a88 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:28.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.403+0000 7f83f7400640 1 -- 172.21.15.133:0/4098675391 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0035020 con 0x7f83f80ce5f0 2024-03-07T06:52:28.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.403+0000 7f83f7400640 1 -- 172.21.15.133:0/4098675391 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e0031640 con 0x7f83f80ce5f0 2024-03-07T06:52:28.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.403+0000 7f83f7400640 1 -- 172.21.15.133:0/4098675391 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e002d040 con 0x7f83f80ce5f0 2024-03-07T06:52:28.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.403+0000 7f83ff200640 1 -- 172.21.15.133:0/4098675391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80ce5f0 msgr2=0x7f83f80b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:28.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.403+0000 7f83ff200640 1 --2- 172.21.15.133:0/4098675391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80ce5f0 0x7f83f80b3c10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009870 tx=0x7f83e002b8f0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.403+0000 7f83ff200640 1 -- 172.21.15.133:0/4098675391 shutdown_connections 2024-03-07T06:52:28.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.403+0000 7f83ff200640 1 --2- 172.21.15.133:0/4098675391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83f80af950 0x7f83f8071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.403+0000 7f83ff200640 1 --2- 172.21.15.133:0/4098675391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80ce5f0 0x7f83f80b3c10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.403+0000 7f83ff200640 1 -- 172.21.15.133:0/4098675391 >> 172.21.15.133:0/4098675391 conn(0x7f83f80b3120 msgr2=0x7f83f80b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:28.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.403+0000 7f83ff200640 1 -- 172.21.15.133:0/4098675391 shutdown_connections 2024-03-07T06:52:28.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.403+0000 7f83ff200640 1 -- 172.21.15.133:0/4098675391 wait complete. 2024-03-07T06:52:28.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.405+0000 7f83ff200640 1 Processor -- start 2024-03-07T06:52:28.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.405+0000 7f83ff200640 1 -- start start 2024-03-07T06:52:28.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.406+0000 7f83ff200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80af950 0x7f83f8159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:28.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.406+0000 7f83ff200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f80ce5f0 0x7f83f809a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:28.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.406+0000 7f83ff200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83f815b1b0 0x7f83f815d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:28.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.406+0000 7f83ff200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f815b590 con 0x7f83f80af950 2024-03-07T06:52:28.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.406+0000 7f83ff200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f806ee30 con 0x7f83f80ce5f0 2024-03-07T06:52:28.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.406+0000 7f83ff200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f815b710 con 0x7f83f815b1b0 2024-03-07T06:52:28.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.407+0000 7f83f7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80af950 0x7f83f8159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:28.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.407+0000 7f83f7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80af950 0x7f83f8159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36586/0 (socket says 172.21.15.133:36586) 2024-03-07T06:52:28.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.407+0000 7f83f7e00640 1 -- 172.21.15.133:0/215116743 learned_addr learned my addr 172.21.15.133:0/215116743 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:28.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.407+0000 7f83fd200640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83f815b1b0 0x7f83f815d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:28.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.407+0000 7f83efe00640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f80ce5f0 0x7f83f809a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:28.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.407+0000 7f83f7e00640 1 -- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83f815b1b0 msgr2=0x7f83f815d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:28.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.407+0000 7f83f7e00640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83f815b1b0 0x7f83f815d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.407+0000 7f83f7e00640 1 -- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f80ce5f0 msgr2=0x7f83f809a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:28.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.407+0000 7f83f7e00640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f80ce5f0 0x7f83f809a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.407+0000 7f83f7e00640 1 -- 172.21.15.133:0/215116743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f815c2c0 con 0x7f83f80af950 2024-03-07T06:52:28.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.407+0000 7f83efe00640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f80ce5f0 0x7f83f809a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:52:28.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.407+0000 7f83f7e00640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80af950 0x7f83f8159fc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f83e800c3c0 tx=0x7f83e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:28.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.408+0000 7f83f4c00640 1 -- 172.21.15.133:0/215116743 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80052d0 con 0x7f83f80af950 2024-03-07T06:52:28.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.408+0000 7f83f4c00640 1 -- 172.21.15.133:0/215116743 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e8005490 con 0x7f83f80af950 2024-03-07T06:52:28.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.408+0000 7f83f4c00640 1 -- 172.21.15.133:0/215116743 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8007260 con 0x7f83f80af950 2024-03-07T06:52:28.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.408+0000 7f83ff200640 1 -- 172.21.15.133:0/215116743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f815c5a0 con 0x7f83f80af950 2024-03-07T06:52:28.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.408+0000 7f83ff200640 1 -- 172.21.15.133:0/215116743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f815cb60 con 0x7f83f80af950 2024-03-07T06:52:28.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.409+0000 7f83ff200640 1 -- 172.21.15.133:0/215116743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f8062c90 con 0x7f83f80af950 2024-03-07T06:52:28.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.410+0000 7f83f4c00640 1 -- 172.21.15.133:0/215116743 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f83e8007400 con 0x7f83f80af950 2024-03-07T06:52:28.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.411+0000 7f83f4c00640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f83d406b4d0 0x7f83d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:28.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.411+0000 7f83f4c00640 1 -- 172.21.15.133:0/215116743 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f83e80893a0 con 0x7f83f80af950 2024-03-07T06:52:28.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.411+0000 7f83efe00640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f83d406b4d0 0x7f83d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:28.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.412+0000 7f83efe00640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f83d406b4d0 0x7f83d406d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f83e0036000 tx=0x7f83e003c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:28.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.416+0000 7f83f4c00640 1 -- 172.21.15.133:0/215116743 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f83e8058560 con 0x7f83f80af950 2024-03-07T06:52:28.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.687+0000 7f83ff200640 1 -- 172.21.15.133:0/215116743 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f83f80cc660 con 0x7f83f80af950 2024-03-07T06:52:28.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.691+0000 7f83f4c00640 1 -- 172.21.15.133:0/215116743 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f83e8057e60 con 0x7f83f80af950 2024-03-07T06:52:28.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.694+0000 7f83ff200640 1 -- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f83d406b4d0 msgr2=0x7f83d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:28.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.694+0000 7f83ff200640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f83d406b4d0 0x7f83d406d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f83e0036000 tx=0x7f83e003c040 comp rx=0 tx=0).stop 2024-03-07T06:52:28.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.694+0000 7f83ff200640 1 -- 172.21.15.133:0/215116743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80af950 msgr2=0x7f83f8159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:28.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.694+0000 7f83ff200640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80af950 0x7f83f8159fc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f83e800c3c0 tx=0x7f83e800c880 comp rx=0 tx=0).stop 2024-03-07T06:52:28.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.695+0000 7f83ff200640 1 -- 172.21.15.133:0/215116743 shutdown_connections 2024-03-07T06:52:28.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.695+0000 7f83ff200640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83f815b1b0 0x7f83f815d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.695+0000 7f83ff200640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f83d406b4d0 0x7f83d406d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.695+0000 7f83ff200640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83f80ce5f0 0x7f83f809a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.695+0000 7f83ff200640 1 --2- 172.21.15.133:0/215116743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83f80af950 0x7f83f8159fc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.695+0000 7f83ff200640 1 -- 172.21.15.133:0/215116743 >> 172.21.15.133:0/215116743 conn(0x7f83f80b3120 msgr2=0x7f83f80b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:28.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.695+0000 7f83ff200640 1 -- 172.21.15.133:0/215116743 shutdown_connections 2024-03-07T06:52:28.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.695+0000 7f83ff200640 1 -- 172.21.15.133:0/215116743 wait complete. 2024-03-07T06:52:28.713 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-07T06:52:28.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.922+0000 7fd127e00640 1 Processor -- start 2024-03-07T06:52:28.922 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.922+0000 7fd127e00640 1 -- start start 2024-03-07T06:52:28.924 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.924+0000 7fd127e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd12015b1b0 0x7fd1200b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:28.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.924+0000 7fd127e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1200ce5f0 0x7fd120071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:28.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.924+0000 7fd127e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12006f140 con 0x7fd1200af950 2024-03-07T06:52:28.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.924+0000 7fd127e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd120071610 con 0x7fd1200ce5f0 2024-03-07T06:52:28.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.924+0000 7fd127e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12006f420 con 0x7fd12015b1b0 2024-03-07T06:52:28.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.924+0000 7fd125e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1200ce5f0 0x7fd120071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:28.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.924+0000 7fd125e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd1200ce5f0 0x7fd120071d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36592/0 (socket says 172.21.15.133:36592) 2024-03-07T06:52:28.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.924+0000 7fd125e00640 1 -- 172.21.15.133:0/2203886025 learned_addr learned my addr 172.21.15.133:0/2203886025 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:28.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.925+0000 7fd125400640 1 --2- 172.21.15.133:0/2203886025 >> v2:172.21.15.161:3300/0 conn(0x7fd12015b1b0 0x7fd1200b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:28.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.925+0000 7fd125e00640 1 -- 172.21.15.133:0/2203886025 >> v2:172.21.15.161:3300/0 conn(0x7fd12015b1b0 msgr2=0x7fd1200b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:28.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.925+0000 7fd125e00640 1 --2- 172.21.15.133:0/2203886025 >> v2:172.21.15.161:3300/0 conn(0x7fd12015b1b0 0x7fd1200b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.925+0000 7fd125e00640 1 -- 172.21.15.133:0/2203886025 >> v1:172.21.15.133:6789/0 conn(0x7fd1200af950 legacy=0x7fd1200cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:28.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.925+0000 7fd125e00640 1 -- 172.21.15.133:0/2203886025 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd120071860 con 0x7fd1200ce5f0 2024-03-07T06:52:28.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.925+0000 7fd125400640 1 --2- 172.21.15.133:0/2203886025 >> v2:172.21.15.161:3300/0 conn(0x7fd12015b1b0 0x7fd1200b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:28.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.926+0000 7fd125e00640 1 --2- 172.21.15.133:0/2203886025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1200ce5f0 0x7fd120071d80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd1200700d0 tx=0x7fd11802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba589c0135a72e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:28.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.926+0000 7fd11fe00640 1 -- 172.21.15.133:0/2203886025 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118035020 con 0x7fd1200ce5f0 2024-03-07T06:52:28.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.926+0000 7fd11fe00640 1 -- 172.21.15.133:0/2203886025 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd118031870 con 0x7fd1200ce5f0 2024-03-07T06:52:28.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.926+0000 7fd11fe00640 1 -- 172.21.15.133:0/2203886025 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118031c40 con 0x7fd1200ce5f0 2024-03-07T06:52:28.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.927+0000 7fd127e00640 1 -- 172.21.15.133:0/2203886025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1200ce5f0 msgr2=0x7fd120071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:28.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.927+0000 7fd127e00640 1 --2- 172.21.15.133:0/2203886025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1200ce5f0 0x7fd120071d80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd1200700d0 tx=0x7fd11802b910 comp rx=0 tx=0).stop 2024-03-07T06:52:28.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.927+0000 7fd127e00640 1 -- 172.21.15.133:0/2203886025 shutdown_connections 2024-03-07T06:52:28.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.927+0000 7fd127e00640 1 --2- 172.21.15.133:0/2203886025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1200ce5f0 0x7fd120071d80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.927+0000 7fd127e00640 1 --2- 172.21.15.133:0/2203886025 >> v2:172.21.15.161:3300/0 conn(0x7fd12015b1b0 0x7fd1200b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.927+0000 7fd127e00640 1 -- 172.21.15.133:0/2203886025 >> 172.21.15.133:0/2203886025 conn(0x7fd1200b3120 msgr2=0x7fd1200b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:28.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.927+0000 7fd127e00640 1 -- 172.21.15.133:0/2203886025 shutdown_connections 2024-03-07T06:52:28.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.927+0000 7fd127e00640 1 -- 172.21.15.133:0/2203886025 wait complete. 2024-03-07T06:52:28.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.929+0000 7fd127e00640 1 Processor -- start 2024-03-07T06:52:28.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.929+0000 7fd127e00640 1 -- start start 2024-03-07T06:52:28.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.930+0000 7fd127e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1200af950 0x7fd12009a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:28.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.930+0000 7fd127e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1200ce5f0 0x7fd1200b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:28.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.930+0000 7fd127e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd12015b1b0 0x7fd120157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:28.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.930+0000 7fd127e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12015b590 con 0x7fd12015b1b0 2024-03-07T06:52:28.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.930+0000 7fd127e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12006ee30 con 0x7fd1200ce5f0 2024-03-07T06:52:28.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.930+0000 7fd127e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12015b710 con 0x7fd1200af950 2024-03-07T06:52:28.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.930+0000 7fd125e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd12015b1b0 0x7fd120157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:28.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.930+0000 7fd125e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd12015b1b0 0x7fd120157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36596/0 (socket says 172.21.15.133:36596) 2024-03-07T06:52:28.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.930+0000 7fd125e00640 1 -- 172.21.15.133:0/1052364506 learned_addr learned my addr 172.21.15.133:0/1052364506 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:28.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.930+0000 7fd125400640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1200af950 0x7fd12009a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:28.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.931+0000 7fd125e00640 1 -- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1200af950 msgr2=0x7fd12009a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:28.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.931+0000 7fd125e00640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1200af950 0x7fd12009a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.931+0000 7fd125e00640 1 -- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1200ce5f0 msgr2=0x7fd1200b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:28.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.931+0000 7fd124a00640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1200ce5f0 0x7fd1200b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:28.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.931+0000 7fd125e00640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1200ce5f0 0x7fd1200b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:28.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.931+0000 7fd125e00640 1 -- 172.21.15.133:0/1052364506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd120156340 con 0x7fd12015b1b0 2024-03-07T06:52:28.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.931+0000 7fd124a00640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1200ce5f0 0x7fd1200b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:28.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.931+0000 7fd125e00640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd12015b1b0 0x7fd120157b20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd11802b950 tx=0x7fd1180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:28.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.931+0000 7fd11d600640 1 -- 172.21.15.133:0/1052364506 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118040070 con 0x7fd12015b1b0 2024-03-07T06:52:28.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.931+0000 7fd11d600640 1 -- 172.21.15.133:0/1052364506 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11803a420 con 0x7fd12015b1b0 2024-03-07T06:52:28.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.931+0000 7fd127e00640 1 -- 172.21.15.133:0/1052364506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd120156620 con 0x7fd12015b1b0 2024-03-07T06:52:28.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.931+0000 7fd11d600640 1 -- 172.21.15.133:0/1052364506 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118035040 con 0x7fd12015b1b0 2024-03-07T06:52:28.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.931+0000 7fd127e00640 1 -- 172.21.15.133:0/1052364506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd120156be0 con 0x7fd12015b1b0 2024-03-07T06:52:28.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.932+0000 7fd127e00640 1 -- 172.21.15.133:0/1052364506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd120062c90 con 0x7fd12015b1b0 2024-03-07T06:52:28.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.933+0000 7fd11d600640 1 -- 172.21.15.133:0/1052364506 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd118038020 con 0x7fd12015b1b0 2024-03-07T06:52:28.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.934+0000 7fd11d600640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd10406b4d0 0x7fd10406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:28.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.934+0000 7fd11d600640 1 -- 172.21.15.133:0/1052364506 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd1180abf60 con 0x7fd12015b1b0 2024-03-07T06:52:28.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.935+0000 7fd125400640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd10406b4d0 0x7fd10406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:28.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.936+0000 7fd125400640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd10406b4d0 0x7fd10406d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd11400b840 tx=0x7fd114008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:28.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:28.938+0000 7fd11d600640 1 -- 172.21.15.133:0/1052364506 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd11807b090 con 0x7fd12015b1b0 2024-03-07T06:52:29.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.182+0000 7fd127e00640 1 -- 172.21.15.133:0/1052364506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fd1200cc660 con 0x7fd12015b1b0 2024-03-07T06:52:29.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.186+0000 7fd11d600640 1 -- 172.21.15.133:0/1052364506 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd11807a990 con 0x7fd12015b1b0 2024-03-07T06:52:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.189+0000 7fd127e00640 1 -- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd10406b4d0 msgr2=0x7fd10406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.189+0000 7fd127e00640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd10406b4d0 0x7fd10406d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd11400b840 tx=0x7fd114008040 comp rx=0 tx=0).stop 2024-03-07T06:52:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.189+0000 7fd127e00640 1 -- 172.21.15.133:0/1052364506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd12015b1b0 msgr2=0x7fd120157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.189+0000 7fd127e00640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd12015b1b0 0x7fd120157b20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd11802b950 tx=0x7fd1180046c0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.189+0000 7fd127e00640 1 -- 172.21.15.133:0/1052364506 shutdown_connections 2024-03-07T06:52:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.189+0000 7fd127e00640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd12015b1b0 0x7fd120157b20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.189+0000 7fd127e00640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd10406b4d0 0x7fd10406d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.189+0000 7fd127e00640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd1200ce5f0 0x7fd1200b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.189+0000 7fd127e00640 1 --2- 172.21.15.133:0/1052364506 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd1200af950 0x7fd12009a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.189+0000 7fd127e00640 1 -- 172.21.15.133:0/1052364506 >> 172.21.15.133:0/1052364506 conn(0x7fd1200b3120 msgr2=0x7fd12015a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:29.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.189+0000 7fd127e00640 1 -- 172.21.15.133:0/1052364506 shutdown_connections 2024-03-07T06:52:29.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.189+0000 7fd127e00640 1 -- 172.21.15.133:0/1052364506 wait complete. 2024-03-07T06:52:29.205 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-07T06:52:29.337 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:52:29.345 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:52:29.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.411+0000 7fbd3b800640 1 Processor -- start 2024-03-07T06:52:29.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.411+0000 7fbd3b800640 1 -- start start 2024-03-07T06:52:29.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.412+0000 7fbd3b800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbd34140ec0 0x7fbd341416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.412+0000 7fbd3b800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbd34161c90 0x7fbd34162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.412+0000 7fbd3b800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd3415e6a0 0x7fbd3415eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.412+0000 7fbd3b800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd3415f420 con 0x7fbd34161c90 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.412+0000 7fbd3b800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd340b4590 con 0x7fbd34140ec0 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.412+0000 7fbd3b800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340b4870 con 0x7fbd3415e6a0 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.412+0000 7fbd33e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbd34161c90 0x7fbd34162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.412+0000 7fbd33e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbd34161c90 0x7fbd34162110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36608/0 (socket says 172.21.15.133:36608) 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.412+0000 7fbd33e00640 1 -- 172.21.15.133:0/429938533 learned_addr learned my addr 172.21.15.133:0/429938533 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.412+0000 7fbd39800640 1 --2- 172.21.15.133:0/429938533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd3415e6a0 0x7fbd3415eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.412+0000 7fbd38e00640 1 --2- 172.21.15.133:0/429938533 >> v2:172.21.15.161:3300/0 conn(0x7fbd34140ec0 0x7fbd341416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.413+0000 7fbd33e00640 1 -- 172.21.15.133:0/429938533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd3415e6a0 msgr2=0x7fbd3415eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.413+0000 7fbd33e00640 1 --2- 172.21.15.133:0/429938533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd3415e6a0 0x7fbd3415eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.413+0000 7fbd33e00640 1 -- 172.21.15.133:0/429938533 >> v2:172.21.15.161:3300/0 conn(0x7fbd34140ec0 msgr2=0x7fbd341416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.413+0000 7fbd33e00640 1 --2- 172.21.15.133:0/429938533 >> v2:172.21.15.161:3300/0 conn(0x7fbd34140ec0 0x7fbd341416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.413+0000 7fbd33e00640 1 -- 172.21.15.133:0/429938533 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd340b4b50 con 0x7fbd34161c90 2024-03-07T06:52:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.414+0000 7fbd33e00640 1 --2- 172.21.15.133:0/429938533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd34161c90 0x7fbd34162110 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbd340d40b0 tx=0x7fbd2002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2ba2b398e1a02e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.414+0000 7fbd33400640 1 -- 172.21.15.133:0/429938533 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20035020 con 0x7fbd34161c90 2024-03-07T06:52:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.414+0000 7fbd33400640 1 -- 172.21.15.133:0/429938533 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd20031870 con 0x7fbd34161c90 2024-03-07T06:52:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.414+0000 7fbd33400640 1 -- 172.21.15.133:0/429938533 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20031c40 con 0x7fbd34161c90 2024-03-07T06:52:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.415+0000 7fbd3b800640 1 -- 172.21.15.133:0/429938533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd34161c90 msgr2=0x7fbd34162110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.415+0000 7fbd3b800640 1 --2- 172.21.15.133:0/429938533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd34161c90 0x7fbd34162110 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbd340d40b0 tx=0x7fbd2002b910 comp rx=0 tx=0).stop 2024-03-07T06:52:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.415+0000 7fbd3b800640 1 -- 172.21.15.133:0/429938533 shutdown_connections 2024-03-07T06:52:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.415+0000 7fbd3b800640 1 --2- 172.21.15.133:0/429938533 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd3415e6a0 0x7fbd3415eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.415+0000 7fbd3b800640 1 --2- 172.21.15.133:0/429938533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd34161c90 0x7fbd34162110 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.415+0000 7fbd3b800640 1 --2- 172.21.15.133:0/429938533 >> v2:172.21.15.161:3300/0 conn(0x7fbd34140ec0 0x7fbd341416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.415+0000 7fbd3b800640 1 -- 172.21.15.133:0/429938533 >> 172.21.15.133:0/429938533 conn(0x7fbd34071340 msgr2=0x7fbd34071720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.415+0000 7fbd3b800640 1 -- 172.21.15.133:0/429938533 shutdown_connections 2024-03-07T06:52:29.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.415+0000 7fbd3b800640 1 -- 172.21.15.133:0/429938533 wait complete. 2024-03-07T06:52:29.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.417+0000 7fbd3b800640 1 Processor -- start 2024-03-07T06:52:29.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.417+0000 7fbd3b800640 1 -- start start 2024-03-07T06:52:29.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd3b800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd34140ec0 0x7fbd3413f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:29.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd3b800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd3415e6a0 0x7fbd3412f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:29.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd3b800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd34161c90 0x7fbd340d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:29.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd3b800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd34141420 con 0x7fbd34140ec0 2024-03-07T06:52:29.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd3b800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd341412a0 con 0x7fbd3415e6a0 2024-03-07T06:52:29.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd3b800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd340b5070 con 0x7fbd34161c90 2024-03-07T06:52:29.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd38e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd34140ec0 0x7fbd3413f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:29.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd38e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd34140ec0 0x7fbd3413f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36624/0 (socket says 172.21.15.133:36624) 2024-03-07T06:52:29.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd38e00640 1 -- 172.21.15.133:0/2519106597 learned_addr learned my addr 172.21.15.133:0/2519106597 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:29.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd33e00640 1 --2- 172.21.15.133:0/2519106597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd3415e6a0 0x7fbd3412f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:29.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd39800640 1 --2- 172.21.15.133:0/2519106597 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd34161c90 0x7fbd340d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd38e00640 1 -- 172.21.15.133:0/2519106597 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd34161c90 msgr2=0x7fbd340d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd38e00640 1 --2- 172.21.15.133:0/2519106597 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd34161c90 0x7fbd340d0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd38e00640 1 -- 172.21.15.133:0/2519106597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd3415e6a0 msgr2=0x7fbd3412f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd38e00640 1 --2- 172.21.15.133:0/2519106597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd3415e6a0 0x7fbd3412f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.418+0000 7fbd38e00640 1 -- 172.21.15.133:0/2519106597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3412e620 con 0x7fbd34140ec0 2024-03-07T06:52:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.419+0000 7fbd38e00640 1 --2- 172.21.15.133:0/2519106597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd34140ec0 0x7fbd3413f720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800e9b0 tx=0x7fbd2800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.419+0000 7fbd30c00640 1 -- 172.21.15.133:0/2519106597 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28017070 con 0x7fbd34140ec0 2024-03-07T06:52:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.419+0000 7fbd30c00640 1 -- 172.21.15.133:0/2519106597 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2800c9a0 con 0x7fbd34140ec0 2024-03-07T06:52:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.419+0000 7fbd3b800640 1 -- 172.21.15.133:0/2519106597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3412e900 con 0x7fbd34140ec0 2024-03-07T06:52:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.419+0000 7fbd30c00640 1 -- 172.21.15.133:0/2519106597 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2800cd90 con 0x7fbd34140ec0 2024-03-07T06:52:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.419+0000 7fbd3b800640 1 -- 172.21.15.133:0/2519106597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3412eec0 con 0x7fbd34140ec0 2024-03-07T06:52:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.420+0000 7fbd3b800640 1 -- 172.21.15.133:0/2519106597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd34062c90 con 0x7fbd34140ec0 2024-03-07T06:52:29.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.421+0000 7fbd30c00640 1 -- 172.21.15.133:0/2519106597 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbd2801a070 con 0x7fbd34140ec0 2024-03-07T06:52:29.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.422+0000 7fbd30c00640 1 --2- 172.21.15.133:0/2519106597 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbd1406b4d0 0x7fbd1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:29.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.422+0000 7fbd30c00640 1 -- 172.21.15.133:0/2519106597 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbd2808a330 con 0x7fbd34140ec0 2024-03-07T06:52:29.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.423+0000 7fbd33e00640 1 --2- 172.21.15.133:0/2519106597 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbd1406b4d0 0x7fbd1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:29.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.423+0000 7fbd33e00640 1 --2- 172.21.15.133:0/2519106597 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbd1406b4d0 0x7fbd1406d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd20036000 tx=0x7fbd2003c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:29.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.427+0000 7fbd30c00640 1 -- 172.21.15.133:0/2519106597 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbd28059460 con 0x7fbd34140ec0 2024-03-07T06:52:29.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.695+0000 7fbd3b800640 1 -- 172.21.15.133:0/2519106597 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fbd340b6000 con 0x7fbd34140ec0 2024-03-07T06:52:29.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.700+0000 7fbd30c00640 1 -- 172.21.15.133:0/2519106597 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbd28058d60 con 0x7fbd34140ec0 2024-03-07T06:52:29.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.703+0000 7fbd3b800640 1 -- 172.21.15.133:0/2519106597 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbd1406b4d0 msgr2=0x7fbd1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:29.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.703+0000 7fbd3b800640 1 --2- 172.21.15.133:0/2519106597 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbd1406b4d0 0x7fbd1406d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd20036000 tx=0x7fbd2003c040 comp rx=0 tx=0).stop 2024-03-07T06:52:29.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.703+0000 7fbd3b800640 1 -- 172.21.15.133:0/2519106597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd34140ec0 msgr2=0x7fbd3413f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:29.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.703+0000 7fbd3b800640 1 --2- 172.21.15.133:0/2519106597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd34140ec0 0x7fbd3413f720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800e9b0 tx=0x7fbd2800ee70 comp rx=0 tx=0).stop 2024-03-07T06:52:29.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.703+0000 7fbd3b800640 1 -- 172.21.15.133:0/2519106597 shutdown_connections 2024-03-07T06:52:29.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.703+0000 7fbd3b800640 1 --2- 172.21.15.133:0/2519106597 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbd34161c90 0x7fbd340d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.703+0000 7fbd3b800640 1 --2- 172.21.15.133:0/2519106597 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbd1406b4d0 0x7fbd1406d990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.703+0000 7fbd3b800640 1 --2- 172.21.15.133:0/2519106597 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbd3415e6a0 0x7fbd3412f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.703+0000 7fbd3b800640 1 --2- 172.21.15.133:0/2519106597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd34140ec0 0x7fbd3413f720 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.703+0000 7fbd3b800640 1 -- 172.21.15.133:0/2519106597 >> 172.21.15.133:0/2519106597 conn(0x7fbd34071340 msgr2=0x7fbd340b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:29.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.703+0000 7fbd3b800640 1 -- 172.21.15.133:0/2519106597 shutdown_connections 2024-03-07T06:52:29.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.703+0000 7fbd3b800640 1 -- 172.21.15.133:0/2519106597 wait complete. 2024-03-07T06:52:29.721 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-07T06:52:29.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.926+0000 7fe43f200640 1 Processor -- start 2024-03-07T06:52:29.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.926+0000 7fe43f200640 1 -- start start 2024-03-07T06:52:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.928+0000 7fe43f200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe44015eac0 0x7fe44015eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.928+0000 7fe43f200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe440072f40 0x7fe4400ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.928+0000 7fe43f200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44006e3f0 con 0x7fe44006dac0 2024-03-07T06:52:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.928+0000 7fe43f200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe44006e6d0 con 0x7fe44015eac0 2024-03-07T06:52:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.928+0000 7fe43f200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe44015f3e0 con 0x7fe440072f40 2024-03-07T06:52:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.928+0000 7fe437e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe44006dac0 0x7fe440072830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43752/0 (socket says 172.21.15.133:43752) 2024-03-07T06:52:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.928+0000 7fe437e00640 1 -- 172.21.15.133:0/1568876180 learned_addr learned my addr 172.21.15.133:0/1568876180 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.928+0000 7fe43de00640 1 --2- 172.21.15.133:0/1568876180 >> v2:172.21.15.133:3300/0 conn(0x7fe44015eac0 0x7fe44015eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:29.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.929+0000 7fe43e800640 1 --2- 172.21.15.133:0/1568876180 >> v2:172.21.15.161:3300/0 conn(0x7fe440072f40 0x7fe4400ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:29.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.929+0000 7fe43d400640 1 -- 172.21.15.133:0/1568876180 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844560249 0 0) 0x7fe44006e3f0 con 0x7fe44006dac0 2024-03-07T06:52:29.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.929+0000 7fe43d400640 1 -- 172.21.15.133:0/1568876180 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe424003520 con 0x7fe44006dac0 2024-03-07T06:52:29.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.929+0000 7fe43de00640 1 -- 172.21.15.133:0/1568876180 >> v2:172.21.15.161:3300/0 conn(0x7fe440072f40 msgr2=0x7fe4400ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:29.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.929+0000 7fe43de00640 1 --2- 172.21.15.133:0/1568876180 >> v2:172.21.15.161:3300/0 conn(0x7fe440072f40 0x7fe4400ae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.929+0000 7fe43de00640 1 -- 172.21.15.133:0/1568876180 >> v1:172.21.15.133:6789/0 conn(0x7fe44006dac0 legacy=0x7fe440072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:29.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.929+0000 7fe43de00640 1 -- 172.21.15.133:0/1568876180 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4400ad9c0 con 0x7fe44015eac0 2024-03-07T06:52:29.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.931+0000 7fe43de00640 1 --2- 172.21.15.133:0/1568876180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44015eac0 0x7fe44015eea0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe4400b3de0 tx=0x7fe43002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4098264e8acdf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:29.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.931+0000 7fe43d400640 1 -- 172.21.15.133:0/1568876180 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe430035020 con 0x7fe44015eac0 2024-03-07T06:52:29.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.931+0000 7fe43d400640 1 -- 172.21.15.133:0/1568876180 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe430031830 con 0x7fe44015eac0 2024-03-07T06:52:29.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.931+0000 7fe43d400640 1 -- 172.21.15.133:0/1568876180 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe430031c00 con 0x7fe44015eac0 2024-03-07T06:52:29.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.932+0000 7fe43f200640 1 -- 172.21.15.133:0/1568876180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44015eac0 msgr2=0x7fe44015eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:29.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.932+0000 7fe43f200640 1 --2- 172.21.15.133:0/1568876180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44015eac0 0x7fe44015eea0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe4400b3de0 tx=0x7fe43002b8d0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.932+0000 7fe43f200640 1 -- 172.21.15.133:0/1568876180 shutdown_connections 2024-03-07T06:52:29.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.932+0000 7fe43f200640 1 --2- 172.21.15.133:0/1568876180 >> v2:172.21.15.161:3300/0 conn(0x7fe440072f40 0x7fe4400ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.932+0000 7fe43f200640 1 --2- 172.21.15.133:0/1568876180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44015eac0 0x7fe44015eea0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.932+0000 7fe43f200640 1 -- 172.21.15.133:0/1568876180 >> 172.21.15.133:0/1568876180 conn(0x7fe4400b2e90 msgr2=0x7fe4400b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:29.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.932+0000 7fe43f200640 1 -- 172.21.15.133:0/1568876180 shutdown_connections 2024-03-07T06:52:29.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.932+0000 7fe43f200640 1 -- 172.21.15.133:0/1568876180 wait complete. 2024-03-07T06:52:29.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.934+0000 7fe43f200640 1 Processor -- start 2024-03-07T06:52:29.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.934+0000 7fe43f200640 1 -- start start 2024-03-07T06:52:29.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.935+0000 7fe43f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44006dac0 0x7fe440155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:29.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.935+0000 7fe43f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe440072f40 0x7fe440150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.935+0000 7fe43f200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe44015eac0 0x7fe44015a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.936+0000 7fe43f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44006df20 con 0x7fe44006dac0 2024-03-07T06:52:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.936+0000 7fe43f200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4400adf80 con 0x7fe440072f40 2024-03-07T06:52:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.936+0000 7fe43f200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44006e0a0 con 0x7fe44015eac0 2024-03-07T06:52:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.936+0000 7fe43de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44006dac0 0x7fe440155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.936+0000 7fe43de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44006dac0 0x7fe440155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36642/0 (socket says 172.21.15.133:36642) 2024-03-07T06:52:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.936+0000 7fe43de00640 1 -- 172.21.15.133:0/1540621140 learned_addr learned my addr 172.21.15.133:0/1540621140 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.936+0000 7fe43de00640 1 -- 172.21.15.133:0/1540621140 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe44015eac0 msgr2=0x7fe44015a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.936+0000 7fe43de00640 1 --2- 172.21.15.133:0/1540621140 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe44015eac0 0x7fe44015a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.936+0000 7fe437e00640 1 --2- 172.21.15.133:0/1540621140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe440072f40 0x7fe440150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.936+0000 7fe43de00640 1 -- 172.21.15.133:0/1540621140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe440072f40 msgr2=0x7fe440150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.936+0000 7fe43de00640 1 --2- 172.21.15.133:0/1540621140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe440072f40 0x7fe440150590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.936+0000 7fe43de00640 1 -- 172.21.15.133:0/1540621140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4400af660 con 0x7fe44006dac0 2024-03-07T06:52:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.937+0000 7fe43de00640 1 --2- 172.21.15.133:0/1540621140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44006dac0 0x7fe440155f60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe430009890 tx=0x7fe4300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:29.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.937+0000 7fe436000640 1 -- 172.21.15.133:0/1540621140 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe430040070 con 0x7fe44006dac0 2024-03-07T06:52:29.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.937+0000 7fe436000640 1 -- 172.21.15.133:0/1540621140 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43003e070 con 0x7fe44006dac0 2024-03-07T06:52:29.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.937+0000 7fe43f200640 1 -- 172.21.15.133:0/1540621140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4400af940 con 0x7fe44006dac0 2024-03-07T06:52:29.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.937+0000 7fe436000640 1 -- 172.21.15.133:0/1540621140 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe430035040 con 0x7fe44006dac0 2024-03-07T06:52:29.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.937+0000 7fe43f200640 1 -- 172.21.15.133:0/1540621140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4401564a0 con 0x7fe44006dac0 2024-03-07T06:52:29.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.938+0000 7fe43f200640 1 -- 172.21.15.133:0/1540621140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe440062c90 con 0x7fe44006dac0 2024-03-07T06:52:29.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.939+0000 7fe436000640 1 -- 172.21.15.133:0/1540621140 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe430038020 con 0x7fe44006dac0 2024-03-07T06:52:29.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.940+0000 7fe436000640 1 --2- 172.21.15.133:0/1540621140 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe41c06b4d0 0x7fe41c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:29.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.940+0000 7fe436000640 1 -- 172.21.15.133:0/1540621140 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe4300ab820 con 0x7fe44006dac0 2024-03-07T06:52:29.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.941+0000 7fe437e00640 1 --2- 172.21.15.133:0/1540621140 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe41c06b4d0 0x7fe41c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:29.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.942+0000 7fe437e00640 1 --2- 172.21.15.133:0/1540621140 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe41c06b4d0 0x7fe41c06d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe440162a80 tx=0x7fe428009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:29.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:29.945+0000 7fe436000640 1 -- 172.21.15.133:0/1540621140 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe43007a9d0 con 0x7fe44006dac0 2024-03-07T06:52:30.212 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.212+0000 7fe43f200640 1 -- 172.21.15.133:0/1540621140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fe4400b33f0 con 0x7fe44006dac0 2024-03-07T06:52:30.216 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.216+0000 7fe436000640 1 -- 172.21.15.133:0/1540621140 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe430031800 con 0x7fe44006dac0 2024-03-07T06:52:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.219+0000 7fe43f200640 1 -- 172.21.15.133:0/1540621140 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe41c06b4d0 msgr2=0x7fe41c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.219+0000 7fe43f200640 1 --2- 172.21.15.133:0/1540621140 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe41c06b4d0 0x7fe41c06d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe440162a80 tx=0x7fe428009040 comp rx=0 tx=0).stop 2024-03-07T06:52:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.219+0000 7fe43f200640 1 -- 172.21.15.133:0/1540621140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44006dac0 msgr2=0x7fe440155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:30.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.219+0000 7fe43f200640 1 --2- 172.21.15.133:0/1540621140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44006dac0 0x7fe440155f60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe430009890 tx=0x7fe4300046c0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.219+0000 7fe43f200640 1 -- 172.21.15.133:0/1540621140 shutdown_connections 2024-03-07T06:52:30.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.219+0000 7fe43f200640 1 --2- 172.21.15.133:0/1540621140 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe44015eac0 0x7fe44015a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.219+0000 7fe43f200640 1 --2- 172.21.15.133:0/1540621140 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe41c06b4d0 0x7fe41c06d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.219+0000 7fe43f200640 1 --2- 172.21.15.133:0/1540621140 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe440072f40 0x7fe440150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.219+0000 7fe43f200640 1 --2- 172.21.15.133:0/1540621140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe44006dac0 0x7fe440155f60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.220+0000 7fe43f200640 1 -- 172.21.15.133:0/1540621140 >> 172.21.15.133:0/1540621140 conn(0x7fe4400b2e90 msgr2=0x7fe4400740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:30.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.220+0000 7fe43f200640 1 -- 172.21.15.133:0/1540621140 shutdown_connections 2024-03-07T06:52:30.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.220+0000 7fe43f200640 1 -- 172.21.15.133:0/1540621140 wait complete. 2024-03-07T06:52:30.236 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-07T06:52:30.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.439+0000 7ff374600640 1 Processor -- start 2024-03-07T06:52:30.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.439+0000 7ff374600640 1 -- start start 2024-03-07T06:52:30.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.441+0000 7ff374600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff36c0cf100 0x7ff36c15e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:30.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.441+0000 7ff374600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff36c0c96b0 con 0x7ff36c0af6a0 2024-03-07T06:52:30.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.441+0000 7ff374600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff36c0c9950 con 0x7ff36c158b50 2024-03-07T06:52:30.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.441+0000 7ff374600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff36c0ca280 con 0x7ff36c0cf100 2024-03-07T06:52:30.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.441+0000 7ff371200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff36c0cf100 0x7ff36c15e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:30.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.441+0000 7ff372600640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff36c0af6a0 0x7ff36c0b5070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:43764/0 (socket says 172.21.15.133:43764) 2024-03-07T06:52:30.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.441+0000 7ff372600640 1 -- 172.21.15.133:0/2727798660 learned_addr learned my addr 172.21.15.133:0/2727798660 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:30.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.442+0000 7ff371200640 1 -- 172.21.15.133:0/2727798660 >> v1:172.21.15.161:6789/0 conn(0x7ff36c158b50 legacy=0x7ff36c0ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:30.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.442+0000 7ff371200640 1 -- 172.21.15.133:0/2727798660 >> v1:172.21.15.133:6789/0 conn(0x7ff36c0af6a0 legacy=0x7ff36c0b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:30.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.442+0000 7ff371200640 1 -- 172.21.15.133:0/2727798660 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36c0c9c30 con 0x7ff36c0cf100 2024-03-07T06:52:30.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.442+0000 7ff36be00640 1 -- 172.21.15.133:0/2727798660 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700783561 0 0) 0x7ff36c0c96b0 con 0x7ff36c0af6a0 2024-03-07T06:52:30.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.443+0000 7ff371200640 1 --2- 172.21.15.133:0/2727798660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff36c0cf100 0x7ff36c15e890 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0097a0 tx=0x7ff35c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cd3078ed1f9ea5 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:30.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.443+0000 7ff36be00640 1 -- 172.21.15.133:0/2727798660 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c034070 con 0x7ff36c0cf100 2024-03-07T06:52:30.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.443+0000 7ff36be00640 1 -- 172.21.15.133:0/2727798660 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35c030610 con 0x7ff36c0cf100 2024-03-07T06:52:30.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.443+0000 7ff36be00640 1 -- 172.21.15.133:0/2727798660 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c0309e0 con 0x7ff36c0cf100 2024-03-07T06:52:30.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.444+0000 7ff374600640 1 -- 172.21.15.133:0/2727798660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff36c0cf100 msgr2=0x7ff36c15e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:30.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.444+0000 7ff374600640 1 --2- 172.21.15.133:0/2727798660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff36c0cf100 0x7ff36c15e890 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff35c0097a0 tx=0x7ff35c02b960 comp rx=0 tx=0).stop 2024-03-07T06:52:30.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.444+0000 7ff374600640 1 -- 172.21.15.133:0/2727798660 shutdown_connections 2024-03-07T06:52:30.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.444+0000 7ff374600640 1 --2- 172.21.15.133:0/2727798660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff36c0cf100 0x7ff36c15e890 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.444+0000 7ff374600640 1 -- 172.21.15.133:0/2727798660 >> 172.21.15.133:0/2727798660 conn(0x7ff36c0b31d0 msgr2=0x7ff36c0b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:30.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.444+0000 7ff374600640 1 -- 172.21.15.133:0/2727798660 shutdown_connections 2024-03-07T06:52:30.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.444+0000 7ff374600640 1 -- 172.21.15.133:0/2727798660 wait complete. 2024-03-07T06:52:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.446+0000 7ff374600640 1 Processor -- start 2024-03-07T06:52:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.446+0000 7ff374600640 1 -- start start 2024-03-07T06:52:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.447+0000 7ff374600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff36c0af6a0 0x7ff36c06f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.447+0000 7ff374600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff36c0cf100 0x7ff36c1f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.447+0000 7ff374600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff36c158b50 0x7ff36c06d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.447+0000 7ff374600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c0b5780 con 0x7ff36c158b50 2024-03-07T06:52:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.447+0000 7ff374600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c0cf600 con 0x7ff36c0cf100 2024-03-07T06:52:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.447+0000 7ff374600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff36c0b5900 con 0x7ff36c0af6a0 2024-03-07T06:52:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.447+0000 7ff372600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff36c158b50 0x7ff36c06d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.447+0000 7ff372600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff36c158b50 0x7ff36c06d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36646/0 (socket says 172.21.15.133:36646) 2024-03-07T06:52:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.447+0000 7ff372600640 1 -- 172.21.15.133:0/82463521 learned_addr learned my addr 172.21.15.133:0/82463521 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.448+0000 7ff372600640 1 -- 172.21.15.133:0/82463521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff36c0af6a0 msgr2=0x7ff36c06f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:30.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.448+0000 7ff372600640 1 --2- 172.21.15.133:0/82463521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff36c0af6a0 0x7ff36c06f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.448+0000 7ff372600640 1 -- 172.21.15.133:0/82463521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff36c0cf100 msgr2=0x7ff36c1f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.448+0000 7ff372600640 1 --2- 172.21.15.133:0/82463521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff36c0cf100 0x7ff36c1f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.448+0000 7ff372600640 1 -- 172.21.15.133:0/82463521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36c06e1b0 con 0x7ff36c158b50 2024-03-07T06:52:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.448+0000 7ff372600640 1 --2- 172.21.15.133:0/82463521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff36c158b50 0x7ff36c06d530 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c800 tx=0x7ff36400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.448+0000 7ff369600640 1 -- 172.21.15.133:0/82463521 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364018070 con 0x7ff36c158b50 2024-03-07T06:52:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.448+0000 7ff369600640 1 -- 172.21.15.133:0/82463521 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff364004d80 con 0x7ff36c158b50 2024-03-07T06:52:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.448+0000 7ff369600640 1 -- 172.21.15.133:0/82463521 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36400a880 con 0x7ff36c158b50 2024-03-07T06:52:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.448+0000 7ff374600640 1 -- 172.21.15.133:0/82463521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36c06e490 con 0x7ff36c158b50 2024-03-07T06:52:30.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.448+0000 7ff374600640 1 -- 172.21.15.133:0/82463521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff36c06ea50 con 0x7ff36c158b50 2024-03-07T06:52:30.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.449+0000 7ff374600640 1 -- 172.21.15.133:0/82463521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff36c06bb20 con 0x7ff36c158b50 2024-03-07T06:52:30.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.450+0000 7ff369600640 1 -- 172.21.15.133:0/82463521 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff36400aa20 con 0x7ff36c158b50 2024-03-07T06:52:30.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.451+0000 7ff369600640 1 --2- 172.21.15.133:0/82463521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff35006b4d0 0x7ff35006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:30.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.451+0000 7ff369600640 1 -- 172.21.15.133:0/82463521 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff3640894f0 con 0x7ff36c158b50 2024-03-07T06:52:30.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.452+0000 7ff371c00640 1 --2- 172.21.15.133:0/82463521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff35006b4d0 0x7ff35006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:30.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.453+0000 7ff371c00640 1 --2- 172.21.15.133:0/82463521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff35006b4d0 0x7ff35006d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff360000f60 tx=0x7ff3600079b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:30.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.456+0000 7ff369600640 1 -- 172.21.15.133:0/82463521 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff36c06bb20 con 0x7ff36c158b50 2024-03-07T06:52:30.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.703+0000 7ff374600640 1 -- 172.21.15.133:0/82463521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7ff36c062c90 con 0x7ff36c158b50 2024-03-07T06:52:30.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.707+0000 7ff369600640 1 -- 172.21.15.133:0/82463521 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff3640586f0 con 0x7ff36c158b50 2024-03-07T06:52:30.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.709+0000 7ff374600640 1 -- 172.21.15.133:0/82463521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff35006b4d0 msgr2=0x7ff35006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:30.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.709+0000 7ff374600640 1 --2- 172.21.15.133:0/82463521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff35006b4d0 0x7ff35006d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff360000f60 tx=0x7ff3600079b0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.709+0000 7ff374600640 1 -- 172.21.15.133:0/82463521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff36c158b50 msgr2=0x7ff36c06d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:30.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.709+0000 7ff374600640 1 --2- 172.21.15.133:0/82463521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff36c158b50 0x7ff36c06d530 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c800 tx=0x7ff36400ccc0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.710+0000 7ff374600640 1 -- 172.21.15.133:0/82463521 shutdown_connections 2024-03-07T06:52:30.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.710+0000 7ff374600640 1 --2- 172.21.15.133:0/82463521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff36c158b50 0x7ff36c06d530 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.710+0000 7ff374600640 1 --2- 172.21.15.133:0/82463521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff35006b4d0 0x7ff35006d990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.710+0000 7ff374600640 1 --2- 172.21.15.133:0/82463521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff36c0cf100 0x7ff36c1f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.710+0000 7ff374600640 1 --2- 172.21.15.133:0/82463521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff36c0af6a0 0x7ff36c06f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.710+0000 7ff374600640 1 -- 172.21.15.133:0/82463521 >> 172.21.15.133:0/82463521 conn(0x7ff36c0b31d0 msgr2=0x7ff36c0740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:30.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.710+0000 7ff374600640 1 -- 172.21.15.133:0/82463521 shutdown_connections 2024-03-07T06:52:30.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.710+0000 7ff374600640 1 -- 172.21.15.133:0/82463521 wait complete. 2024-03-07T06:52:30.728 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-07T06:52:30.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.937+0000 7f2344e00640 1 Processor -- start 2024-03-07T06:52:30.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.937+0000 7f2344e00640 1 -- start start 2024-03-07T06:52:30.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.939+0000 7f2344e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f234015fbb0 0x7f23400cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:30.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.939+0000 7f2344e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f23400cdee0 0x7f234015c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:30.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.939+0000 7f2344e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23400731b0 con 0x7f2340072950 2024-03-07T06:52:30.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.939+0000 7f2344e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2340072f60 con 0x7f23400cdee0 2024-03-07T06:52:30.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.939+0000 7f2344e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2340073490 con 0x7f234015fbb0 2024-03-07T06:52:30.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.939+0000 7f233e800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f23400cdee0 0x7f234015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:30.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.939+0000 7f2335400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2340072950 0x7f234015ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34224/0 (socket says 172.21.15.133:34224) 2024-03-07T06:52:30.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.939+0000 7f233e800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f23400cdee0 0x7f234015c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43452/0 (socket says 172.21.15.133:43452) 2024-03-07T06:52:30.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.939+0000 7f2335400640 1 -- 172.21.15.133:0/2571312257 learned_addr learned my addr 172.21.15.133:0/2571312257 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:30.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.940+0000 7f233de00640 1 --2- 172.21.15.133:0/2571312257 >> v2:172.21.15.161:3300/0 conn(0x7f234015fbb0 0x7f23400cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:30.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.940+0000 7f233e800640 1 -- 172.21.15.133:0/2571312257 >> v2:172.21.15.161:3300/0 conn(0x7f234015fbb0 msgr2=0x7f23400cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:30.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.940+0000 7f233e800640 1 --2- 172.21.15.133:0/2571312257 >> v2:172.21.15.161:3300/0 conn(0x7f234015fbb0 0x7f23400cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.940+0000 7f233e800640 1 -- 172.21.15.133:0/2571312257 >> v1:172.21.15.133:6789/0 conn(0x7f2340072950 legacy=0x7f234015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:30.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.940+0000 7f233e800640 1 -- 172.21.15.133:0/2571312257 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2340072150 con 0x7f23400cdee0 2024-03-07T06:52:30.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.940+0000 7f233de00640 1 --2- 172.21.15.133:0/2571312257 >> v2:172.21.15.161:3300/0 conn(0x7f234015fbb0 0x7f23400cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:30.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.941+0000 7f233e800640 1 --2- 172.21.15.133:0/2571312257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23400cdee0 0x7f234015c550 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f234006ffa0 tx=0x7f233802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2362d473d0c032 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:30.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.941+0000 7f233d400640 1 -- 172.21.15.133:0/2571312257 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23380315a0 con 0x7f23400cdee0 2024-03-07T06:52:30.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.941+0000 7f233d400640 1 -- 172.21.15.133:0/2571312257 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2338037070 con 0x7f23400cdee0 2024-03-07T06:52:30.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.941+0000 7f233d400640 1 -- 172.21.15.133:0/2571312257 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338031960 con 0x7f23400cdee0 2024-03-07T06:52:30.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.942+0000 7f2344e00640 1 -- 172.21.15.133:0/2571312257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23400cdee0 msgr2=0x7f234015c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:30.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.942+0000 7f2344e00640 1 --2- 172.21.15.133:0/2571312257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23400cdee0 0x7f234015c550 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f234006ffa0 tx=0x7f233802b840 comp rx=0 tx=0).stop 2024-03-07T06:52:30.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.942+0000 7f2344e00640 1 -- 172.21.15.133:0/2571312257 shutdown_connections 2024-03-07T06:52:30.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.942+0000 7f2344e00640 1 --2- 172.21.15.133:0/2571312257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23400cdee0 0x7f234015c550 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.942+0000 7f2344e00640 1 --2- 172.21.15.133:0/2571312257 >> v2:172.21.15.161:3300/0 conn(0x7f234015fbb0 0x7f23400cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.942+0000 7f2344e00640 1 -- 172.21.15.133:0/2571312257 >> 172.21.15.133:0/2571312257 conn(0x7f23400b3120 msgr2=0x7f23400b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:30.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.942+0000 7f2344e00640 1 -- 172.21.15.133:0/2571312257 shutdown_connections 2024-03-07T06:52:30.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.942+0000 7f2344e00640 1 -- 172.21.15.133:0/2571312257 wait complete. 2024-03-07T06:52:30.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.944+0000 7f2344e00640 1 Processor -- start 2024-03-07T06:52:30.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.944+0000 7f2344e00640 1 -- start start 2024-03-07T06:52:30.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.945+0000 7f2344e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23400cdee0 0x7f23400b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:30.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.945+0000 7f2344e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f234015fbb0 0x7f23400ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:30.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.945+0000 7f2344e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23400b5bb0 0x7f23400ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:30.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.945+0000 7f2344e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23400ce2c0 con 0x7f23400b5bb0 2024-03-07T06:52:30.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.945+0000 7f2344e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2340160980 con 0x7f23400cdee0 2024-03-07T06:52:30.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.945+0000 7f2344e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23400b1a00 con 0x7f234015fbb0 2024-03-07T06:52:30.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.945+0000 7f233e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23400b5bb0 0x7f23400ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:30.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.945+0000 7f233e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23400b5bb0 0x7f23400ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43456/0 (socket says 172.21.15.133:43456) 2024-03-07T06:52:30.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.945+0000 7f233e800640 1 -- 172.21.15.133:0/2320234215 learned_addr learned my addr 172.21.15.133:0/2320234215 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:30.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.946+0000 7f233de00640 1 --2- 172.21.15.133:0/2320234215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23400cdee0 0x7f23400b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:30.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.946+0000 7f2335400640 1 --2- 172.21.15.133:0/2320234215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f234015fbb0 0x7f23400ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:30.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.946+0000 7f233e800640 1 -- 172.21.15.133:0/2320234215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f234015fbb0 msgr2=0x7f23400ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:30.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.946+0000 7f233e800640 1 --2- 172.21.15.133:0/2320234215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f234015fbb0 0x7f23400ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.946+0000 7f233e800640 1 -- 172.21.15.133:0/2320234215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23400cdee0 msgr2=0x7f23400b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:30.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.946+0000 7f233e800640 1 --2- 172.21.15.133:0/2320234215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23400cdee0 0x7f23400b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:30.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.946+0000 7f233e800640 1 -- 172.21.15.133:0/2320234215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234006d3d0 con 0x7f23400b5bb0 2024-03-07T06:52:30.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.946+0000 7f233e800640 1 --2- 172.21.15.133:0/2320234215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23400b5bb0 0x7f23400ccae0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2338034040 tx=0x7f23380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:30.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.946+0000 7f2336a00640 1 -- 172.21.15.133:0/2320234215 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338032840 con 0x7f23400b5bb0 2024-03-07T06:52:30.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.947+0000 7f2336a00640 1 -- 172.21.15.133:0/2320234215 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23380329e0 con 0x7f23400b5bb0 2024-03-07T06:52:30.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.947+0000 7f2344e00640 1 -- 172.21.15.133:0/2320234215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234006d6b0 con 0x7f23400b5bb0 2024-03-07T06:52:30.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.947+0000 7f2336a00640 1 -- 172.21.15.133:0/2320234215 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338032dd0 con 0x7f23400b5bb0 2024-03-07T06:52:30.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.947+0000 7f2344e00640 1 -- 172.21.15.133:0/2320234215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234006dc70 con 0x7f23400b5bb0 2024-03-07T06:52:30.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.948+0000 7f2344e00640 1 -- 172.21.15.133:0/2320234215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2340062c90 con 0x7f23400b5bb0 2024-03-07T06:52:30.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.949+0000 7f2336a00640 1 -- 172.21.15.133:0/2320234215 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f233804c020 con 0x7f23400b5bb0 2024-03-07T06:52:30.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.950+0000 7f2336a00640 1 --2- 172.21.15.133:0/2320234215 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f231c06b4d0 0x7f231c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:30.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.950+0000 7f2336a00640 1 -- 172.21.15.133:0/2320234215 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f233802f020 con 0x7f23400b5bb0 2024-03-07T06:52:30.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.951+0000 7f233de00640 1 --2- 172.21.15.133:0/2320234215 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f231c06b4d0 0x7f231c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:30.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.952+0000 7f233de00640 1 --2- 172.21.15.133:0/2320234215 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f231c06b4d0 0x7f231c06d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f23300075b0 tx=0x7f2330007540 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:30.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:30.955+0000 7f2336a00640 1 -- 172.21.15.133:0/2320234215 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2338040070 con 0x7f23400b5bb0 2024-03-07T06:52:31.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.201+0000 7f2344e00640 1 -- 172.21.15.133:0/2320234215 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f2340068340 con 0x7f23400b5bb0 2024-03-07T06:52:31.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.207+0000 7f2336a00640 1 -- 172.21.15.133:0/2320234215 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f233807af40 con 0x7f23400b5bb0 2024-03-07T06:52:31.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.209+0000 7f2344e00640 1 -- 172.21.15.133:0/2320234215 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f231c06b4d0 msgr2=0x7f231c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:31.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.209+0000 7f2344e00640 1 --2- 172.21.15.133:0/2320234215 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f231c06b4d0 0x7f231c06d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f23300075b0 tx=0x7f2330007540 comp rx=0 tx=0).stop 2024-03-07T06:52:31.209 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.209+0000 7f2344e00640 1 -- 172.21.15.133:0/2320234215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23400b5bb0 msgr2=0x7f23400ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:31.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.209+0000 7f2344e00640 1 --2- 172.21.15.133:0/2320234215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23400b5bb0 0x7f23400ccae0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2338034040 tx=0x7f23380046c0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.210+0000 7f2344e00640 1 -- 172.21.15.133:0/2320234215 shutdown_connections 2024-03-07T06:52:31.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.210+0000 7f2344e00640 1 --2- 172.21.15.133:0/2320234215 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23400b5bb0 0x7f23400ccae0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.210+0000 7f2344e00640 1 --2- 172.21.15.133:0/2320234215 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f234015fbb0 0x7f23400ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.210+0000 7f2344e00640 1 --2- 172.21.15.133:0/2320234215 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f231c06b4d0 0x7f231c06d990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.210+0000 7f2344e00640 1 --2- 172.21.15.133:0/2320234215 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23400cdee0 0x7f23400b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.210+0000 7f2344e00640 1 -- 172.21.15.133:0/2320234215 >> 172.21.15.133:0/2320234215 conn(0x7f23400b3120 msgr2=0x7f2340071610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:31.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.210+0000 7f2344e00640 1 -- 172.21.15.133:0/2320234215 shutdown_connections 2024-03-07T06:52:31.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.210+0000 7f2344e00640 1 -- 172.21.15.133:0/2320234215 wait complete. 2024-03-07T06:52:31.229 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:52:31.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.438+0000 7fc5d7a00640 1 Processor -- start 2024-03-07T06:52:31.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.438+0000 7fc5d7a00640 1 -- start start 2024-03-07T06:52:31.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.439+0000 7fc5d7a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc5d015b1b0 0x7fc5d00b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:31.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.439+0000 7fc5d7a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5d00ce5f0 0x7fc5d0071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:31.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.440+0000 7fc5d7a00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d006f140 con 0x7fc5d00af950 2024-03-07T06:52:31.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.440+0000 7fc5d7a00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d0071610 con 0x7fc5d015b1b0 2024-03-07T06:52:31.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.440+0000 7fc5d7a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d006f420 con 0x7fc5d00ce5f0 2024-03-07T06:52:31.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.440+0000 7fc5d5000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc5d015b1b0 0x7fc5d00b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:31.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.440+0000 7fc5c7e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc5d00af950 0x7fc5d00cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34240/0 (socket says 172.21.15.133:34240) 2024-03-07T06:52:31.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.440+0000 7fc5c7e00640 1 -- 172.21.15.133:0/123039407 learned_addr learned my addr 172.21.15.133:0/123039407 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:31.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.440+0000 7fc5d5000640 1 -- 172.21.15.133:0/123039407 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5d00ce5f0 msgr2=0x7fc5d0071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:31.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.440+0000 7fc5d5000640 1 --2- 172.21.15.133:0/123039407 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5d00ce5f0 0x7fc5d0071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.440+0000 7fc5d5000640 1 -- 172.21.15.133:0/123039407 >> v1:172.21.15.133:6789/0 conn(0x7fc5d00af950 legacy=0x7fc5d00cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:31.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.440+0000 7fc5d5000640 1 -- 172.21.15.133:0/123039407 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d0071860 con 0x7fc5d015b1b0 2024-03-07T06:52:31.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.441+0000 7fc5d5000640 1 --2- 172.21.15.133:0/123039407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d015b1b0 0x7fc5d00b3c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80097a0 tx=0x7fc5b802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6735c6305b333a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:31.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.442+0000 7fc5cfe00640 1 -- 172.21.15.133:0/123039407 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8031780 con 0x7fc5d015b1b0 2024-03-07T06:52:31.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.442+0000 7fc5cfe00640 1 -- 172.21.15.133:0/123039407 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b8037070 con 0x7fc5d015b1b0 2024-03-07T06:52:31.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.442+0000 7fc5cfe00640 1 -- 172.21.15.133:0/123039407 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8031b40 con 0x7fc5d015b1b0 2024-03-07T06:52:31.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.442+0000 7fc5d7a00640 1 -- 172.21.15.133:0/123039407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d015b1b0 msgr2=0x7fc5d00b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:31.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.442+0000 7fc5d7a00640 1 --2- 172.21.15.133:0/123039407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d015b1b0 0x7fc5d00b3c10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc5b80097a0 tx=0x7fc5b802b820 comp rx=0 tx=0).stop 2024-03-07T06:52:31.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.442+0000 7fc5d7a00640 1 -- 172.21.15.133:0/123039407 shutdown_connections 2024-03-07T06:52:31.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.442+0000 7fc5d7a00640 1 --2- 172.21.15.133:0/123039407 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5d00ce5f0 0x7fc5d0071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.442+0000 7fc5d7a00640 1 --2- 172.21.15.133:0/123039407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d015b1b0 0x7fc5d00b3c10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.442+0000 7fc5d7a00640 1 -- 172.21.15.133:0/123039407 >> 172.21.15.133:0/123039407 conn(0x7fc5d00b3120 msgr2=0x7fc5d00b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:31.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.442+0000 7fc5d7a00640 1 -- 172.21.15.133:0/123039407 shutdown_connections 2024-03-07T06:52:31.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.443+0000 7fc5d7a00640 1 -- 172.21.15.133:0/123039407 wait complete. 2024-03-07T06:52:31.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.444+0000 7fc5d7a00640 1 Processor -- start 2024-03-07T06:52:31.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.444+0000 7fc5d7a00640 1 -- start start 2024-03-07T06:52:31.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.445+0000 7fc5d7a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5d00af950 0x7fc5d00aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:31.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.445+0000 7fc5d7a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00ce5f0 0x7fc5d014a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:31.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.445+0000 7fc5d7a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5d015b1b0 0x7fc5d015cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:31.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.445+0000 7fc5d7a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d015b590 con 0x7fc5d00ce5f0 2024-03-07T06:52:31.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.445+0000 7fc5d7a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d006ee30 con 0x7fc5d00af950 2024-03-07T06:52:31.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.445+0000 7fc5d7a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d015b710 con 0x7fc5d015b1b0 2024-03-07T06:52:31.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.445+0000 7fc5c7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00ce5f0 0x7fc5d014a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:31.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.446+0000 7fc5c7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00ce5f0 0x7fc5d014a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43486/0 (socket says 172.21.15.133:43486) 2024-03-07T06:52:31.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.446+0000 7fc5c7e00640 1 -- 172.21.15.133:0/3065434721 learned_addr learned my addr 172.21.15.133:0/3065434721 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:31.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.446+0000 7fc5d5000640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5d00af950 0x7fc5d00aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:31.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.446+0000 7fc5d5a00640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5d015b1b0 0x7fc5d015cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:31.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.446+0000 7fc5c7e00640 1 -- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5d015b1b0 msgr2=0x7fc5d015cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:31.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.446+0000 7fc5c7e00640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5d015b1b0 0x7fc5d015cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.446+0000 7fc5c7e00640 1 -- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5d00af950 msgr2=0x7fc5d00aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:31.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.446+0000 7fc5c7e00640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5d00af950 0x7fc5d00aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.446+0000 7fc5c7e00640 1 -- 172.21.15.133:0/3065434721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d00adaf0 con 0x7fc5d00ce5f0 2024-03-07T06:52:31.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.446+0000 7fc5d5000640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5d00af950 0x7fc5d00aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:31.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.446+0000 7fc5c7e00640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00ce5f0 0x7fc5d014a960 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000c430 tx=0x7fc5c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:31.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.447+0000 7fc5cd600640 1 -- 172.21.15.133:0/3065434721 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0017070 con 0x7fc5d00ce5f0 2024-03-07T06:52:31.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.447+0000 7fc5cd600640 1 -- 172.21.15.133:0/3065434721 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c0005ce0 con 0x7fc5d00ce5f0 2024-03-07T06:52:31.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.447+0000 7fc5cd600640 1 -- 172.21.15.133:0/3065434721 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00072f0 con 0x7fc5d00ce5f0 2024-03-07T06:52:31.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.447+0000 7fc5d7a00640 1 -- 172.21.15.133:0/3065434721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d00addd0 con 0x7fc5d00ce5f0 2024-03-07T06:52:31.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.447+0000 7fc5d7a00640 1 -- 172.21.15.133:0/3065434721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d00ae670 con 0x7fc5d00ce5f0 2024-03-07T06:52:31.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.448+0000 7fc5d7a00640 1 -- 172.21.15.133:0/3065434721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d00adfa0 con 0x7fc5d00ce5f0 2024-03-07T06:52:31.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.449+0000 7fc5cd600640 1 -- 172.21.15.133:0/3065434721 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc5c001a070 con 0x7fc5d00ce5f0 2024-03-07T06:52:31.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.450+0000 7fc5cd600640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc5ac06b4d0 0x7fc5ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:31.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.450+0000 7fc5cd600640 1 -- 172.21.15.133:0/3065434721 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc5c00892d0 con 0x7fc5d00ce5f0 2024-03-07T06:52:31.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.451+0000 7fc5d5000640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc5ac06b4d0 0x7fc5ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:31.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.452+0000 7fc5d5000640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc5ac06b4d0 0x7fc5ac06d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8036000 tx=0x7fc5b8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:31.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.455+0000 7fc5cd600640 1 -- 172.21.15.133:0/3065434721 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc5c00584a0 con 0x7fc5d00ce5f0 2024-03-07T06:52:31.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.636+0000 7fc5d7a00640 1 -- 172.21.15.133:0/3065434721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5d00ae870 con 0x7fc5d00ce5f0 2024-03-07T06:52:31.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.638+0000 7fc5cd600640 1 -- 172.21.15.133:0/3065434721 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+12939 (secure 0 0 0) 0x7fc5d00ae870 con 0x7fc5d00ce5f0 2024-03-07T06:52:31.638 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:31.638 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":31,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:52:27.652508+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:52:31.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.641+0000 7fc5d7a00640 1 -- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc5ac06b4d0 msgr2=0x7fc5ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:31.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.641+0000 7fc5d7a00640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc5ac06b4d0 0x7fc5ac06d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8036000 tx=0x7fc5b8008030 comp rx=0 tx=0).stop 2024-03-07T06:52:31.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.641+0000 7fc5d7a00640 1 -- 172.21.15.133:0/3065434721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00ce5f0 msgr2=0x7fc5d014a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:31.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.641+0000 7fc5d7a00640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00ce5f0 0x7fc5d014a960 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc5c000c430 tx=0x7fc5c000c8f0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.641+0000 7fc5d7a00640 1 -- 172.21.15.133:0/3065434721 shutdown_connections 2024-03-07T06:52:31.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.641+0000 7fc5d7a00640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5d015b1b0 0x7fc5d015cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.641+0000 7fc5d7a00640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc5d00ce5f0 0x7fc5d014a960 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.641+0000 7fc5d7a00640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc5ac06b4d0 0x7fc5ac06d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.641+0000 7fc5d7a00640 1 --2- 172.21.15.133:0/3065434721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5d00af950 0x7fc5d00aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.641+0000 7fc5d7a00640 1 -- 172.21.15.133:0/3065434721 >> 172.21.15.133:0/3065434721 conn(0x7fc5d00b3120 msgr2=0x7fc5d00b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:31.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.641+0000 7fc5d7a00640 1 -- 172.21.15.133:0/3065434721 shutdown_connections 2024-03-07T06:52:31.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.641+0000 7fc5d7a00640 1 -- 172.21.15.133:0/3065434721 wait complete. 2024-03-07T06:52:31.660 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:52:31.660 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:52:31.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.872+0000 7f1f3a800640 1 Processor -- start 2024-03-07T06:52:31.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.872+0000 7f1f3a800640 1 -- start start 2024-03-07T06:52:31.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.874+0000 7f1f3a800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1f34071050 0x7f1f340d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:31.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.874+0000 7f1f3a800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f34071610 con 0x7f1f34162110 2024-03-07T06:52:31.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.874+0000 7f1f3a800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f340718f0 con 0x7f1f34070bc0 2024-03-07T06:52:31.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.874+0000 7f1f3a800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f34071bd0 con 0x7f1f34071050 2024-03-07T06:52:31.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.874+0000 7f1f33400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f1f34162110 0x7f1f341624f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34252/0 (socket says 172.21.15.133:34252) 2024-03-07T06:52:31.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.874+0000 7f1f33400640 1 -- 172.21.15.133:0/2253370648 learned_addr learned my addr 172.21.15.133:0/2253370648 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:31.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.874+0000 7f1f33e00640 1 --2- 172.21.15.133:0/2253370648 >> v2:172.21.15.133:3300/0 conn(0x7f1f34071050 0x7f1f340d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:31.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.875+0000 7f1f32000640 1 -- 172.21.15.133:0/2253370648 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981293133 0 0) 0x7f1f34071610 con 0x7f1f34162110 2024-03-07T06:52:31.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.875+0000 7f1f32000640 1 -- 172.21.15.133:0/2253370648 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f18003520 con 0x7f1f34162110 2024-03-07T06:52:31.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.875+0000 7f1f33e00640 1 -- 172.21.15.133:0/2253370648 >> v1:172.21.15.161:6789/0 conn(0x7f1f34070bc0 legacy=0x7f1f340d26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:31.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.875+0000 7f1f33e00640 1 -- 172.21.15.133:0/2253370648 >> v1:172.21.15.133:6789/0 conn(0x7f1f34162110 legacy=0x7f1f341624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:31.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.875+0000 7f1f33e00640 1 -- 172.21.15.133:0/2253370648 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f34071eb0 con 0x7f1f34071050 2024-03-07T06:52:31.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.876+0000 7f1f33e00640 1 --2- 172.21.15.133:0/2253370648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f34071050 0x7f1f340d2e00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0097a0 tx=0x7f1f2c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc19377fc6b45fba server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.876+0000 7f1f32000640 1 -- 172.21.15.133:0/2253370648 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c0315a0 con 0x7f1f34071050 2024-03-07T06:52:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.877+0000 7f1f32000640 1 -- 172.21.15.133:0/2253370648 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c037070 con 0x7f1f34071050 2024-03-07T06:52:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.877+0000 7f1f32000640 1 -- 172.21.15.133:0/2253370648 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c031960 con 0x7f1f34071050 2024-03-07T06:52:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.877+0000 7f1f3a800640 1 -- 172.21.15.133:0/2253370648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f34071050 msgr2=0x7f1f340d2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.877+0000 7f1f3a800640 1 --2- 172.21.15.133:0/2253370648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f34071050 0x7f1f340d2e00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c0097a0 tx=0x7f1f2c02b840 comp rx=0 tx=0).stop 2024-03-07T06:52:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.877+0000 7f1f3a800640 1 -- 172.21.15.133:0/2253370648 shutdown_connections 2024-03-07T06:52:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.877+0000 7f1f3a800640 1 --2- 172.21.15.133:0/2253370648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f34071050 0x7f1f340d2e00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.877+0000 7f1f3a800640 1 -- 172.21.15.133:0/2253370648 >> 172.21.15.133:0/2253370648 conn(0x7f1f340d0e10 msgr2=0x7f1f340d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:31.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.877+0000 7f1f3a800640 1 -- 172.21.15.133:0/2253370648 shutdown_connections 2024-03-07T06:52:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.877+0000 7f1f3a800640 1 -- 172.21.15.133:0/2253370648 wait complete. 2024-03-07T06:52:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.879+0000 7f1f3a800640 1 Processor -- start 2024-03-07T06:52:31.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.879+0000 7f1f3a800640 1 -- start start 2024-03-07T06:52:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.880+0000 7f1f3a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f34070bc0 0x7f1f34143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.880+0000 7f1f3a800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1f34071050 0x7f1f3415b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.880+0000 7f1f3a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f34162110 0x7f1f340b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.880+0000 7f1f3a800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34162f80 con 0x7f1f34070bc0 2024-03-07T06:52:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.880+0000 7f1f3a800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f34162e00 con 0x7f1f34162110 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.880+0000 7f1f3a800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3415f8e0 con 0x7f1f34071050 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.880+0000 7f1f33400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f34070bc0 0x7f1f34143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.880+0000 7f1f33400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f34070bc0 0x7f1f34143b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43506/0 (socket says 172.21.15.133:43506) 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.880+0000 7f1f33400640 1 -- 172.21.15.133:0/467364491 learned_addr learned my addr 172.21.15.133:0/467364491 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.881+0000 7f1f33400640 1 -- 172.21.15.133:0/467364491 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1f34071050 msgr2=0x7f1f3415b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.881+0000 7f1f33400640 1 --2- 172.21.15.133:0/467364491 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1f34071050 0x7f1f3415b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.881+0000 7f1f33400640 1 -- 172.21.15.133:0/467364491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f34162110 msgr2=0x7f1f340b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.881+0000 7f1f33400640 1 --2- 172.21.15.133:0/467364491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f34162110 0x7f1f340b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.881+0000 7f1f33400640 1 -- 172.21.15.133:0/467364491 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3415bf00 con 0x7f1f34070bc0 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.881+0000 7f1f33400640 1 --2- 172.21.15.133:0/467364491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f34070bc0 0x7f1f34143b90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00c430 tx=0x7f1f1c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.881+0000 7f1f2b400640 1 -- 172.21.15.133:0/467364491 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c017070 con 0x7f1f34070bc0 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.881+0000 7f1f2b400640 1 -- 172.21.15.133:0/467364491 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1c005ce0 con 0x7f1f34070bc0 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.881+0000 7f1f3a800640 1 -- 172.21.15.133:0/467364491 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f3415c1e0 con 0x7f1f34070bc0 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.881+0000 7f1f2b400640 1 -- 172.21.15.133:0/467364491 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c0072f0 con 0x7f1f34070bc0 2024-03-07T06:52:31.882 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.881+0000 7f1f3a800640 1 -- 172.21.15.133:0/467364491 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3415c7a0 con 0x7f1f34070bc0 2024-03-07T06:52:31.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.883+0000 7f1f3a800640 1 -- 172.21.15.133:0/467364491 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f34062c90 con 0x7f1f34070bc0 2024-03-07T06:52:31.883 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.883+0000 7f1f2b400640 1 -- 172.21.15.133:0/467364491 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1f1c01a070 con 0x7f1f34070bc0 2024-03-07T06:52:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.884+0000 7f1f2b400640 1 --2- 172.21.15.133:0/467364491 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1f1006b4d0 0x7f1f1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.884+0000 7f1f2b400640 1 -- 172.21.15.133:0/467364491 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1f1c0893c0 con 0x7f1f34070bc0 2024-03-07T06:52:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.885+0000 7f1f32a00640 1 --2- 172.21.15.133:0/467364491 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1f1006b4d0 0x7f1f1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.886+0000 7f1f32a00640 1 --2- 172.21.15.133:0/467364491 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1f1006b4d0 0x7f1f1006d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f340ce0d0 tx=0x7f1f24009230 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:31.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:31.889+0000 7f1f2b400640 1 -- 172.21.15.133:0/467364491 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1f1c0584f0 con 0x7f1f34070bc0 2024-03-07T06:52:32.083 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.082+0000 7f1f3a800640 1 -- 172.21.15.133:0/467364491 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f340cab80 con 0x7f1f34070bc0 2024-03-07T06:52:32.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.083+0000 7f1f2b400640 1 -- 172.21.15.133:0/467364491 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f1c057df0 con 0x7f1f34070bc0 2024-03-07T06:52:32.084 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:52:32.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.085+0000 7f1f3a800640 1 -- 172.21.15.133:0/467364491 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1f1006b4d0 msgr2=0x7f1f1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:32.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.085+0000 7f1f3a800640 1 --2- 172.21.15.133:0/467364491 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1f1006b4d0 0x7f1f1006d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f340ce0d0 tx=0x7f1f24009230 comp rx=0 tx=0).stop 2024-03-07T06:52:32.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.086+0000 7f1f3a800640 1 -- 172.21.15.133:0/467364491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f34070bc0 msgr2=0x7f1f34143b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:32.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.086+0000 7f1f3a800640 1 --2- 172.21.15.133:0/467364491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f34070bc0 0x7f1f34143b90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c00c430 tx=0x7f1f1c00c8f0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.086+0000 7f1f3a800640 1 -- 172.21.15.133:0/467364491 shutdown_connections 2024-03-07T06:52:32.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.086+0000 7f1f3a800640 1 --2- 172.21.15.133:0/467364491 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1f1006b4d0 0x7f1f1006d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.086+0000 7f1f3a800640 1 --2- 172.21.15.133:0/467364491 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1f34162110 0x7f1f340b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.086+0000 7f1f3a800640 1 --2- 172.21.15.133:0/467364491 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1f34071050 0x7f1f3415b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.086+0000 7f1f3a800640 1 --2- 172.21.15.133:0/467364491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f34070bc0 0x7f1f34143b90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.086+0000 7f1f3a800640 1 -- 172.21.15.133:0/467364491 >> 172.21.15.133:0/467364491 conn(0x7f1f340d0e10 msgr2=0x7f1f340b7410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:32.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.086+0000 7f1f3a800640 1 -- 172.21.15.133:0/467364491 shutdown_connections 2024-03-07T06:52:32.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.086+0000 7f1f3a800640 1 -- 172.21.15.133:0/467364491 wait complete. 2024-03-07T06:52:32.105 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-07T06:52:32.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.312+0000 7f7cbae00640 1 Processor -- start 2024-03-07T06:52:32.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.312+0000 7f7cbae00640 1 -- start start 2024-03-07T06:52:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.313+0000 7f7cbae00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7cb40cf5d0 0x7f7cb4072830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.313+0000 7f7cbae00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cb4072d70 0x7f7cb415c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.313+0000 7f7cbae00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb4160760 con 0x7f7cb415bd30 2024-03-07T06:52:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.313+0000 7f7cbae00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40731c0 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.313+0000 7f7cbae00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb415f9a0 con 0x7f7cb4072d70 2024-03-07T06:52:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.314+0000 7f7cb9000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7cb40cf5d0 0x7f7cb4072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.314+0000 7f7cb9000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7cb40cf5d0 0x7f7cb4072830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43520/0 (socket says 172.21.15.133:43520) 2024-03-07T06:52:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.314+0000 7f7cb9000640 1 -- 172.21.15.133:0/3725792696 learned_addr learned my addr 172.21.15.133:0/3725792696 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.314+0000 7f7cba400640 1 --2- 172.21.15.133:0/3725792696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cb4072d70 0x7f7cb415c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.314+0000 7f7cb9000640 1 -- 172.21.15.133:0/3725792696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cb4072d70 msgr2=0x7f7cb415c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.314+0000 7f7cb9000640 1 --2- 172.21.15.133:0/3725792696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cb4072d70 0x7f7cb415c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.314+0000 7f7cb9000640 1 -- 172.21.15.133:0/3725792696 >> v1:172.21.15.161:6789/0 conn(0x7f7cb415bd30 legacy=0x7f7cb406e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:32.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.314+0000 7f7cb9000640 1 -- 172.21.15.133:0/3725792696 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb415fc80 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.315+0000 7f7cb9000640 1 --2- 172.21.15.133:0/3725792696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb40cf5d0 0x7f7cb4072830 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009870 tx=0x7f7ca002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460c3d4402e4d01b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.315+0000 7f7cb3e00640 1 -- 172.21.15.133:0/3725792696 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0035020 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.315+0000 7f7cb3e00640 1 -- 172.21.15.133:0/3725792696 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca0031640 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.315+0000 7f7cb3e00640 1 -- 172.21.15.133:0/3725792696 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca002d040 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.316+0000 7f7cbae00640 1 -- 172.21.15.133:0/3725792696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb40cf5d0 msgr2=0x7f7cb4072830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.316+0000 7f7cbae00640 1 --2- 172.21.15.133:0/3725792696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb40cf5d0 0x7f7cb4072830 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009870 tx=0x7f7ca002b8f0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.316+0000 7f7cbae00640 1 -- 172.21.15.133:0/3725792696 shutdown_connections 2024-03-07T06:52:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.316+0000 7f7cbae00640 1 --2- 172.21.15.133:0/3725792696 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cb4072d70 0x7f7cb415c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.316+0000 7f7cbae00640 1 --2- 172.21.15.133:0/3725792696 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb40cf5d0 0x7f7cb4072830 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.316+0000 7f7cbae00640 1 -- 172.21.15.133:0/3725792696 >> 172.21.15.133:0/3725792696 conn(0x7f7cb40b2e90 msgr2=0x7f7cb40b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:32.316 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.316+0000 7f7cbae00640 1 -- 172.21.15.133:0/3725792696 shutdown_connections 2024-03-07T06:52:32.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.316+0000 7f7cbae00640 1 -- 172.21.15.133:0/3725792696 wait complete. 2024-03-07T06:52:32.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.318+0000 7f7cbae00640 1 Processor -- start 2024-03-07T06:52:32.317 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.318+0000 7f7cbae00640 1 -- start start 2024-03-07T06:52:32.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.319+0000 7f7cbae00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cb4072d70 0x7f7cb4153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:32.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.319+0000 7f7cbae00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb40cf5d0 0x7f7cb415ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:32.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.319+0000 7f7cbae00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cb415bd30 0x7f7cb415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:32.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.319+0000 7f7cbae00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb406f130 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.319+0000 7f7cbae00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4160240 con 0x7f7cb415bd30 2024-03-07T06:52:32.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.319+0000 7f7cbae00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb415c650 con 0x7f7cb4072d70 2024-03-07T06:52:32.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.319+0000 7f7cb9000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb40cf5d0 0x7f7cb415ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:32.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.319+0000 7f7cb9000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb40cf5d0 0x7f7cb415ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43530/0 (socket says 172.21.15.133:43530) 2024-03-07T06:52:32.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.319+0000 7f7cb9000640 1 -- 172.21.15.133:0/913005204 learned_addr learned my addr 172.21.15.133:0/913005204 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:32.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.319+0000 7f7cb9a00640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cb4072d70 0x7f7cb4153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:32.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.320+0000 7f7cba400640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cb415bd30 0x7f7cb415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:32.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.320+0000 7f7cb9000640 1 -- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cb4072d70 msgr2=0x7f7cb4153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:32.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.320+0000 7f7cb9000640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cb4072d70 0x7f7cb4153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.320+0000 7f7cb9000640 1 -- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cb415bd30 msgr2=0x7f7cb415ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:32.320 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.320+0000 7f7cb9000640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cb415bd30 0x7f7cb415ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.320+0000 7f7cb9000640 1 -- 172.21.15.133:0/913005204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb41f41d0 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.320+0000 7f7cb9a00640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cb4072d70 0x7f7cb4153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:32.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.320+0000 7f7cb9000640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb40cf5d0 0x7f7cb415ca60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7ca002b930 tx=0x7f7ca00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:32.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.320+0000 7f7cb1600640 1 -- 172.21.15.133:0/913005204 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0040070 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.320+0000 7f7cb1600640 1 -- 172.21.15.133:0/913005204 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca0039c10 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.320+0000 7f7cbae00640 1 -- 172.21.15.133:0/913005204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb41f44b0 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.320+0000 7f7cb1600640 1 -- 172.21.15.133:0/913005204 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0035040 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.321+0000 7f7cbae00640 1 -- 172.21.15.133:0/913005204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb41f4a70 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.322+0000 7f7cbae00640 1 -- 172.21.15.133:0/913005204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb4062c90 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.323+0000 7f7cb1600640 1 -- 172.21.15.133:0/913005204 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7ca004b060 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.323+0000 7f7cb1600640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7c9406b4d0 0x7f7c9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:32.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.324+0000 7f7cb1600640 1 -- 172.21.15.133:0/913005204 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7ca00aba60 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.324+0000 7f7cb9a00640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7c9406b4d0 0x7f7c9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:32.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.325+0000 7f7cb9a00640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7c9406b4d0 0x7f7c9406d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800a830 tx=0x7f7ca8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:32.327 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.328+0000 7f7cb1600640 1 -- 172.21.15.133:0/913005204 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7ca007abb0 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.510 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.510+0000 7f7cbae00640 1 -- 172.21.15.133:0/913005204 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7cb40734a0 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.511 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.511+0000 7f7cb1600640 1 -- 172.21.15.133:0/913005204 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7ca007a4b0 con 0x7f7cb40cf5d0 2024-03-07T06:52:32.511 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:32.511 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-03-07T06:52:32.511 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:32.511 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-03-07T06:52:32.511 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-03-07T06:52:32.511 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:32.511 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-03-07T06:52:32.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.514+0000 7f7cbae00640 1 -- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7c9406b4d0 msgr2=0x7f7c9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:32.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.514+0000 7f7cbae00640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7c9406b4d0 0x7f7c9406d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800a830 tx=0x7f7ca8008040 comp rx=0 tx=0).stop 2024-03-07T06:52:32.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.514+0000 7f7cbae00640 1 -- 172.21.15.133:0/913005204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb40cf5d0 msgr2=0x7f7cb415ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:32.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.514+0000 7f7cbae00640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb40cf5d0 0x7f7cb415ca60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7ca002b930 tx=0x7f7ca00046c0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.514+0000 7f7cbae00640 1 -- 172.21.15.133:0/913005204 shutdown_connections 2024-03-07T06:52:32.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.514+0000 7f7cbae00640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7c9406b4d0 0x7f7c9406d990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.514+0000 7f7cbae00640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7cb415bd30 0x7f7cb415ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.514+0000 7f7cbae00640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7cb40cf5d0 0x7f7cb415ca60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.514+0000 7f7cbae00640 1 --2- 172.21.15.133:0/913005204 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7cb4072d70 0x7f7cb4153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.514+0000 7f7cbae00640 1 -- 172.21.15.133:0/913005204 >> 172.21.15.133:0/913005204 conn(0x7f7cb40b2e90 msgr2=0x7f7cb406ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:32.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.514+0000 7f7cbae00640 1 -- 172.21.15.133:0/913005204 shutdown_connections 2024-03-07T06:52:32.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.514+0000 7f7cbae00640 1 -- 172.21.15.133:0/913005204 wait complete. 2024-03-07T06:52:32.533 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-07T06:52:32.533 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-07T06:52:32.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.745+0000 7f5395c00640 1 Processor -- start 2024-03-07T06:52:32.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.745+0000 7f5395c00640 1 -- start start 2024-03-07T06:52:32.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.747+0000 7f5395c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53900cc6f0 0x7f53900b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:32.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.747+0000 7f5395c00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53900b2190 con 0x7f5390070820 2024-03-07T06:52:32.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.747+0000 7f5395c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53900b2430 con 0x7f53900cdf00 2024-03-07T06:52:32.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.747+0000 7f5395c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53900b2710 con 0x7f53900cc6f0 2024-03-07T06:52:32.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.747+0000 7f538f400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5390070820 0x7f539015fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34264/0 (socket says 172.21.15.133:34264) 2024-03-07T06:52:32.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.747+0000 7f538f400640 1 -- 172.21.15.133:0/2173342772 learned_addr learned my addr 172.21.15.133:0/2173342772 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538ea00640 1 --2- 172.21.15.133:0/2173342772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53900cc6f0 0x7f53900b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 -- 172.21.15.133:0/2173342772 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472758396 0 0) 0x7f53900b2190 con 0x7f5390070820 2024-03-07T06:52:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 -- 172.21.15.133:0/2173342772 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5374003520 con 0x7f5390070820 2024-03-07T06:52:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 -- 172.21.15.133:0/2173342772 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 838673212 0 0) 0x7f5374003520 con 0x7f5390070820 2024-03-07T06:52:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 -- 172.21.15.133:0/2173342772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53900cc6f0 msgr2=0x7f53900b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 --2- 172.21.15.133:0/2173342772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53900cc6f0 0x7f53900b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 -- 172.21.15.133:0/2173342772 >> v1:172.21.15.161:6789/0 conn(0x7f53900cdf00 legacy=0x7f539015ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 -- 172.21.15.133:0/2173342772 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53900b29f0 con 0x7f5390070820 2024-03-07T06:52:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 -- 172.21.15.133:0/2173342772 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f5388001670 con 0x7f5390070820 2024-03-07T06:52:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 -- 172.21.15.133:0/2173342772 >> v1:172.21.15.133:6789/0 conn(0x7f5390070820 legacy=0x7f539015fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 --2- 172.21.15.133:0/2173342772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53900cc6f0 0x7f53740039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 --2- 172.21.15.133:0/2173342772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5374003f60 0x7f5374004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:32.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 --2- 172.21.15.133:0/2173342772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5374004d40 0x7f53740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:32.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 -- 172.21.15.133:0/2173342772 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53900b2710 con 0x7f5374003f60 2024-03-07T06:52:32.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 -- 172.21.15.133:0/2173342772 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537400baa0 con 0x7f5374004d40 2024-03-07T06:52:32.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.748+0000 7f538d600640 1 -- 172.21.15.133:0/2173342772 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537400bca0 con 0x7f53900cc6f0 2024-03-07T06:52:32.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.749+0000 7f538e000640 1 --2- 172.21.15.133:0/2173342772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5374003f60 0x7f5374004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:32.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.749+0000 7f538ea00640 1 --2- 172.21.15.133:0/2173342772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53900cc6f0 0x7f53740039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:32.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.749+0000 7f538f400640 1 --2- 172.21.15.133:0/2173342772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5374004d40 0x7f53740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:32.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.749+0000 7f538e000640 1 -- 172.21.15.133:0/2173342772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53900cc6f0 msgr2=0x7f53740039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:32.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.749+0000 7f538e000640 1 --2- 172.21.15.133:0/2173342772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53900cc6f0 0x7f53740039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.749+0000 7f538e000640 1 -- 172.21.15.133:0/2173342772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5374004d40 msgr2=0x7f53740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:32.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.749+0000 7f538e000640 1 --2- 172.21.15.133:0/2173342772 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5374004d40 0x7f53740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.749+0000 7f538e000640 1 -- 172.21.15.133:0/2173342772 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5374003520 con 0x7f5374003f60 2024-03-07T06:52:32.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.750+0000 7f538e000640 1 --2- 172.21.15.133:0/2173342772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5374003f60 0x7f5374004800 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5378001750 tx=0x7f5374005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7e629b9d01da38 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:32.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.750+0000 7f538d600640 1 -- 172.21.15.133:0/2173342772 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378001370 con 0x7f5374003f60 2024-03-07T06:52:32.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.750+0000 7f538d600640 1 -- 172.21.15.133:0/2173342772 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53780052d0 con 0x7f5374003f60 2024-03-07T06:52:32.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.750+0000 7f538d600640 1 -- 172.21.15.133:0/2173342772 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378003830 con 0x7f5374003f60 2024-03-07T06:52:32.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.751+0000 7f5395c00640 1 -- 172.21.15.133:0/2173342772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5374003f60 msgr2=0x7f5374004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:32.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.751+0000 7f5395c00640 1 --2- 172.21.15.133:0/2173342772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5374003f60 0x7f5374004800 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5378001750 tx=0x7f5374005830 comp rx=0 tx=0).stop 2024-03-07T06:52:32.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.751+0000 7f538f400640 1 -- 172.21.15.133:0/2173342772 reap_dead start 2024-03-07T06:52:32.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.751+0000 7f5395c00640 1 -- 172.21.15.133:0/2173342772 shutdown_connections 2024-03-07T06:52:32.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.751+0000 7f5395c00640 1 -- 172.21.15.133:0/2173342772 >> 172.21.15.133:0/2173342772 conn(0x7f5390066eb0 msgr2=0x7f5390067290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:32.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.751+0000 7f5395c00640 1 -- 172.21.15.133:0/2173342772 shutdown_connections 2024-03-07T06:52:32.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.751+0000 7f5395c00640 1 -- 172.21.15.133:0/2173342772 wait complete. 2024-03-07T06:52:32.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.753+0000 7f5395c00640 1 Processor -- start 2024-03-07T06:52:32.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.753+0000 7f5395c00640 1 -- start start 2024-03-07T06:52:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.754+0000 7f5395c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5374003f60 0x7f539014f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.754+0000 7f5395c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f539014faa0 0x7f539014ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.754+0000 7f5395c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53901553a0 0x7f5390155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.754+0000 7f5395c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53900b1c50 con 0x7f5374003f60 2024-03-07T06:52:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.754+0000 7f5395c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539015c460 con 0x7f53901553a0 2024-03-07T06:52:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.754+0000 7f5395c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53900b1dd0 con 0x7f539014faa0 2024-03-07T06:52:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.754+0000 7f538ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5374003f60 0x7f539014f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.754+0000 7f538ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5374003f60 0x7f539014f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43542/0 (socket says 172.21.15.133:43542) 2024-03-07T06:52:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.754+0000 7f538ea00640 1 -- 172.21.15.133:0/2455060882 learned_addr learned my addr 172.21.15.133:0/2455060882 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.754+0000 7f538ea00640 1 -- 172.21.15.133:0/2455060882 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f539014faa0 msgr2=0x7f539014ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.754+0000 7f538ea00640 1 --2- 172.21.15.133:0/2455060882 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f539014faa0 0x7f539014ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.755+0000 7f538ea00640 1 -- 172.21.15.133:0/2455060882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53901553a0 msgr2=0x7f5390155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:32.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.755+0000 7f538ea00640 1 --2- 172.21.15.133:0/2455060882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53901553a0 0x7f5390155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:32.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.755+0000 7f538ea00640 1 -- 172.21.15.133:0/2455060882 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53901f3710 con 0x7f5374003f60 2024-03-07T06:52:32.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.755+0000 7f538ea00640 1 --2- 172.21.15.133:0/2455060882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5374003f60 0x7f539014f4a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5380007f90 tx=0x7f5380011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:32.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.755+0000 7f5386a00640 1 -- 172.21.15.133:0/2455060882 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538001a020 con 0x7f5374003f60 2024-03-07T06:52:32.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.755+0000 7f5386a00640 1 -- 172.21.15.133:0/2455060882 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5380016470 con 0x7f5374003f60 2024-03-07T06:52:32.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.755+0000 7f5386a00640 1 -- 172.21.15.133:0/2455060882 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380016860 con 0x7f5374003f60 2024-03-07T06:52:32.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.755+0000 7f5395c00640 1 -- 172.21.15.133:0/2455060882 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539015a120 con 0x7f5374003f60 2024-03-07T06:52:32.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.755+0000 7f5395c00640 1 -- 172.21.15.133:0/2455060882 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539015a6e0 con 0x7f5374003f60 2024-03-07T06:52:32.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.756+0000 7f5395c00640 1 -- 172.21.15.133:0/2455060882 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5390062c90 con 0x7f5374003f60 2024-03-07T06:52:32.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.757+0000 7f5386a00640 1 -- 172.21.15.133:0/2455060882 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5380005d10 con 0x7f5374003f60 2024-03-07T06:52:32.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.758+0000 7f5386a00640 1 --2- 172.21.15.133:0/2455060882 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f536c06b4d0 0x7f536c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:32.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.758+0000 7f5386a00640 1 -- 172.21.15.133:0/2455060882 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f538008ae70 con 0x7f5374003f60 2024-03-07T06:52:32.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.759+0000 7f538e000640 1 --2- 172.21.15.133:0/2455060882 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f536c06b4d0 0x7f536c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:32.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.760+0000 7f538e000640 1 --2- 172.21.15.133:0/2455060882 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f536c06b4d0 0x7f536c06d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5378033000 tx=0x7f537802f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:32.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.762+0000 7f5386a00640 1 -- 172.21.15.133:0/2455060882 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5380024030 con 0x7f5374003f60 2024-03-07T06:52:32.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:32.957+0000 7f5395c00640 1 -- 172.21.15.133:0/2455060882 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f539006dcb0 con 0x7f5374003f60 2024-03-07T06:52:33.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.708+0000 7f5386a00640 1 -- 172.21.15.133:0/2455060882 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5380059fa0 con 0x7f5374003f60 2024-03-07T06:52:33.708 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-03-07T06:52:33.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.711+0000 7f5395c00640 1 -- 172.21.15.133:0/2455060882 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f536c06b4d0 msgr2=0x7f536c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:33.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.712+0000 7f5395c00640 1 --2- 172.21.15.133:0/2455060882 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f536c06b4d0 0x7f536c06d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5378033000 tx=0x7f537802f000 comp rx=0 tx=0).stop 2024-03-07T06:52:33.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.712+0000 7f5395c00640 1 -- 172.21.15.133:0/2455060882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5374003f60 msgr2=0x7f539014f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:33.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.712+0000 7f5395c00640 1 --2- 172.21.15.133:0/2455060882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5374003f60 0x7f539014f4a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5380007f90 tx=0x7f5380011660 comp rx=0 tx=0).stop 2024-03-07T06:52:33.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.712+0000 7f5395c00640 1 -- 172.21.15.133:0/2455060882 shutdown_connections 2024-03-07T06:52:33.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.712+0000 7f5395c00640 1 --2- 172.21.15.133:0/2455060882 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f536c06b4d0 0x7f536c06d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:33.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.712+0000 7f5395c00640 1 --2- 172.21.15.133:0/2455060882 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53901553a0 0x7f5390155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:33.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.712+0000 7f5395c00640 1 --2- 172.21.15.133:0/2455060882 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f539014faa0 0x7f539014ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:33.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.712+0000 7f5395c00640 1 --2- 172.21.15.133:0/2455060882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5374003f60 0x7f539014f4a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:33.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.712+0000 7f5395c00640 1 -- 172.21.15.133:0/2455060882 >> 172.21.15.133:0/2455060882 conn(0x7f5390066eb0 msgr2=0x7f53900b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:33.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.712+0000 7f5395c00640 1 -- 172.21.15.133:0/2455060882 shutdown_connections 2024-03-07T06:52:33.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.712+0000 7f5395c00640 1 -- 172.21.15.133:0/2455060882 wait complete. 2024-03-07T06:52:33.735 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-07T06:52:33.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.936+0000 7ff2b9800640 1 Processor -- start 2024-03-07T06:52:33.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.937+0000 7ff2b9800640 1 -- start start 2024-03-07T06:52:33.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.938+0000 7ff2b9800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2b40acd90 0x7ff2b415fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:33.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.938+0000 7ff2b9800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff2b40b4670 0x7ff2b40b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:33.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.938+0000 7ff2b9800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b4160480 con 0x7ff2b415c330 2024-03-07T06:52:33.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.938+0000 7ff2b9800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b4160760 con 0x7ff2b40b4670 2024-03-07T06:52:33.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.938+0000 7ff2b9800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b4160a40 con 0x7ff2b40acd90 2024-03-07T06:52:33.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.939+0000 7ff2b3400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff2b40b4670 0x7ff2b40b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:33.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.939+0000 7ff2b3400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff2b40b4670 0x7ff2b40b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43554/0 (socket says 172.21.15.133:43554) 2024-03-07T06:52:33.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.939+0000 7ff2b3400640 1 -- 172.21.15.133:0/575086338 learned_addr learned my addr 172.21.15.133:0/575086338 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:33.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.939+0000 7ff2b2a00640 1 --1- 172.21.15.133:0/575086338 >> v1:172.21.15.133:6789/0 conn(0x7ff2b415c330 0x7ff2b40cca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34272/0 (socket says 172.21.15.133:34272) 2024-03-07T06:52:33.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.939+0000 7ff2b2000640 1 --2- 172.21.15.133:0/575086338 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2b40acd90 0x7ff2b415fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:33.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.939+0000 7ff2b3400640 1 -- 172.21.15.133:0/575086338 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2b40acd90 msgr2=0x7ff2b415fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:33.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.939+0000 7ff2b3400640 1 --2- 172.21.15.133:0/575086338 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2b40acd90 0x7ff2b415fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:33.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.939+0000 7ff2b3400640 1 -- 172.21.15.133:0/575086338 >> v1:172.21.15.133:6789/0 conn(0x7ff2b415c330 legacy=0x7ff2b40cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:33.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.939+0000 7ff2b3400640 1 -- 172.21.15.133:0/575086338 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b40b2ea0 con 0x7ff2b40b4670 2024-03-07T06:52:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.941+0000 7ff2b3400640 1 --2- 172.21.15.133:0/575086338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b40b4670 0x7ff2b40b4b00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009e50 tx=0x7ff2ac02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81339ae94132526c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.941+0000 7ff2b1600640 1 -- 172.21.15.133:0/575086338 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac02cb00 con 0x7ff2b40b4670 2024-03-07T06:52:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.941+0000 7ff2b1600640 1 -- 172.21.15.133:0/575086338 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ac02cca0 con 0x7ff2b40b4670 2024-03-07T06:52:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.941+0000 7ff2b1600640 1 -- 172.21.15.133:0/575086338 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac034610 con 0x7ff2b40b4670 2024-03-07T06:52:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.942+0000 7ff2b9800640 1 -- 172.21.15.133:0/575086338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b40b4670 msgr2=0x7ff2b40b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.942+0000 7ff2b9800640 1 --2- 172.21.15.133:0/575086338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b40b4670 0x7ff2b40b4b00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009e50 tx=0x7ff2ac02bf60 comp rx=0 tx=0).stop 2024-03-07T06:52:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.942+0000 7ff2b9800640 1 -- 172.21.15.133:0/575086338 shutdown_connections 2024-03-07T06:52:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.942+0000 7ff2b9800640 1 --2- 172.21.15.133:0/575086338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b40b4670 0x7ff2b40b4b00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.942+0000 7ff2b9800640 1 --2- 172.21.15.133:0/575086338 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2b40acd90 0x7ff2b415fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.942+0000 7ff2b9800640 1 -- 172.21.15.133:0/575086338 >> 172.21.15.133:0/575086338 conn(0x7ff2b4070400 msgr2=0x7ff2b40707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:33.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.942+0000 7ff2b9800640 1 -- 172.21.15.133:0/575086338 shutdown_connections 2024-03-07T06:52:33.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.942+0000 7ff2b9800640 1 -- 172.21.15.133:0/575086338 wait complete. 2024-03-07T06:52:33.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.944+0000 7ff2b9800640 1 Processor -- start 2024-03-07T06:52:33.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.944+0000 7ff2b9800640 1 -- start start 2024-03-07T06:52:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.945+0000 7ff2b9800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b40acd90 0x7ff2b41506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.945+0000 7ff2b9800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b40b4670 0x7ff2b4157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.945+0000 7ff2b9800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2b415c330 0x7ff2b4157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.945+0000 7ff2b9800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40b3540 con 0x7ff2b40b4670 2024-03-07T06:52:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.945+0000 7ff2b9800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40b33c0 con 0x7ff2b40acd90 2024-03-07T06:52:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.945+0000 7ff2b9800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40b36c0 con 0x7ff2b415c330 2024-03-07T06:52:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.945+0000 7ff2b2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b40b4670 0x7ff2b4157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.945+0000 7ff2b2000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b40b4670 0x7ff2b4157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43568/0 (socket says 172.21.15.133:43568) 2024-03-07T06:52:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.945+0000 7ff2b2000640 1 -- 172.21.15.133:0/3952165198 learned_addr learned my addr 172.21.15.133:0/3952165198 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.946+0000 7ff2b2000640 1 -- 172.21.15.133:0/3952165198 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2b415c330 msgr2=0x7ff2b4157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.946+0000 7ff2b3400640 1 --2- 172.21.15.133:0/3952165198 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2b415c330 0x7ff2b4157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.946+0000 7ff2b2000640 1 --2- 172.21.15.133:0/3952165198 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2b415c330 0x7ff2b4157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.946+0000 7ff2b2000640 1 -- 172.21.15.133:0/3952165198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b40acd90 msgr2=0x7ff2b41506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.946+0000 7ff2b2000640 1 --2- 172.21.15.133:0/3952165198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b40acd90 0x7ff2b41506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.946+0000 7ff2b2000640 1 -- 172.21.15.133:0/3952165198 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b4150c10 con 0x7ff2b40b4670 2024-03-07T06:52:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.946+0000 7ff2b2000640 1 --2- 172.21.15.133:0/3952165198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b40b4670 0x7ff2b4157a00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000e9e0 tx=0x7ff2a000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.946+0000 7ff2aaa00640 1 -- 172.21.15.133:0/3952165198 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0017070 con 0x7ff2b40b4670 2024-03-07T06:52:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.946+0000 7ff2aaa00640 1 -- 172.21.15.133:0/3952165198 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a000c9d0 con 0x7ff2b40b4670 2024-03-07T06:52:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.946+0000 7ff2aaa00640 1 -- 172.21.15.133:0/3952165198 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a000cdc0 con 0x7ff2b40b4670 2024-03-07T06:52:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.946+0000 7ff2b9800640 1 -- 172.21.15.133:0/3952165198 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b4149fe0 con 0x7ff2b40b4670 2024-03-07T06:52:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.947+0000 7ff2b9800640 1 -- 172.21.15.133:0/3952165198 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b414a5a0 con 0x7ff2b40b4670 2024-03-07T06:52:33.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.948+0000 7ff2b9800640 1 -- 172.21.15.133:0/3952165198 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b4062c90 con 0x7ff2b40b4670 2024-03-07T06:52:33.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.949+0000 7ff2aaa00640 1 -- 172.21.15.133:0/3952165198 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff2a001a070 con 0x7ff2b40b4670 2024-03-07T06:52:33.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.950+0000 7ff2aaa00640 1 --2- 172.21.15.133:0/3952165198 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff29406b4d0 0x7ff29406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:33.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.950+0000 7ff2aaa00640 1 -- 172.21.15.133:0/3952165198 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff2a008a720 con 0x7ff2b40b4670 2024-03-07T06:52:33.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.951+0000 7ff2b2a00640 1 --2- 172.21.15.133:0/3952165198 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff29406b4d0 0x7ff29406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:33.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.952+0000 7ff2b2a00640 1 --2- 172.21.15.133:0/3952165198 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff29406b4d0 0x7ff29406d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4000f60 tx=0x7ff2a4007a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:33.953 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:33.954+0000 7ff2aaa00640 1 -- 172.21.15.133:0/3952165198 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff2a00597f0 con 0x7ff2b40b4670 2024-03-07T06:52:34.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.150+0000 7ff2b9800640 1 -- 172.21.15.133:0/3952165198 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff2b4068340 con 0x7ff2b40b4670 2024-03-07T06:52:34.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.713+0000 7ff2aaa00640 1 -- 172.21.15.133:0/3952165198 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff2a00590f0 con 0x7ff2b40b4670 2024-03-07T06:52:34.713 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-03-07T06:52:34.717 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.717+0000 7ff2b9800640 1 -- 172.21.15.133:0/3952165198 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff29406b4d0 msgr2=0x7ff29406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:34.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.717+0000 7ff2b9800640 1 --2- 172.21.15.133:0/3952165198 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff29406b4d0 0x7ff29406d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4000f60 tx=0x7ff2a4007a00 comp rx=0 tx=0).stop 2024-03-07T06:52:34.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.718+0000 7ff2b9800640 1 -- 172.21.15.133:0/3952165198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b40b4670 msgr2=0x7ff2b4157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:34.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.718+0000 7ff2b9800640 1 --2- 172.21.15.133:0/3952165198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b40b4670 0x7ff2b4157a00 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000e9e0 tx=0x7ff2a000eea0 comp rx=0 tx=0).stop 2024-03-07T06:52:34.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.718+0000 7ff2b9800640 1 -- 172.21.15.133:0/3952165198 shutdown_connections 2024-03-07T06:52:34.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.718+0000 7ff2b9800640 1 --2- 172.21.15.133:0/3952165198 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff2b415c330 0x7ff2b4157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:34.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.718+0000 7ff2b9800640 1 --2- 172.21.15.133:0/3952165198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2b40b4670 0x7ff2b4157a00 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:34.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.718+0000 7ff2b9800640 1 --2- 172.21.15.133:0/3952165198 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff29406b4d0 0x7ff29406d990 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4000f60 tx=0x7ff2a4007a00 comp rx=0 tx=0).stop 2024-03-07T06:52:34.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.718+0000 7ff2b9800640 1 --2- 172.21.15.133:0/3952165198 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff2b40acd90 0x7ff2b41506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:34.718 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.718+0000 7ff2b9800640 1 -- 172.21.15.133:0/3952165198 >> 172.21.15.133:0/3952165198 conn(0x7ff2b4070400 msgr2=0x7ff2b415fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:34.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.718+0000 7ff2b9800640 1 -- 172.21.15.133:0/3952165198 shutdown_connections 2024-03-07T06:52:34.719 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.718+0000 7ff2b9800640 1 -- 172.21.15.133:0/3952165198 wait complete. 2024-03-07T06:52:34.751 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-07T06:52:34.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.929+0000 7f4329c00640 1 Processor -- start 2024-03-07T06:52:34.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.929+0000 7f4329c00640 1 -- start start 2024-03-07T06:52:34.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.931+0000 7f4329c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4324145670 0x7f43240cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:34.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.931+0000 7f4329c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f43240cdbf0 0x7f43240719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:34.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.931+0000 7f4329c00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432406e150 con 0x7f432415cf90 2024-03-07T06:52:34.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.931+0000 7f4329c00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432406ed60 con 0x7f43240cdbf0 2024-03-07T06:52:34.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.931+0000 7f4329c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432406e430 con 0x7f4324145670 2024-03-07T06:52:34.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.931+0000 7f4323400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f43240cdbf0 0x7f43240719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:34.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.931+0000 7f4323400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f43240cdbf0 0x7f43240719a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43578/0 (socket says 172.21.15.133:43578) 2024-03-07T06:52:34.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.931+0000 7f4323400640 1 -- 172.21.15.133:0/887630210 learned_addr learned my addr 172.21.15.133:0/887630210 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:34.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.931+0000 7f4322a00640 1 --2- 172.21.15.133:0/887630210 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4324145670 0x7f43240cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:34.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.931+0000 7f4323400640 1 -- 172.21.15.133:0/887630210 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4324145670 msgr2=0x7f43240cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:34.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.931+0000 7f4323400640 1 --2- 172.21.15.133:0/887630210 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4324145670 0x7f43240cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:34.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.931+0000 7f4323400640 1 -- 172.21.15.133:0/887630210 >> v1:172.21.15.133:6789/0 conn(0x7f432415cf90 legacy=0x7f43240cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:34.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.931+0000 7f4323400640 1 -- 172.21.15.133:0/887630210 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432406e710 con 0x7f43240cdbf0 2024-03-07T06:52:34.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.931+0000 7f4321600640 1 -- 172.21.15.133:0/887630210 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818889142 0 0) 0x7f432406e150 con 0x7f432415cf90 2024-03-07T06:52:34.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.932+0000 7f4322a00640 1 --2- 172.21.15.133:0/887630210 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4324145670 0x7f43240cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:34.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.933+0000 7f4323400640 1 --2- 172.21.15.133:0/887630210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43240cdbf0 0x7f43240719a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f431c009e50 tx=0x7f431c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad1fc8614edeaac server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:34.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.933+0000 7f4321600640 1 -- 172.21.15.133:0/887630210 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c02ca70 con 0x7f43240cdbf0 2024-03-07T06:52:34.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.933+0000 7f4321600640 1 -- 172.21.15.133:0/887630210 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c02cc10 con 0x7f43240cdbf0 2024-03-07T06:52:34.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.933+0000 7f4321600640 1 -- 172.21.15.133:0/887630210 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0343f0 con 0x7f43240cdbf0 2024-03-07T06:52:34.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.933+0000 7f4329c00640 1 -- 172.21.15.133:0/887630210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43240cdbf0 msgr2=0x7f43240719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:34.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.933+0000 7f4329c00640 1 --2- 172.21.15.133:0/887630210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43240cdbf0 0x7f43240719a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f431c009e50 tx=0x7f431c02bed0 comp rx=0 tx=0).stop 2024-03-07T06:52:34.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.933+0000 7f4329c00640 1 -- 172.21.15.133:0/887630210 shutdown_connections 2024-03-07T06:52:34.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.933+0000 7f4329c00640 1 --2- 172.21.15.133:0/887630210 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43240cdbf0 0x7f43240719a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:34.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.933+0000 7f4329c00640 1 --2- 172.21.15.133:0/887630210 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4324145670 0x7f43240cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:34.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.933+0000 7f4329c00640 1 -- 172.21.15.133:0/887630210 >> 172.21.15.133:0/887630210 conn(0x7f43240b1d30 msgr2=0x7f43240b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:34.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.934+0000 7f4329c00640 1 -- 172.21.15.133:0/887630210 shutdown_connections 2024-03-07T06:52:34.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.934+0000 7f4329c00640 1 -- 172.21.15.133:0/887630210 wait complete. 2024-03-07T06:52:34.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.935+0000 7f4329c00640 1 Processor -- start 2024-03-07T06:52:34.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.935+0000 7f4329c00640 1 -- start start 2024-03-07T06:52:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.936+0000 7f4329c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43240cdbf0 0x7f4324158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:34.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.936+0000 7f4329c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4324145670 0x7f43240c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.936+0000 7f4329c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f432415cf90 0x7f43240ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.936+0000 7f4329c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4324145a50 con 0x7f4324145670 2024-03-07T06:52:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.936+0000 7f4329c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432415d3f0 con 0x7f43240cdbf0 2024-03-07T06:52:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.936+0000 7f4329c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4324145bd0 con 0x7f432415cf90 2024-03-07T06:52:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.936+0000 7f4322000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4324145670 0x7f43240c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.936+0000 7f4322000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4324145670 0x7f43240c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43584/0 (socket says 172.21.15.133:43584) 2024-03-07T06:52:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.936+0000 7f4322000640 1 -- 172.21.15.133:0/314460715 learned_addr learned my addr 172.21.15.133:0/314460715 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.937+0000 7f4322000640 1 -- 172.21.15.133:0/314460715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f432415cf90 msgr2=0x7f43240ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.937+0000 7f4322000640 1 --2- 172.21.15.133:0/314460715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f432415cf90 0x7f43240ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.937+0000 7f4322000640 1 -- 172.21.15.133:0/314460715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43240cdbf0 msgr2=0x7f4324158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.937+0000 7f4322000640 1 --2- 172.21.15.133:0/314460715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43240cdbf0 0x7f4324158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.937+0000 7f4322000640 1 -- 172.21.15.133:0/314460715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4324158a90 con 0x7f4324145670 2024-03-07T06:52:34.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.937+0000 7f4322000640 1 --2- 172.21.15.133:0/314460715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4324145670 0x7f43240c91d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c4e0 tx=0x7f430c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.937+0000 7f431aa00640 1 -- 172.21.15.133:0/314460715 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c017070 con 0x7f4324145670 2024-03-07T06:52:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.937+0000 7f431aa00640 1 -- 172.21.15.133:0/314460715 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430c0035f0 con 0x7f4324145670 2024-03-07T06:52:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.937+0000 7f431aa00640 1 -- 172.21.15.133:0/314460715 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430c0039e0 con 0x7f4324145670 2024-03-07T06:52:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.937+0000 7f4329c00640 1 -- 172.21.15.133:0/314460715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43240cba90 con 0x7f4324145670 2024-03-07T06:52:34.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.937+0000 7f4329c00640 1 -- 172.21.15.133:0/314460715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43240cc050 con 0x7f4324145670 2024-03-07T06:52:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.939+0000 7f4329c00640 1 -- 172.21.15.133:0/314460715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4324062c90 con 0x7f4324145670 2024-03-07T06:52:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.939+0000 7f431aa00640 1 -- 172.21.15.133:0/314460715 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f430c01a070 con 0x7f4324145670 2024-03-07T06:52:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.940+0000 7f431aa00640 1 --2- 172.21.15.133:0/314460715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f430006b4d0 0x7f430006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.940+0000 7f431aa00640 1 -- 172.21.15.133:0/314460715 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f430c089430 con 0x7f4324145670 2024-03-07T06:52:34.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.941+0000 7f4322a00640 1 --2- 172.21.15.133:0/314460715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f430006b4d0 0x7f430006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:34.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.942+0000 7f4322a00640 1 --2- 172.21.15.133:0/314460715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f430006b4d0 0x7f430006d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4314006fd0 tx=0x7f4314008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:34.945 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:34.945+0000 7f431aa00640 1 -- 172.21.15.133:0/314460715 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f430c057c20 con 0x7f4324145670 2024-03-07T06:52:35.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.179+0000 7f4329c00640 1 -- 172.21.15.133:0/314460715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f43240b3aa0 con 0x7f4324145670 2024-03-07T06:52:35.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.764+0000 7f431aa00640 1 -- 172.21.15.133:0/314460715 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f430c05bcc0 con 0x7f4324145670 2024-03-07T06:52:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 4 and data pool 5 2024-03-07T06:52:35.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.771+0000 7f4329c00640 1 -- 172.21.15.133:0/314460715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f430006b4d0 msgr2=0x7f430006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:35.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.771+0000 7f4329c00640 1 --2- 172.21.15.133:0/314460715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f430006b4d0 0x7f430006d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4314006fd0 tx=0x7f4314008040 comp rx=0 tx=0).stop 2024-03-07T06:52:35.771 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.771+0000 7f4329c00640 1 -- 172.21.15.133:0/314460715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4324145670 msgr2=0x7f43240c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:35.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.771+0000 7f4329c00640 1 --2- 172.21.15.133:0/314460715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4324145670 0x7f43240c91d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f430c00c4e0 tx=0x7f430c00c9a0 comp rx=0 tx=0).stop 2024-03-07T06:52:35.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.772+0000 7f4329c00640 1 -- 172.21.15.133:0/314460715 shutdown_connections 2024-03-07T06:52:35.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.772+0000 7f4329c00640 1 --2- 172.21.15.133:0/314460715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f432415cf90 0x7f43240ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:35.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.772+0000 7f4329c00640 1 --2- 172.21.15.133:0/314460715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4324145670 0x7f43240c91d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:35.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.772+0000 7f4329c00640 1 --2- 172.21.15.133:0/314460715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f430006b4d0 0x7f430006d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:35.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.772+0000 7f4329c00640 1 --2- 172.21.15.133:0/314460715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f43240cdbf0 0x7f4324158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:35.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.772+0000 7f4329c00640 1 -- 172.21.15.133:0/314460715 >> 172.21.15.133:0/314460715 conn(0x7f43240b1d30 msgr2=0x7f43240b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:35.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.774+0000 7f4329c00640 1 -- 172.21.15.133:0/314460715 shutdown_connections 2024-03-07T06:52:35.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.774+0000 7f4329c00640 1 -- 172.21.15.133:0/314460715 wait complete. 2024-03-07T06:52:35.807 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:52:35.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.976+0000 7f55e7200640 1 Processor -- start 2024-03-07T06:52:35.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.976+0000 7f55e7200640 1 -- start start 2024-03-07T06:52:35.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.977+0000 7f55e7200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55e015fbb0 0x7f55e00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:35.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.977+0000 7f55e7200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e00731b0 con 0x7f55e00cdee0 2024-03-07T06:52:35.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.977+0000 7f55e7200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e0072150 con 0x7f55e0072950 2024-03-07T06:52:35.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.977+0000 7f55e7200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e0072f60 con 0x7f55e015fbb0 2024-03-07T06:52:35.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.978+0000 7f55e5200640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f55e00cdee0 0x7f55e015c550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34282/0 (socket says 172.21.15.133:34282) 2024-03-07T06:52:35.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.978+0000 7f55e5200640 1 -- 172.21.15.133:0/2660800164 learned_addr learned my addr 172.21.15.133:0/2660800164 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:35.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.978+0000 7f55dfe00640 1 --2- 172.21.15.133:0/2660800164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55e015fbb0 0x7f55e00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:35.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.978+0000 7f55dea00640 1 -- 172.21.15.133:0/2660800164 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651777639 0 0) 0x7f55e00731b0 con 0x7f55e00cdee0 2024-03-07T06:52:35.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.978+0000 7f55dea00640 1 -- 172.21.15.133:0/2660800164 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55c4003520 con 0x7f55e00cdee0 2024-03-07T06:52:35.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.978+0000 7f55dea00640 1 -- 172.21.15.133:0/2660800164 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3655423307 0 0) 0x7f55c4003520 con 0x7f55e00cdee0 2024-03-07T06:52:35.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.978+0000 7f55dea00640 1 -- 172.21.15.133:0/2660800164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55e015fbb0 msgr2=0x7f55e00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:35.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.978+0000 7f55dea00640 1 --2- 172.21.15.133:0/2660800164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55e015fbb0 0x7f55e00cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.978+0000 7f55dea00640 1 -- 172.21.15.133:0/2660800164 >> v1:172.21.15.161:6789/0 conn(0x7f55e0072950 legacy=0x7f55e015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.978+0000 7f55dea00640 1 -- 172.21.15.133:0/2660800164 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e0072430 con 0x7f55e00cdee0 2024-03-07T06:52:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.978+0000 7f55dea00640 1 -- 172.21.15.133:0/2660800164 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f55d80019f0 con 0x7f55e00cdee0 2024-03-07T06:52:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.978+0000 7f55dea00640 1 -- 172.21.15.133:0/2660800164 >> v1:172.21.15.133:6789/0 conn(0x7f55e00cdee0 legacy=0x7f55e015c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.978+0000 7f55dea00640 1 --2- 172.21.15.133:0/2660800164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c40039e0 0x7f55c4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.979+0000 7f55dea00640 1 --2- 172.21.15.133:0/2660800164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c4004800 0x7f55c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.979+0000 7f55dea00640 1 --2- 172.21.15.133:0/2660800164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55e015fbb0 0x7f55c4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.979+0000 7f55dea00640 1 -- 172.21.15.133:0/2660800164 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e0072f60 con 0x7f55c40039e0 2024-03-07T06:52:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.979+0000 7f55dea00640 1 -- 172.21.15.133:0/2660800164 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c400baf0 con 0x7f55c4004800 2024-03-07T06:52:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.979+0000 7f55dea00640 1 -- 172.21.15.133:0/2660800164 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c400bcf0 con 0x7f55e015fbb0 2024-03-07T06:52:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.979+0000 7f55dfe00640 1 --2- 172.21.15.133:0/2660800164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c40039e0 0x7f55c4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.979+0000 7f55dfe00640 1 -- 172.21.15.133:0/2660800164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55e015fbb0 msgr2=0x7f55c4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.979+0000 7f55dfe00640 1 --2- 172.21.15.133:0/2660800164 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55e015fbb0 0x7f55c4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:35.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.979+0000 7f55dfe00640 1 -- 172.21.15.133:0/2660800164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c4004800 msgr2=0x7f55c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.979+0000 7f55df400640 1 --2- 172.21.15.133:0/2660800164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c4004800 0x7f55c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.979+0000 7f55dfe00640 1 --2- 172.21.15.133:0/2660800164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c4004800 0x7f55c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.979+0000 7f55dfe00640 1 -- 172.21.15.133:0/2660800164 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c4003520 con 0x7f55c40039e0 2024-03-07T06:52:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.979+0000 7f55df400640 1 --2- 172.21.15.133:0/2660800164 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c4004800 0x7f55c4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.980+0000 7f55dfe00640 1 --2- 172.21.15.133:0/2660800164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c40039e0 0x7f55c4003e00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009ac0 tx=0x7f55c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644e1f56f8a04348 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.980+0000 7f55dea00640 1 -- 172.21.15.133:0/2660800164 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d0039070 con 0x7f55c40039e0 2024-03-07T06:52:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.981+0000 7f55dea00640 1 -- 172.21.15.133:0/2660800164 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d002c710 con 0x7f55c40039e0 2024-03-07T06:52:35.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.981+0000 7f55dea00640 1 -- 172.21.15.133:0/2660800164 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d0009690 con 0x7f55c40039e0 2024-03-07T06:52:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.981+0000 7f55e7200640 1 -- 172.21.15.133:0/2660800164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c40039e0 msgr2=0x7f55c4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.981+0000 7f55e7200640 1 --2- 172.21.15.133:0/2660800164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55c40039e0 0x7f55c4003e00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009ac0 tx=0x7f55c4005830 comp rx=0 tx=0).stop 2024-03-07T06:52:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.981+0000 7f55e5200640 1 -- 172.21.15.133:0/2660800164 reap_dead start 2024-03-07T06:52:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.981+0000 7f55e7200640 1 -- 172.21.15.133:0/2660800164 shutdown_connections 2024-03-07T06:52:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.981+0000 7f55e7200640 1 -- 172.21.15.133:0/2660800164 >> 172.21.15.133:0/2660800164 conn(0x7f55e00b3120 msgr2=0x7f55e00b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.981+0000 7f55e7200640 1 -- 172.21.15.133:0/2660800164 shutdown_connections 2024-03-07T06:52:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.981+0000 7f55e7200640 1 -- 172.21.15.133:0/2660800164 wait complete. 2024-03-07T06:52:35.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.983+0000 7f55e7200640 1 Processor -- start 2024-03-07T06:52:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.983+0000 7f55e7200640 1 -- start start 2024-03-07T06:52:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.984+0000 7f55e7200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c40039e0 0x7f55e015dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.984+0000 7f55e7200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55e00c94a0 0x7f55e01f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.984+0000 7f55e7200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55e014b900 0x7f55e00afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.984+0000 7f55e7200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e0160980 con 0x7f55e014b900 2024-03-07T06:52:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.984+0000 7f55e7200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e0073490 con 0x7f55c40039e0 2024-03-07T06:52:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.984+0000 7f55e7200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e00b1a00 con 0x7f55e00c94a0 2024-03-07T06:52:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.984+0000 7f55e5200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55e014b900 0x7f55e00afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.984+0000 7f55e5200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55e014b900 0x7f55e00afc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43612/0 (socket says 172.21.15.133:43612) 2024-03-07T06:52:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.984+0000 7f55e5200640 1 -- 172.21.15.133:0/3516166770 learned_addr learned my addr 172.21.15.133:0/3516166770 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.985+0000 7f55dfe00640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c40039e0 0x7f55e015dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.985+0000 7f55df400640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55e00c94a0 0x7f55e01f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:35.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.985+0000 7f55e5200640 1 -- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55e00c94a0 msgr2=0x7f55e01f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:35.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.985+0000 7f55e5200640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55e00c94a0 0x7f55e01f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:35.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.985+0000 7f55e5200640 1 -- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c40039e0 msgr2=0x7f55e015dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:35.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.985+0000 7f55e5200640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c40039e0 0x7f55e015dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:35.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.985+0000 7f55e5200640 1 -- 172.21.15.133:0/3516166770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e01f4b80 con 0x7f55e014b900 2024-03-07T06:52:35.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.985+0000 7f55dfe00640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c40039e0 0x7f55e015dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:35.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.985+0000 7f55e5200640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55e014b900 0x7f55e00afc40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f55e0068cd0 tx=0x7f55d800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:35.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.985+0000 7f55d7e00640 1 -- 172.21.15.133:0/3516166770 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d8004d60 con 0x7f55e014b900 2024-03-07T06:52:35.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.985+0000 7f55d7e00640 1 -- 172.21.15.133:0/3516166770 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d800a610 con 0x7f55e014b900 2024-03-07T06:52:35.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.985+0000 7f55d7e00640 1 -- 172.21.15.133:0/3516166770 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d800aa20 con 0x7f55e014b900 2024-03-07T06:52:35.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.985+0000 7f55e7200640 1 -- 172.21.15.133:0/3516166770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e00c9960 con 0x7f55e014b900 2024-03-07T06:52:35.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.985+0000 7f55e7200640 1 -- 172.21.15.133:0/3516166770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e015d7f0 con 0x7f55e014b900 2024-03-07T06:52:35.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.987+0000 7f55e7200640 1 -- 172.21.15.133:0/3516166770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e0062c90 con 0x7f55e014b900 2024-03-07T06:52:35.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.988+0000 7f55d7e00640 1 -- 172.21.15.133:0/3516166770 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f55d800abc0 con 0x7f55e014b900 2024-03-07T06:52:35.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.989+0000 7f55d7e00640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f55c006b520 0x7f55c006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:35.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.989+0000 7f55d7e00640 1 -- 172.21.15.133:0/3516166770 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55d8089a20 con 0x7f55e014b900 2024-03-07T06:52:35.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.989+0000 7f55dfe00640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f55c006b520 0x7f55c006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:35.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.990+0000 7f55dfe00640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f55c006b520 0x7f55c006d9e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f55d0036000 tx=0x7f55d0030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:35.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:35.993+0000 7f55d7e00640 1 -- 172.21.15.133:0/3516166770 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f55d8058880 con 0x7f55e014b900 2024-03-07T06:52:36.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.185+0000 7f55e7200640 1 -- 172.21.15.133:0/3516166770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55e0072d40 con 0x7f55e014b900 2024-03-07T06:52:36.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.187+0000 7f55d7e00640 1 -- 172.21.15.133:0/3516166770 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16730 (secure 0 0 0) 0x7f55d8058180 con 0x7f55e014b900 2024-03-07T06:52:36.187 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:36.187 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":34,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:52:35.181797+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:52:33.007428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-07T06:52:34.201590+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:52:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.190+0000 7f55e7200640 1 -- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f55c006b520 msgr2=0x7f55c006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.190+0000 7f55e7200640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f55c006b520 0x7f55c006d9e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f55d0036000 tx=0x7f55d0030000 comp rx=0 tx=0).stop 2024-03-07T06:52:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.190+0000 7f55e7200640 1 -- 172.21.15.133:0/3516166770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55e014b900 msgr2=0x7f55e00afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.190+0000 7f55e7200640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55e014b900 0x7f55e00afc40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f55e0068cd0 tx=0x7f55d800cb90 comp rx=0 tx=0).stop 2024-03-07T06:52:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.190+0000 7f55e7200640 1 -- 172.21.15.133:0/3516166770 shutdown_connections 2024-03-07T06:52:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.190+0000 7f55e7200640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f55e014b900 0x7f55e00afc40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.190+0000 7f55e7200640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55e00c94a0 0x7f55e01f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.190+0000 7f55e7200640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f55c006b520 0x7f55c006d9e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.190+0000 7f55e7200640 1 --2- 172.21.15.133:0/3516166770 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55c40039e0 0x7f55e015dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.190+0000 7f55e7200640 1 -- 172.21.15.133:0/3516166770 >> 172.21.15.133:0/3516166770 conn(0x7f55e00b3120 msgr2=0x7f55e00adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.190+0000 7f55e7200640 1 -- 172.21.15.133:0/3516166770 shutdown_connections 2024-03-07T06:52:36.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.190+0000 7f55e7200640 1 -- 172.21.15.133:0/3516166770 wait complete. 2024-03-07T06:52:36.209 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:52:36.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.414+0000 7f78d0200640 1 Processor -- start 2024-03-07T06:52:36.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.414+0000 7f78d0200640 1 -- start start 2024-03-07T06:52:36.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.415+0000 7f78d0200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f78c80cf7d0 0x7f78c80cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:36.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.415+0000 7f78d0200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c80731b0 con 0x7f78c815fbb0 2024-03-07T06:52:36.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.415+0000 7f78d0200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c8072150 con 0x7f78c8072950 2024-03-07T06:52:36.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.415+0000 7f78d0200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c8072f60 con 0x7f78c80cf7d0 2024-03-07T06:52:36.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.415+0000 7f78ce200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f78c80cf7d0 0x7f78c80cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:36.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.415+0000 7f78ce200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f78c80cf7d0 0x7f78c80cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43618/0 (socket says 172.21.15.133:43618) 2024-03-07T06:52:36.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.415+0000 7f78ce200640 1 -- 172.21.15.133:0/909568277 learned_addr learned my addr 172.21.15.133:0/909568277 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:36.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.416+0000 7f78ce200640 1 -- 172.21.15.133:0/909568277 >> v1:172.21.15.161:6789/0 conn(0x7f78c8072950 legacy=0x7f78c815c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:36.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.416+0000 7f78ce200640 1 -- 172.21.15.133:0/909568277 >> v1:172.21.15.133:6789/0 conn(0x7f78c815fbb0 legacy=0x7f78c815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:36.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.416+0000 7f78ce200640 1 -- 172.21.15.133:0/909568277 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c8072430 con 0x7f78c80cf7d0 2024-03-07T06:52:36.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.417+0000 7f78ce200640 1 --2- 172.21.15.133:0/909568277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c80cf7d0 0x7f78c80cdee0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f78c80730e0 tx=0x7f78c002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decd87c3940236 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:36.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.417+0000 7f78c7e00640 1 -- 172.21.15.133:0/909568277 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c002ca60 con 0x7f78c80cf7d0 2024-03-07T06:52:36.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.417+0000 7f78c7e00640 1 -- 172.21.15.133:0/909568277 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c002cc00 con 0x7f78c80cf7d0 2024-03-07T06:52:36.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.417+0000 7f78c7e00640 1 -- 172.21.15.133:0/909568277 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c00343f0 con 0x7f78c80cf7d0 2024-03-07T06:52:36.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.418+0000 7f78d0200640 1 -- 172.21.15.133:0/909568277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c80cf7d0 msgr2=0x7f78c80cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:36.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.418+0000 7f78d0200640 1 --2- 172.21.15.133:0/909568277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c80cf7d0 0x7f78c80cdee0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f78c80730e0 tx=0x7f78c002bec0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.418+0000 7f78d0200640 1 -- 172.21.15.133:0/909568277 shutdown_connections 2024-03-07T06:52:36.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.418+0000 7f78d0200640 1 --2- 172.21.15.133:0/909568277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c80cf7d0 0x7f78c80cdee0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.418+0000 7f78d0200640 1 -- 172.21.15.133:0/909568277 >> 172.21.15.133:0/909568277 conn(0x7f78c80b3120 msgr2=0x7f78c80b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:36.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.418+0000 7f78d0200640 1 -- 172.21.15.133:0/909568277 shutdown_connections 2024-03-07T06:52:36.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.418+0000 7f78d0200640 1 -- 172.21.15.133:0/909568277 wait complete. 2024-03-07T06:52:36.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.420+0000 7f78d0200640 1 Processor -- start 2024-03-07T06:52:36.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.420+0000 7f78d0200640 1 -- start start 2024-03-07T06:52:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78d0200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78c80cf7d0 0x7f78c8153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78d0200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c815fbb0 0x7f78c81e7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78d0200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78c81e7ed0 0x7f78c80afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78d0200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c8073490 con 0x7f78c815fbb0 2024-03-07T06:52:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78d0200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c80cfc00 con 0x7f78c81e7ed0 2024-03-07T06:52:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78d0200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c80b1a00 con 0x7f78c80cf7d0 2024-03-07T06:52:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78cce00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c815fbb0 0x7f78c81e7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78cce00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c815fbb0 0x7f78c81e7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43622/0 (socket says 172.21.15.133:43622) 2024-03-07T06:52:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78cce00640 1 -- 172.21.15.133:0/3675909986 learned_addr learned my addr 172.21.15.133:0/3675909986 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78ce200640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78c81e7ed0 0x7f78c80afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:36.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78cd800640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78c80cf7d0 0x7f78c8153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:36.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78cce00640 1 -- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78c80cf7d0 msgr2=0x7f78c8153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:36.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78cce00640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78c80cf7d0 0x7f78c8153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78cce00640 1 -- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78c81e7ed0 msgr2=0x7f78c80afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:36.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78cce00640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78c81e7ed0 0x7f78c80afc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78cce00640 1 -- 172.21.15.133:0/3675909986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c8153c90 con 0x7f78c815fbb0 2024-03-07T06:52:36.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.421+0000 7f78cd800640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78c80cf7d0 0x7f78c8153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:36.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.422+0000 7f78cce00640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c815fbb0 0x7f78c81e7990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f78b400b3c0 tx=0x7f78b400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:36.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.422+0000 7f78c5600640 1 -- 172.21.15.133:0/3675909986 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b4011830 con 0x7f78c815fbb0 2024-03-07T06:52:36.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.422+0000 7f78c5600640 1 -- 172.21.15.133:0/3675909986 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b4016070 con 0x7f78c815fbb0 2024-03-07T06:52:36.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.422+0000 7f78c5600640 1 -- 172.21.15.133:0/3675909986 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b4011c10 con 0x7f78c815fbb0 2024-03-07T06:52:36.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.422+0000 7f78d0200640 1 -- 172.21.15.133:0/3675909986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c81e8350 con 0x7f78c815fbb0 2024-03-07T06:52:36.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.422+0000 7f78d0200640 1 -- 172.21.15.133:0/3675909986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c81580a0 con 0x7f78c815fbb0 2024-03-07T06:52:36.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.423+0000 7f78d0200640 1 -- 172.21.15.133:0/3675909986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c8157bd0 con 0x7f78c815fbb0 2024-03-07T06:52:36.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.424+0000 7f78c5600640 1 -- 172.21.15.133:0/3675909986 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f78b401b460 con 0x7f78c815fbb0 2024-03-07T06:52:36.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.425+0000 7f78c5600640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f78a806b4d0 0x7f78a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:36.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.425+0000 7f78c5600640 1 -- 172.21.15.133:0/3675909986 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78b4089d50 con 0x7f78c815fbb0 2024-03-07T06:52:36.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.426+0000 7f78cd800640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f78a806b4d0 0x7f78a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:36.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.427+0000 7f78cd800640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f78a806b4d0 0x7f78a806d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f78bc001680 tx=0x7f78bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:36.429 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.430+0000 7f78c5600640 1 -- 172.21.15.133:0/3675909986 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f78b4020020 con 0x7f78c815fbb0 2024-03-07T06:52:36.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.629+0000 7f78d0200640 1 -- 172.21.15.133:0/3675909986 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78c815e260 con 0x7f78c815fbb0 2024-03-07T06:52:36.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.630+0000 7f78c5600640 1 -- 172.21.15.133:0/3675909986 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16730 (secure 0 0 0) 0x7f78b4058b30 con 0x7f78c815fbb0 2024-03-07T06:52:36.631 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:36.631 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":34,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:52:35.181797+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:52:33.007428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-07T06:52:34.201590+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:52:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.633+0000 7f78d0200640 1 -- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f78a806b4d0 msgr2=0x7f78a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.633+0000 7f78d0200640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f78a806b4d0 0x7f78a806d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f78bc001680 tx=0x7f78bc009040 comp rx=0 tx=0).stop 2024-03-07T06:52:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.633+0000 7f78d0200640 1 -- 172.21.15.133:0/3675909986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c815fbb0 msgr2=0x7f78c81e7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.633+0000 7f78d0200640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c815fbb0 0x7f78c81e7990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f78b400b3c0 tx=0x7f78b400b880 comp rx=0 tx=0).stop 2024-03-07T06:52:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.633+0000 7f78d0200640 1 -- 172.21.15.133:0/3675909986 shutdown_connections 2024-03-07T06:52:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.633+0000 7f78d0200640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f78a806b4d0 0x7f78a806d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.633+0000 7f78d0200640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78c81e7ed0 0x7f78c80afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.633+0000 7f78d0200640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78c815fbb0 0x7f78c81e7990 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.633+0000 7f78d0200640 1 --2- 172.21.15.133:0/3675909986 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78c80cf7d0 0x7f78c8153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.633+0000 7f78d0200640 1 -- 172.21.15.133:0/3675909986 >> 172.21.15.133:0/3675909986 conn(0x7f78c80b3120 msgr2=0x7f78c8074300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.634+0000 7f78d0200640 1 -- 172.21.15.133:0/3675909986 shutdown_connections 2024-03-07T06:52:36.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.634+0000 7f78d0200640 1 -- 172.21.15.133:0/3675909986 wait complete. 2024-03-07T06:52:36.651 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-07T06:52:36.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.862+0000 7f86a6400640 1 Processor -- start 2024-03-07T06:52:36.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.862+0000 7f86a6400640 1 -- start start 2024-03-07T06:52:36.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.864+0000 7f86a6400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86a00b57f0 0x7f86a0071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:36.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.864+0000 7f86a6400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f86a00b2f80 0x7f86a00b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:36.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.864+0000 7f86a6400640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a0071d30 con 0x7f86a01477e0 2024-03-07T06:52:36.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.864+0000 7f86a6400640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a00b3930 con 0x7f86a00b57f0 2024-03-07T06:52:36.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.864+0000 7f86a6400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a0072010 con 0x7f86a00b2f80 2024-03-07T06:52:36.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.864+0000 7f869ea00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f86a00b57f0 0x7f86a0071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:36.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.864+0000 7f869f400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f86a01477e0 0x7f86a0162630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34292/0 (socket says 172.21.15.133:34292) 2024-03-07T06:52:36.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.864+0000 7f869f400640 1 -- 172.21.15.133:0/4205383576 learned_addr learned my addr 172.21.15.133:0/4205383576 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:36.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.864+0000 7f869fe00640 1 --2- 172.21.15.133:0/4205383576 >> v2:172.21.15.161:3300/0 conn(0x7f86a00b2f80 0x7f86a00b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:36.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.864+0000 7f869ea00640 1 -- 172.21.15.133:0/4205383576 >> v2:172.21.15.161:3300/0 conn(0x7f86a00b2f80 msgr2=0x7f86a00b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:36.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.865+0000 7f869ea00640 1 --2- 172.21.15.133:0/4205383576 >> v2:172.21.15.161:3300/0 conn(0x7f86a00b2f80 0x7f86a00b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.865+0000 7f869ea00640 1 -- 172.21.15.133:0/4205383576 >> v1:172.21.15.133:6789/0 conn(0x7f86a01477e0 legacy=0x7f86a0162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:36.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.865+0000 7f869ea00640 1 -- 172.21.15.133:0/4205383576 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a0161e30 con 0x7f86a00b57f0 2024-03-07T06:52:36.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.865+0000 7f869e000640 1 -- 172.21.15.133:0/4205383576 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401257606 0 0) 0x7f86a0071d30 con 0x7f86a01477e0 2024-03-07T06:52:36.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.866+0000 7f869ea00640 1 --2- 172.21.15.133:0/4205383576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86a00b57f0 0x7f86a0071430 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f86a006e110 tx=0x7f868c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce271f721c69d0e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:36.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.867+0000 7f869e000640 1 -- 172.21.15.133:0/4205383576 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c035020 con 0x7f86a00b57f0 2024-03-07T06:52:36.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.867+0000 7f869e000640 1 -- 172.21.15.133:0/4205383576 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c031830 con 0x7f86a00b57f0 2024-03-07T06:52:36.866 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.867+0000 7f869e000640 1 -- 172.21.15.133:0/4205383576 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c031c00 con 0x7f86a00b57f0 2024-03-07T06:52:36.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.867+0000 7f86a6400640 1 -- 172.21.15.133:0/4205383576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86a00b57f0 msgr2=0x7f86a0071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:36.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.867+0000 7f86a6400640 1 --2- 172.21.15.133:0/4205383576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86a00b57f0 0x7f86a0071430 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f86a006e110 tx=0x7f868c02b8d0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.868+0000 7f86a6400640 1 -- 172.21.15.133:0/4205383576 shutdown_connections 2024-03-07T06:52:36.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.868+0000 7f86a6400640 1 --2- 172.21.15.133:0/4205383576 >> v2:172.21.15.161:3300/0 conn(0x7f86a00b2f80 0x7f86a00b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.868+0000 7f86a6400640 1 --2- 172.21.15.133:0/4205383576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86a00b57f0 0x7f86a0071430 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.868+0000 7f86a6400640 1 -- 172.21.15.133:0/4205383576 >> 172.21.15.133:0/4205383576 conn(0x7f86a00b64d0 msgr2=0x7f86a00b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:36.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.868+0000 7f86a6400640 1 -- 172.21.15.133:0/4205383576 shutdown_connections 2024-03-07T06:52:36.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.868+0000 7f86a6400640 1 -- 172.21.15.133:0/4205383576 wait complete. 2024-03-07T06:52:36.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.869+0000 7f86a6400640 1 Processor -- start 2024-03-07T06:52:36.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.869+0000 7f86a6400640 1 -- start start 2024-03-07T06:52:36.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.870+0000 7f86a6400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86a00b2f80 0x7f86a013b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:36.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.870+0000 7f86a6400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86a00b57f0 0x7f86a00c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:36.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.870+0000 7f86a6400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86a01477e0 0x7f86a00c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:36.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.870+0000 7f86a6400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a0071970 con 0x7f86a01477e0 2024-03-07T06:52:36.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.870+0000 7f86a6400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a0147bc0 con 0x7f86a00b2f80 2024-03-07T06:52:36.875 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.870+0000 7f86a6400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a0071af0 con 0x7f86a00b57f0 2024-03-07T06:52:36.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.871+0000 7f869fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86a01477e0 0x7f86a00c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:36.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.871+0000 7f869fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86a01477e0 0x7f86a00c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43646/0 (socket says 172.21.15.133:43646) 2024-03-07T06:52:36.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.871+0000 7f869fe00640 1 -- 172.21.15.133:0/1257399914 learned_addr learned my addr 172.21.15.133:0/1257399914 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:36.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.871+0000 7f869f400640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86a00b2f80 0x7f86a013b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:36.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.871+0000 7f869ea00640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86a00b57f0 0x7f86a00c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:36.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.871+0000 7f869fe00640 1 -- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86a00b57f0 msgr2=0x7f86a00c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:36.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.871+0000 7f869fe00640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86a00b57f0 0x7f86a00c8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.871+0000 7f869fe00640 1 -- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86a00b2f80 msgr2=0x7f86a013b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:36.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.871+0000 7f869fe00640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86a00b2f80 0x7f86a013b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.871+0000 7f869fe00640 1 -- 172.21.15.133:0/1257399914 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a013bb30 con 0x7f86a01477e0 2024-03-07T06:52:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.871+0000 7f869f400640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86a00b2f80 0x7f86a013b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.871+0000 7f869fe00640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86a01477e0 0x7f86a00c9460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8698007e90 tx=0x7f869800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.872+0000 7f8697400640 1 -- 172.21.15.133:0/1257399914 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698014420 con 0x7f86a01477e0 2024-03-07T06:52:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.872+0000 7f8697400640 1 -- 172.21.15.133:0/1257399914 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8698019070 con 0x7f86a01477e0 2024-03-07T06:52:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.872+0000 7f86a6400640 1 -- 172.21.15.133:0/1257399914 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a00c9a90 con 0x7f86a01477e0 2024-03-07T06:52:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.872+0000 7f8697400640 1 -- 172.21.15.133:0/1257399914 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698014800 con 0x7f86a01477e0 2024-03-07T06:52:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.872+0000 7f86a6400640 1 -- 172.21.15.133:0/1257399914 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a00ca050 con 0x7f86a01477e0 2024-03-07T06:52:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.873+0000 7f86a6400640 1 -- 172.21.15.133:0/1257399914 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a0062c90 con 0x7f86a01477e0 2024-03-07T06:52:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.874+0000 7f8697400640 1 -- 172.21.15.133:0/1257399914 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f86980149a0 con 0x7f86a01477e0 2024-03-07T06:52:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.875+0000 7f8697400640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f868006b4d0 0x7f868006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.875+0000 7f8697400640 1 -- 172.21.15.133:0/1257399914 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f869808aa60 con 0x7f86a01477e0 2024-03-07T06:52:36.877 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.875+0000 7f869f400640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f868006b4d0 0x7f868006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:36.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.877+0000 7f869f400640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f868006b4d0 0x7f868006d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8690000f60 tx=0x7f8690009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:36.878 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:36.879+0000 7f8697400640 1 -- 172.21.15.133:0/1257399914 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f86980598e0 con 0x7f86a01477e0 2024-03-07T06:52:37.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:37.107+0000 7f86a6400640 1 -- 172.21.15.133:0/1257399914 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f86a0070d00 con 0x7f86a01477e0 2024-03-07T06:52:37.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:37.772+0000 7f8697400640 1 -- 172.21.15.133:0/1257399914 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f86980591e0 con 0x7f86a01477e0 2024-03-07T06:52:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:37.776+0000 7f86a6400640 1 -- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f868006b4d0 msgr2=0x7f868006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:37.776+0000 7f86a6400640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f868006b4d0 0x7f868006d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8690000f60 tx=0x7f8690009040 comp rx=0 tx=0).stop 2024-03-07T06:52:37.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:37.776+0000 7f86a6400640 1 -- 172.21.15.133:0/1257399914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86a01477e0 msgr2=0x7f86a00c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:37.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:37.776+0000 7f86a6400640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86a01477e0 0x7f86a00c9460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8698007e90 tx=0x7f869800e4b0 comp rx=0 tx=0).stop 2024-03-07T06:52:37.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:37.777+0000 7f86a6400640 1 -- 172.21.15.133:0/1257399914 shutdown_connections 2024-03-07T06:52:37.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:37.777+0000 7f86a6400640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86a01477e0 0x7f86a00c9460 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:37.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:37.777+0000 7f86a6400640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f86a00b57f0 0x7f86a00c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:37.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:37.777+0000 7f86a6400640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f868006b4d0 0x7f868006d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:37.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:37.777+0000 7f86a6400640 1 --2- 172.21.15.133:0/1257399914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86a00b2f80 0x7f86a013b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:37.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:37.777+0000 7f86a6400640 1 -- 172.21.15.133:0/1257399914 >> 172.21.15.133:0/1257399914 conn(0x7f86a00b64d0 msgr2=0x7f86a00b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:37.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:37.777+0000 7f86a6400640 1 -- 172.21.15.133:0/1257399914 shutdown_connections 2024-03-07T06:52:37.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:37.777+0000 7f86a6400640 1 -- 172.21.15.133:0/1257399914 wait complete. 2024-03-07T06:52:37.799 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-07T06:52:37.800 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-07T06:52:38.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.018+0000 7f2ffa600640 1 Processor -- start 2024-03-07T06:52:38.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.018+0000 7f2ffa600640 1 -- start start 2024-03-07T06:52:38.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.021+0000 7f2ffa600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ff415bb10 0x7f2ff40b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:38.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.021+0000 7f2ffa600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2ff4071f20 0x7f2ff4161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:38.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.021+0000 7f2ffa600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff415e4a0 con 0x7f2ff415f220 2024-03-07T06:52:38.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.021+0000 7f2ffa600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff415bef0 con 0x7f2ff4071f20 2024-03-07T06:52:38.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.021+0000 7f2ffa600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff415ed90 con 0x7f2ff415bb10 2024-03-07T06:52:38.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.022+0000 7f2febe00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f2ff415f220 0x7f2ff4071810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34296/0 (socket says 172.21.15.133:34296) 2024-03-07T06:52:38.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.022+0000 7f2febe00640 1 -- 172.21.15.133:0/3823093379 learned_addr learned my addr 172.21.15.133:0/3823093379 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:38.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.022+0000 7f2ff9c00640 1 --2- 172.21.15.133:0/3823093379 >> v2:172.21.15.161:3300/0 conn(0x7f2ff4071f20 0x7f2ff4161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:38.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.022+0000 7f2ff9200640 1 --2- 172.21.15.133:0/3823093379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ff415bb10 0x7f2ff40b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:38.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.022+0000 7f2ff3e00640 1 -- 172.21.15.133:0/3823093379 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86474206 0 0) 0x7f2ff415e4a0 con 0x7f2ff415f220 2024-03-07T06:52:38.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.022+0000 7f2ff3e00640 1 -- 172.21.15.133:0/3823093379 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fd8003520 con 0x7f2ff415f220 2024-03-07T06:52:38.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff3e00640 1 -- 172.21.15.133:0/3823093379 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2101692027 0 0) 0x7f2fd8003520 con 0x7f2ff415f220 2024-03-07T06:52:38.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff3e00640 1 -- 172.21.15.133:0/3823093379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ff415bb10 msgr2=0x7f2ff40b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:38.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff3e00640 1 --2- 172.21.15.133:0/3823093379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ff415bb10 0x7f2ff40b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:38.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff3e00640 1 -- 172.21.15.133:0/3823093379 >> v2:172.21.15.161:3300/0 conn(0x7f2ff4071f20 msgr2=0x7f2ff4161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:38.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff3e00640 1 --2- 172.21.15.133:0/3823093379 >> v2:172.21.15.161:3300/0 conn(0x7f2ff4071f20 0x7f2ff4161ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:38.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff3e00640 1 -- 172.21.15.133:0/3823093379 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff40b5750 con 0x7f2ff415f220 2024-03-07T06:52:38.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff3e00640 1 -- 172.21.15.133:0/3823093379 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f2fdc001690 con 0x7f2ff415f220 2024-03-07T06:52:38.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff3e00640 1 -- 172.21.15.133:0/3823093379 >> v1:172.21.15.133:6789/0 conn(0x7f2ff415f220 legacy=0x7f2ff4071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff3e00640 1 --2- 172.21.15.133:0/3823093379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ff415bb10 0x7f2fd80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff3e00640 1 --2- 172.21.15.133:0/3823093379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fd8003f80 0x7f2fd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff3e00640 1 --2- 172.21.15.133:0/3823093379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd8004d40 0x7f2fd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff3e00640 1 -- 172.21.15.133:0/3823093379 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff415ed90 con 0x7f2fd8004d40 2024-03-07T06:52:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff3e00640 1 -- 172.21.15.133:0/3823093379 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd800b9c0 con 0x7f2fd8003f80 2024-03-07T06:52:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff3e00640 1 -- 172.21.15.133:0/3823093379 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd800bb40 con 0x7f2ff415bb10 2024-03-07T06:52:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.023+0000 7f2ff9c00640 1 --2- 172.21.15.133:0/3823093379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd8004d40 0x7f2fd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.024+0000 7f2ff9200640 1 --2- 172.21.15.133:0/3823093379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ff415bb10 0x7f2fd80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:38.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.024+0000 7f2ff9c00640 1 -- 172.21.15.133:0/3823093379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ff415bb10 msgr2=0x7f2fd80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.024+0000 7f2ff9c00640 1 --2- 172.21.15.133:0/3823093379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ff415bb10 0x7f2fd80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.024+0000 7f2febe00640 1 --2- 172.21.15.133:0/3823093379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fd8003f80 0x7f2fd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.024+0000 7f2ff9c00640 1 -- 172.21.15.133:0/3823093379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fd8003f80 msgr2=0x7f2fd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.024+0000 7f2ff9c00640 1 --2- 172.21.15.133:0/3823093379 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2fd8003f80 0x7f2fd8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.024+0000 7f2ff9c00640 1 -- 172.21.15.133:0/3823093379 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd8003520 con 0x7f2fd8004d40 2024-03-07T06:52:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.024+0000 7f2ff9200640 1 --2- 172.21.15.133:0/3823093379 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ff415bb10 0x7f2fd80039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.025+0000 7f2ff9c00640 1 --2- 172.21.15.133:0/3823093379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd8004d40 0x7f2fd80051c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009f90 tx=0x7f2fd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7914c959a30d0676 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.025+0000 7f2ff3e00640 1 -- 172.21.15.133:0/3823093379 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec039070 con 0x7f2fd8004d40 2024-03-07T06:52:38.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.025+0000 7f2ff3e00640 1 -- 172.21.15.133:0/3823093379 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fec02cbd0 con 0x7f2fd8004d40 2024-03-07T06:52:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.025+0000 7f2ff3e00640 1 -- 172.21.15.133:0/3823093379 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec009c10 con 0x7f2fd8004d40 2024-03-07T06:52:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.025+0000 7f2ffa600640 1 -- 172.21.15.133:0/3823093379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd8004d40 msgr2=0x7f2fd80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.025+0000 7f2ffa600640 1 --2- 172.21.15.133:0/3823093379 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd8004d40 0x7f2fd80051c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009f90 tx=0x7f2fd8005830 comp rx=0 tx=0).stop 2024-03-07T06:52:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.026+0000 7f2ff9c00640 1 -- 172.21.15.133:0/3823093379 reap_dead start 2024-03-07T06:52:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.026+0000 7f2ffa600640 1 -- 172.21.15.133:0/3823093379 shutdown_connections 2024-03-07T06:52:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.026+0000 7f2ffa600640 1 -- 172.21.15.133:0/3823093379 >> 172.21.15.133:0/3823093379 conn(0x7f2ff4072a30 msgr2=0x7f2ff4072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.026+0000 7f2ffa600640 1 -- 172.21.15.133:0/3823093379 shutdown_connections 2024-03-07T06:52:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.026+0000 7f2ffa600640 1 -- 172.21.15.133:0/3823093379 wait complete. 2024-03-07T06:52:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.027+0000 7f2ffa600640 1 Processor -- start 2024-03-07T06:52:38.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.028+0000 7f2ffa600640 1 -- start start 2024-03-07T06:52:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ffa600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd8004d40 0x7f2ff414cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ffa600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ff41f6a90 0x7f2ff41f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ffa600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ff4158510 0x7f2ff4158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ffa600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff415ea20 con 0x7f2fd8004d40 2024-03-07T06:52:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ffa600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff40b5a30 con 0x7f2ff4158510 2024-03-07T06:52:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ffa600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff40b3ea0 con 0x7f2ff41f6a90 2024-03-07T06:52:38.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ff9200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd8004d40 0x7f2ff414cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ff9200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd8004d40 0x7f2ff414cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43668/0 (socket says 172.21.15.133:43668) 2024-03-07T06:52:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ff9200640 1 -- 172.21.15.133:0/2334128862 learned_addr learned my addr 172.21.15.133:0/2334128862 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ff9c00640 1 --2- 172.21.15.133:0/2334128862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ff4158510 0x7f2ff4158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2febe00640 1 --2- 172.21.15.133:0/2334128862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ff41f6a90 0x7f2ff41f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ff9200640 1 -- 172.21.15.133:0/2334128862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ff41f6a90 msgr2=0x7f2ff41f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:38.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ff9200640 1 --2- 172.21.15.133:0/2334128862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ff41f6a90 0x7f2ff41f6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:38.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ff9200640 1 -- 172.21.15.133:0/2334128862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ff4158510 msgr2=0x7f2ff4158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:38.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ff9200640 1 --2- 172.21.15.133:0/2334128862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ff4158510 0x7f2ff4158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:38.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.029+0000 7f2ff9200640 1 -- 172.21.15.133:0/2334128862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff40c5db0 con 0x7f2fd8004d40 2024-03-07T06:52:38.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.030+0000 7f2ff9200640 1 --2- 172.21.15.133:0/2334128862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd8004d40 0x7f2ff414cb00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4010910 tx=0x7f2fe4010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:38.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.030+0000 7f2ff1600640 1 -- 172.21.15.133:0/2334128862 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe400f040 con 0x7f2fd8004d40 2024-03-07T06:52:38.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.030+0000 7f2ff1600640 1 -- 172.21.15.133:0/2334128862 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe400ec20 con 0x7f2fd8004d40 2024-03-07T06:52:38.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.030+0000 7f2ffa600640 1 -- 172.21.15.133:0/2334128862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff40c6090 con 0x7f2fd8004d40 2024-03-07T06:52:38.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.030+0000 7f2ff1600640 1 -- 172.21.15.133:0/2334128862 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe401c400 con 0x7f2fd8004d40 2024-03-07T06:52:38.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.030+0000 7f2ffa600640 1 -- 172.21.15.133:0/2334128862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff40c6650 con 0x7f2fd8004d40 2024-03-07T06:52:38.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.031+0000 7f2ffa600640 1 -- 172.21.15.133:0/2334128862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff4067c80 con 0x7f2fd8004d40 2024-03-07T06:52:38.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.032+0000 7f2ff1600640 1 -- 172.21.15.133:0/2334128862 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2fe401c5a0 con 0x7f2fd8004d40 2024-03-07T06:52:38.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.034+0000 7f2ff1600640 1 --2- 172.21.15.133:0/2334128862 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f2fd006b4d0 0x7f2fd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:38.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.034+0000 7f2ff1600640 1 -- 172.21.15.133:0/2334128862 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fe408a900 con 0x7f2fd8004d40 2024-03-07T06:52:38.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.034+0000 7f2febe00640 1 --2- 172.21.15.133:0/2334128862 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f2fd006b4d0 0x7f2fd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:38.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.036+0000 7f2febe00640 1 --2- 172.21.15.133:0/2334128862 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f2fd006b4d0 0x7f2fd006d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4073050 tx=0x7f2fdc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:38.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.038+0000 7f2ff1600640 1 -- 172.21.15.133:0/2334128862 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2fe4018030 con 0x7f2fd8004d40 2024-03-07T06:52:38.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.266+0000 7f2ffa600640 1 -- 172.21.15.133:0/2334128862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2ff4062c90 con 0x7f2fd8004d40 2024-03-07T06:52:38.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.785+0000 7f2ff1600640 1 -- 172.21.15.133:0/2334128862 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2fe408a070 con 0x7f2fd8004d40 2024-03-07T06:52:38.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.789+0000 7f2ffa600640 1 -- 172.21.15.133:0/2334128862 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f2fd006b4d0 msgr2=0x7f2fd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:38.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.789+0000 7f2ffa600640 1 --2- 172.21.15.133:0/2334128862 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f2fd006b4d0 0x7f2fd006d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4073050 tx=0x7f2fdc009040 comp rx=0 tx=0).stop 2024-03-07T06:52:38.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.789+0000 7f2ffa600640 1 -- 172.21.15.133:0/2334128862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd8004d40 msgr2=0x7f2ff414cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:38.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.789+0000 7f2ffa600640 1 --2- 172.21.15.133:0/2334128862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd8004d40 0x7f2ff414cb00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4010910 tx=0x7f2fe4010dd0 comp rx=0 tx=0).stop 2024-03-07T06:52:38.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.789+0000 7f2ffa600640 1 -- 172.21.15.133:0/2334128862 shutdown_connections 2024-03-07T06:52:38.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.789+0000 7f2ffa600640 1 --2- 172.21.15.133:0/2334128862 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f2fd006b4d0 0x7f2fd006d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:38.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.789+0000 7f2ffa600640 1 --2- 172.21.15.133:0/2334128862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2ff4158510 0x7f2ff4158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:38.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.789+0000 7f2ffa600640 1 --2- 172.21.15.133:0/2334128862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2ff41f6a90 0x7f2ff41f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:38.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.789+0000 7f2ffa600640 1 --2- 172.21.15.133:0/2334128862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2fd8004d40 0x7f2ff414cb00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:38.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.789+0000 7f2ffa600640 1 -- 172.21.15.133:0/2334128862 >> 172.21.15.133:0/2334128862 conn(0x7f2ff4072a30 msgr2=0x7f2ff406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:38.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.790+0000 7f2ffa600640 1 -- 172.21.15.133:0/2334128862 shutdown_connections 2024-03-07T06:52:38.790 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:38.790+0000 7f2ffa600640 1 -- 172.21.15.133:0/2334128862 wait complete. 2024-03-07T06:52:38.808 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-07T06:52:38.808 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:52:39.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.018+0000 7f3c6c200640 1 Processor -- start 2024-03-07T06:52:39.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.018+0000 7f3c6c200640 1 -- start start 2024-03-07T06:52:39.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.021+0000 7f3c6c200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3c640af950 0x7f3c640b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.021+0000 7f3c6c200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6406f140 con 0x7f3c6415b1b0 2024-03-07T06:52:39.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.021+0000 7f3c6c200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6406f420 con 0x7f3c640ce5f0 2024-03-07T06:52:39.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.021+0000 7f3c6c200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c64071610 con 0x7f3c640af950 2024-03-07T06:52:39.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.021+0000 7f3c69800640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3c6415b1b0 0x7f3c640cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34302/0 (socket says 172.21.15.133:34302) 2024-03-07T06:52:39.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.021+0000 7f3c69800640 1 -- 172.21.15.133:0/1047363969 learned_addr learned my addr 172.21.15.133:0/1047363969 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:39.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.021+0000 7f3c6a200640 1 --2- 172.21.15.133:0/1047363969 >> v2:172.21.15.133:3300/0 conn(0x7f3c640af950 0x7f3c640b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.022+0000 7f3c63e00640 1 -- 172.21.15.133:0/1047363969 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396100112 0 0) 0x7f3c6406f140 con 0x7f3c6415b1b0 2024-03-07T06:52:39.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.022+0000 7f3c63e00640 1 -- 172.21.15.133:0/1047363969 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c4c005730 con 0x7f3c6415b1b0 2024-03-07T06:52:39.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.022+0000 7f3c6a200640 1 -- 172.21.15.133:0/1047363969 >> v1:172.21.15.161:6789/0 conn(0x7f3c640ce5f0 legacy=0x7f3c64071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.022+0000 7f3c6a200640 1 -- 172.21.15.133:0/1047363969 >> v1:172.21.15.133:6789/0 conn(0x7f3c6415b1b0 legacy=0x7f3c640cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.022+0000 7f3c6a200640 1 -- 172.21.15.133:0/1047363969 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c64071860 con 0x7f3c640af950 2024-03-07T06:52:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.023+0000 7f3c6a200640 1 --2- 172.21.15.133:0/1047363969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c640af950 0x7f3c640b3c10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009d90 tx=0x7f3c5c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ffa35a987de46b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.024+0000 7f3c63e00640 1 -- 172.21.15.133:0/1047363969 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c039070 con 0x7f3c640af950 2024-03-07T06:52:39.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.024+0000 7f3c63e00640 1 -- 172.21.15.133:0/1047363969 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c02cc60 con 0x7f3c640af950 2024-03-07T06:52:39.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.024+0000 7f3c63e00640 1 -- 172.21.15.133:0/1047363969 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c034640 con 0x7f3c640af950 2024-03-07T06:52:39.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.024+0000 7f3c6c200640 1 -- 172.21.15.133:0/1047363969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c640af950 msgr2=0x7f3c640b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.024 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.024+0000 7f3c6c200640 1 --2- 172.21.15.133:0/1047363969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c640af950 0x7f3c640b3c10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009d90 tx=0x7f3c5c02beb0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.024+0000 7f3c6c200640 1 -- 172.21.15.133:0/1047363969 shutdown_connections 2024-03-07T06:52:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.024+0000 7f3c6c200640 1 --2- 172.21.15.133:0/1047363969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c640af950 0x7f3c640b3c10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.024+0000 7f3c6c200640 1 -- 172.21.15.133:0/1047363969 >> 172.21.15.133:0/1047363969 conn(0x7f3c640b3120 msgr2=0x7f3c640b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.025+0000 7f3c6c200640 1 -- 172.21.15.133:0/1047363969 shutdown_connections 2024-03-07T06:52:39.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.025+0000 7f3c6c200640 1 -- 172.21.15.133:0/1047363969 wait complete. 2024-03-07T06:52:39.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.026+0000 7f3c6c200640 1 Processor -- start 2024-03-07T06:52:39.026 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.026+0000 7f3c6c200640 1 -- start start 2024-03-07T06:52:39.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.027+0000 7f3c6c200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c640af950 0x7f3c64157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.027+0000 7f3c6c200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c640ce5f0 0x7f3c6414aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.027+0000 7f3c6c200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c6415b1b0 0x7f3c640adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.027+0000 7f3c6c200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6415b590 con 0x7f3c640ce5f0 2024-03-07T06:52:39.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.027+0000 7f3c6c200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6406ee30 con 0x7f3c640af950 2024-03-07T06:52:39.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.027+0000 7f3c6c200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6415b710 con 0x7f3c6415b1b0 2024-03-07T06:52:39.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.028+0000 7f3c68e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c640ce5f0 0x7f3c6414aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.028+0000 7f3c68e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c640ce5f0 0x7f3c6414aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43684/0 (socket says 172.21.15.133:43684) 2024-03-07T06:52:39.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.028+0000 7f3c68e00640 1 -- 172.21.15.133:0/3641403363 learned_addr learned my addr 172.21.15.133:0/3641403363 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:39.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.028+0000 7f3c69800640 1 --2- 172.21.15.133:0/3641403363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c640af950 0x7f3c64157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.028+0000 7f3c6a200640 1 --2- 172.21.15.133:0/3641403363 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c6415b1b0 0x7f3c640adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.028+0000 7f3c68e00640 1 -- 172.21.15.133:0/3641403363 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c6415b1b0 msgr2=0x7f3c640adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.028+0000 7f3c68e00640 1 --2- 172.21.15.133:0/3641403363 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c6415b1b0 0x7f3c640adb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.028+0000 7f3c68e00640 1 -- 172.21.15.133:0/3641403363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c640af950 msgr2=0x7f3c64157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.028+0000 7f3c68e00640 1 --2- 172.21.15.133:0/3641403363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c640af950 0x7f3c64157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.028+0000 7f3c68e00640 1 -- 172.21.15.133:0/3641403363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c64156b00 con 0x7f3c640ce5f0 2024-03-07T06:52:39.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.028+0000 7f3c68e00640 1 --2- 172.21.15.133:0/3641403363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c640ce5f0 0x7f3c6414aa60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000c470 tx=0x7f3c50001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:39.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.029+0000 7f3c61600640 1 -- 172.21.15.133:0/3641403363 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c500052d0 con 0x7f3c640ce5f0 2024-03-07T06:52:39.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.029+0000 7f3c61600640 1 -- 172.21.15.133:0/3641403363 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c50005470 con 0x7f3c640ce5f0 2024-03-07T06:52:39.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.029+0000 7f3c61600640 1 -- 172.21.15.133:0/3641403363 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50007260 con 0x7f3c640ce5f0 2024-03-07T06:52:39.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.029+0000 7f3c6c200640 1 -- 172.21.15.133:0/3641403363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c64156de0 con 0x7f3c640ce5f0 2024-03-07T06:52:39.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.029+0000 7f3c6c200640 1 -- 172.21.15.133:0/3641403363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c641573a0 con 0x7f3c640ce5f0 2024-03-07T06:52:39.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.030+0000 7f3c6c200640 1 -- 172.21.15.133:0/3641403363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c64062c90 con 0x7f3c640ce5f0 2024-03-07T06:52:39.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.032+0000 7f3c61600640 1 -- 172.21.15.133:0/3641403363 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3c50007400 con 0x7f3c640ce5f0 2024-03-07T06:52:39.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.033+0000 7f3c61600640 1 --2- 172.21.15.133:0/3641403363 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3c4406b4d0 0x7f3c4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.033+0000 7f3c61600640 1 -- 172.21.15.133:0/3641403363 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c50089710 con 0x7f3c640ce5f0 2024-03-07T06:52:39.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.034+0000 7f3c69800640 1 --2- 172.21.15.133:0/3641403363 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3c4406b4d0 0x7f3c4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.035+0000 7f3c69800640 1 --2- 172.21.15.133:0/3641403363 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3c4406b4d0 0x7f3c4406d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3c580011e0 tx=0x7f3c58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:39.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.038+0000 7f3c61600640 1 -- 172.21.15.133:0/3641403363 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3c50058590 con 0x7f3c640ce5f0 2024-03-07T06:52:39.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.273+0000 7f3c6c200640 1 -- 172.21.15.133:0/3641403363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c640b3710 con 0x7f3c640ce5f0 2024-03-07T06:52:39.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.275+0000 7f3c61600640 1 -- 172.21.15.133:0/3641403363 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4983 (secure 0 0 0) 0x7f3c50057e90 con 0x7f3c640ce5f0 2024-03-07T06:52:39.274 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:39.275 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3269394125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3269394125},{"type":"v1","addr":"172.21.15.161:6835","nonce":3269394125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/72243172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":72243172},{"type":"v1","addr":"172.21.15.161:6837","nonce":72243172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3714291615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3714291615},{"type":"v1","addr":"172.21.15.133:6834","nonce":3714291615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:52:35.182567+0000","modified":"2024-03-07T06:52:38.268724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6835/4067860709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":4067860709},{"type":"v1","addr":"172.21.15.133:6835","nonce":4067860709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5100,"qdb_cluster":[5100]},"id":2}]} 2024-03-07T06:52:39.275 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 18 2024-03-07T06:52:39.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.277+0000 7f3c6c200640 1 -- 172.21.15.133:0/3641403363 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3c4406b4d0 msgr2=0x7f3c4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.277+0000 7f3c6c200640 1 --2- 172.21.15.133:0/3641403363 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3c4406b4d0 0x7f3c4406d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3c580011e0 tx=0x7f3c58009040 comp rx=0 tx=0).stop 2024-03-07T06:52:39.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.277+0000 7f3c6c200640 1 -- 172.21.15.133:0/3641403363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c640ce5f0 msgr2=0x7f3c6414aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.277+0000 7f3c6c200640 1 --2- 172.21.15.133:0/3641403363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c640ce5f0 0x7f3c6414aa60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000c470 tx=0x7f3c50001e70 comp rx=0 tx=0).stop 2024-03-07T06:52:39.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.278+0000 7f3c6c200640 1 -- 172.21.15.133:0/3641403363 shutdown_connections 2024-03-07T06:52:39.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.278+0000 7f3c6c200640 1 --2- 172.21.15.133:0/3641403363 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c6415b1b0 0x7f3c640adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.278+0000 7f3c6c200640 1 --2- 172.21.15.133:0/3641403363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c640ce5f0 0x7f3c6414aa60 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.278+0000 7f3c6c200640 1 --2- 172.21.15.133:0/3641403363 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3c4406b4d0 0x7f3c4406d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.278+0000 7f3c6c200640 1 --2- 172.21.15.133:0/3641403363 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c640af950 0x7f3c64157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.278+0000 7f3c6c200640 1 -- 172.21.15.133:0/3641403363 >> 172.21.15.133:0/3641403363 conn(0x7f3c640b3120 msgr2=0x7f3c640b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:39.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.278+0000 7f3c6c200640 1 -- 172.21.15.133:0/3641403363 shutdown_connections 2024-03-07T06:52:39.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.278+0000 7f3c6c200640 1 -- 172.21.15.133:0/3641403363 wait complete. 2024-03-07T06:52:39.304 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:52:39.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.489+0000 7ff4aa800640 1 Processor -- start 2024-03-07T06:52:39.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.490+0000 7ff4aa800640 1 -- start start 2024-03-07T06:52:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.492+0000 7ff4aa800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff4a415d570 0x7ff4a415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.492+0000 7ff4aa800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff4a413fc20 0x7ff4a41400b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.492+0000 7ff4aa800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a415de90 con 0x7ff4a415b560 2024-03-07T06:52:39.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.492+0000 7ff4aa800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a415c3b0 con 0x7ff4a415d570 2024-03-07T06:52:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.492+0000 7ff4aa800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a413df00 con 0x7ff4a413fc20 2024-03-07T06:52:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.493+0000 7ff4a3400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff4a415d570 0x7ff4a415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.493+0000 7ff49be00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7ff4a415b560 0x7ff4a415bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34314/0 (socket says 172.21.15.133:34314) 2024-03-07T06:52:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.493+0000 7ff49be00640 1 -- 172.21.15.133:0/1140573775 learned_addr learned my addr 172.21.15.133:0/1140573775 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.493+0000 7ff4a3e00640 1 --2- 172.21.15.133:0/1140573775 >> v2:172.21.15.161:3300/0 conn(0x7ff4a413fc20 0x7ff4a41400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.493+0000 7ff4a3400640 1 -- 172.21.15.133:0/1140573775 >> v2:172.21.15.161:3300/0 conn(0x7ff4a413fc20 msgr2=0x7ff4a41400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:39.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.493+0000 7ff4a3400640 1 --2- 172.21.15.133:0/1140573775 >> v2:172.21.15.161:3300/0 conn(0x7ff4a413fc20 0x7ff4a41400b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.493+0000 7ff4a3400640 1 -- 172.21.15.133:0/1140573775 >> v1:172.21.15.133:6789/0 conn(0x7ff4a415b560 legacy=0x7ff4a415bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.493+0000 7ff4a3400640 1 -- 172.21.15.133:0/1140573775 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a413e1e0 con 0x7ff4a415d570 2024-03-07T06:52:39.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.494+0000 7ff4a3400640 1 --2- 172.21.15.133:0/1140573775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4a415d570 0x7ff4a415d950 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40ac6a0 tx=0x7ff49402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4538984a784b72a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:39.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.495+0000 7ff4a2a00640 1 -- 172.21.15.133:0/1140573775 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494035020 con 0x7ff4a415d570 2024-03-07T06:52:39.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.495+0000 7ff4a2a00640 1 -- 172.21.15.133:0/1140573775 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff494031830 con 0x7ff4a415d570 2024-03-07T06:52:39.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.495+0000 7ff4a2a00640 1 -- 172.21.15.133:0/1140573775 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494031c00 con 0x7ff4a415d570 2024-03-07T06:52:39.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.495+0000 7ff4aa800640 1 -- 172.21.15.133:0/1140573775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4a415d570 msgr2=0x7ff4a415d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.495+0000 7ff4aa800640 1 --2- 172.21.15.133:0/1140573775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4a415d570 0x7ff4a415d950 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff4a40ac6a0 tx=0x7ff49402b8d0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.495+0000 7ff4aa800640 1 -- 172.21.15.133:0/1140573775 shutdown_connections 2024-03-07T06:52:39.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.495+0000 7ff4aa800640 1 --2- 172.21.15.133:0/1140573775 >> v2:172.21.15.161:3300/0 conn(0x7ff4a413fc20 0x7ff4a41400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.495+0000 7ff4aa800640 1 --2- 172.21.15.133:0/1140573775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4a415d570 0x7ff4a415d950 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.496+0000 7ff4aa800640 1 -- 172.21.15.133:0/1140573775 >> 172.21.15.133:0/1140573775 conn(0x7ff4a4073030 msgr2=0x7ff4a4070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:39.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.496+0000 7ff4aa800640 1 -- 172.21.15.133:0/1140573775 shutdown_connections 2024-03-07T06:52:39.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.496+0000 7ff4aa800640 1 -- 172.21.15.133:0/1140573775 wait complete. 2024-03-07T06:52:39.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.497+0000 7ff4aa800640 1 Processor -- start 2024-03-07T06:52:39.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.498+0000 7ff4aa800640 1 -- start start 2024-03-07T06:52:39.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.498+0000 7ff4aa800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4a413fc20 0x7ff4a413c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.498+0000 7ff4aa800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4a415b560 0x7ff4a40ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.498+0000 7ff4aa800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4a415d570 0x7ff4a4137f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.498+0000 7ff4aa800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a413ea80 con 0x7ff4a415d570 2024-03-07T06:52:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.498+0000 7ff4aa800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a413e7a0 con 0x7ff4a415b560 2024-03-07T06:52:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.498+0000 7ff4aa800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a415b9f0 con 0x7ff4a413fc20 2024-03-07T06:52:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.499+0000 7ff4a3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4a415d570 0x7ff4a4137f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.499+0000 7ff4a3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4a415d570 0x7ff4a4137f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43706/0 (socket says 172.21.15.133:43706) 2024-03-07T06:52:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.499+0000 7ff4a3e00640 1 -- 172.21.15.133:0/504378820 learned_addr learned my addr 172.21.15.133:0/504378820 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.499+0000 7ff4a3400640 1 --2- 172.21.15.133:0/504378820 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4a413fc20 0x7ff4a413c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.499+0000 7ff49be00640 1 --2- 172.21.15.133:0/504378820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4a415b560 0x7ff4a40ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.499+0000 7ff4a3e00640 1 -- 172.21.15.133:0/504378820 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4a413fc20 msgr2=0x7ff4a413c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.499+0000 7ff4a3e00640 1 --2- 172.21.15.133:0/504378820 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4a413fc20 0x7ff4a413c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.499+0000 7ff4a3e00640 1 -- 172.21.15.133:0/504378820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4a415b560 msgr2=0x7ff4a40ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.499+0000 7ff4a3e00640 1 --2- 172.21.15.133:0/504378820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4a415b560 0x7ff4a40ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.499+0000 7ff4a3e00640 1 -- 172.21.15.133:0/504378820 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a40aded0 con 0x7ff4a415d570 2024-03-07T06:52:39.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.500+0000 7ff4a3e00640 1 --2- 172.21.15.133:0/504378820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4a415d570 0x7ff4a4137f50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00cc30 tx=0x7ff49c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:39.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.500+0000 7ff49b400640 1 -- 172.21.15.133:0/504378820 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00ce40 con 0x7ff4a415d570 2024-03-07T06:52:39.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.500+0000 7ff49b400640 1 -- 172.21.15.133:0/504378820 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c003990 con 0x7ff4a415d570 2024-03-07T06:52:39.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.500+0000 7ff4aa800640 1 -- 172.21.15.133:0/504378820 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a4138490 con 0x7ff4a415d570 2024-03-07T06:52:39.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.500+0000 7ff49b400640 1 -- 172.21.15.133:0/504378820 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c003d40 con 0x7ff4a415d570 2024-03-07T06:52:39.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.500+0000 7ff4aa800640 1 -- 172.21.15.133:0/504378820 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a413cf70 con 0x7ff4a415d570 2024-03-07T06:52:39.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.501+0000 7ff4aa800640 1 -- 172.21.15.133:0/504378820 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a40ad1b0 con 0x7ff4a415d570 2024-03-07T06:52:39.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.502+0000 7ff49b400640 1 -- 172.21.15.133:0/504378820 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff49c003ee0 con 0x7ff4a415d570 2024-03-07T06:52:39.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.504+0000 7ff49b400640 1 --2- 172.21.15.133:0/504378820 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff48006b500 0x7ff48006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.504+0000 7ff49b400640 1 -- 172.21.15.133:0/504378820 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff49c08a6c0 con 0x7ff4a415d570 2024-03-07T06:52:39.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.504+0000 7ff4a3400640 1 --2- 172.21.15.133:0/504378820 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff48006b500 0x7ff48006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.505+0000 7ff4a3400640 1 --2- 172.21.15.133:0/504378820 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff48006b500 0x7ff48006d9c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff494036000 tx=0x7ff494008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:39.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.508+0000 7ff49b400640 1 -- 172.21.15.133:0/504378820 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff4a40ad1b0 con 0x7ff4a415d570 2024-03-07T06:52:39.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.705+0000 7ff4aa800640 1 -- 172.21.15.133:0/504378820 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4a4062c90 con 0x7ff4a415d570 2024-03-07T06:52:39.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.708+0000 7ff49b400640 1 -- 172.21.15.133:0/504378820 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16717 (secure 0 0 0) 0x7ff4a4138490 con 0x7ff4a415d570 2024-03-07T06:52:39.708 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:39.709 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":35,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:52:36.759644+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:52:33.007428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-07T06:52:34.201590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:52:39.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.711+0000 7ff4aa800640 1 -- 172.21.15.133:0/504378820 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff48006b500 msgr2=0x7ff48006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.711+0000 7ff4aa800640 1 --2- 172.21.15.133:0/504378820 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff48006b500 0x7ff48006d9c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff494036000 tx=0x7ff494008030 comp rx=0 tx=0).stop 2024-03-07T06:52:39.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.711+0000 7ff4aa800640 1 -- 172.21.15.133:0/504378820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4a415d570 msgr2=0x7ff4a4137f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.711+0000 7ff4aa800640 1 --2- 172.21.15.133:0/504378820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4a415d570 0x7ff4a4137f50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00cc30 tx=0x7ff49c00a5c0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.711+0000 7ff4aa800640 1 -- 172.21.15.133:0/504378820 shutdown_connections 2024-03-07T06:52:39.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.711+0000 7ff4aa800640 1 --2- 172.21.15.133:0/504378820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4a415d570 0x7ff4a4137f50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.711+0000 7ff4aa800640 1 --2- 172.21.15.133:0/504378820 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff48006b500 0x7ff48006d9c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.712+0000 7ff4aa800640 1 --2- 172.21.15.133:0/504378820 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff4a415b560 0x7ff4a40ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.712+0000 7ff4aa800640 1 --2- 172.21.15.133:0/504378820 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff4a413fc20 0x7ff4a413c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.712+0000 7ff4aa800640 1 -- 172.21.15.133:0/504378820 >> 172.21.15.133:0/504378820 conn(0x7ff4a4073030 msgr2=0x7ff4a406e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.712+0000 7ff4aa800640 1 -- 172.21.15.133:0/504378820 shutdown_connections 2024-03-07T06:52:39.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.712+0000 7ff4aa800640 1 -- 172.21.15.133:0/504378820 wait complete. 2024-03-07T06:52:39.730 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-07T06:52:39.730 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:52:39.927 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.928+0000 7f8578c00640 1 Processor -- start 2024-03-07T06:52:39.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.928+0000 7f8578c00640 1 -- start start 2024-03-07T06:52:39.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.931+0000 7f8578c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f857415e680 0x7f857406da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.931+0000 7f8578c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8574072880 0x7f8574071c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.931+0000 7f8578c00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8574072d60 con 0x7f857415e2a0 2024-03-07T06:52:39.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.931+0000 7f8578c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85740ccd60 con 0x7f8574072880 2024-03-07T06:52:39.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.931+0000 7f8578c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85740cfcf0 con 0x7f857415e680 2024-03-07T06:52:39.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.931+0000 7f8571e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f857415e2a0 0x7f857406e0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34318/0 (socket says 172.21.15.133:34318) 2024-03-07T06:52:39.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.931+0000 7f8571e00640 1 -- 172.21.15.133:0/3137480657 learned_addr learned my addr 172.21.15.133:0/3137480657 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:39.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.932+0000 7f8569400640 1 --2- 172.21.15.133:0/3137480657 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f857415e680 0x7f857406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.932+0000 7f8572800640 1 --2- 172.21.15.133:0/3137480657 >> v2:172.21.15.161:3300/0 conn(0x7f8574072880 0x7f8574071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.932+0000 7f8571400640 1 -- 172.21.15.133:0/3137480657 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853687182 0 0) 0x7f8574072d60 con 0x7f857415e2a0 2024-03-07T06:52:39.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.932+0000 7f8571400640 1 -- 172.21.15.133:0/3137480657 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8558003520 con 0x7f857415e2a0 2024-03-07T06:52:39.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.932+0000 7f8571400640 1 -- 172.21.15.133:0/3137480657 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 395179223 0 0) 0x7f8558003520 con 0x7f857415e2a0 2024-03-07T06:52:39.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.932+0000 7f8571400640 1 -- 172.21.15.133:0/3137480657 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f857415e680 msgr2=0x7f857406da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:39.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.932+0000 7f8571400640 1 --2- 172.21.15.133:0/3137480657 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f857415e680 0x7f857406da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.932+0000 7f8571400640 1 -- 172.21.15.133:0/3137480657 >> v2:172.21.15.161:3300/0 conn(0x7f8574072880 msgr2=0x7f8574071c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:39.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.932+0000 7f8571400640 1 --2- 172.21.15.133:0/3137480657 >> v2:172.21.15.161:3300/0 conn(0x7f8574072880 0x7f8574071c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.932+0000 7f8571400640 1 -- 172.21.15.133:0/3137480657 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85740cffd0 con 0x7f857415e2a0 2024-03-07T06:52:39.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.932+0000 7f8571400640 1 -- 172.21.15.133:0/3137480657 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f8564001670 con 0x7f857415e2a0 2024-03-07T06:52:39.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.932+0000 7f8571400640 1 -- 172.21.15.133:0/3137480657 >> v1:172.21.15.133:6789/0 conn(0x7f857415e2a0 legacy=0x7f857406e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.932+0000 7f8571400640 1 --2- 172.21.15.133:0/3137480657 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f857415e680 0x7f85580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.932+0000 7f8571400640 1 --2- 172.21.15.133:0/3137480657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8558003f30 0x7f8558004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.933+0000 7f8571400640 1 --2- 172.21.15.133:0/3137480657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8558004d40 0x7f85580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.933+0000 7f8571400640 1 -- 172.21.15.133:0/3137480657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85740ccd60 con 0x7f8558004d40 2024-03-07T06:52:39.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.933+0000 7f8571400640 1 -- 172.21.15.133:0/3137480657 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85740cfcf0 con 0x7f8558003f30 2024-03-07T06:52:39.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.933+0000 7f8571400640 1 -- 172.21.15.133:0/3137480657 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855800b9f0 con 0x7f857415e680 2024-03-07T06:52:39.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.933+0000 7f8572800640 1 --2- 172.21.15.133:0/3137480657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8558004d40 0x7f85580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.933+0000 7f8572800640 1 -- 172.21.15.133:0/3137480657 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f857415e680 msgr2=0x7f85580039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:39.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.933+0000 7f8572800640 1 --2- 172.21.15.133:0/3137480657 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f857415e680 0x7f85580039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.933+0000 7f8572800640 1 -- 172.21.15.133:0/3137480657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8558003f30 msgr2=0x7f8558004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:39.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.933+0000 7f8572800640 1 --2- 172.21.15.133:0/3137480657 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8558003f30 0x7f8558004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.933+0000 7f8572800640 1 -- 172.21.15.133:0/3137480657 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8558003520 con 0x7f8558004d40 2024-03-07T06:52:39.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.934+0000 7f8572800640 1 --2- 172.21.15.133:0/3137480657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8558004d40 0x7f85580051c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f85740b35b0 tx=0x7f8558005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25dc1bc213d4f636 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:39.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.935+0000 7f8571400640 1 -- 172.21.15.133:0/3137480657 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c039070 con 0x7f8558004d40 2024-03-07T06:52:39.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.935+0000 7f8571400640 1 -- 172.21.15.133:0/3137480657 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856c02cb90 con 0x7f8558004d40 2024-03-07T06:52:39.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.935+0000 7f8571400640 1 -- 172.21.15.133:0/3137480657 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c009c10 con 0x7f8558004d40 2024-03-07T06:52:39.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.935+0000 7f8578c00640 1 -- 172.21.15.133:0/3137480657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8558004d40 msgr2=0x7f85580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.935+0000 7f8578c00640 1 --2- 172.21.15.133:0/3137480657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8558004d40 0x7f85580051c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f85740b35b0 tx=0x7f8558005830 comp rx=0 tx=0).stop 2024-03-07T06:52:39.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.935+0000 7f8572800640 1 -- 172.21.15.133:0/3137480657 reap_dead start 2024-03-07T06:52:39.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.935+0000 7f8578c00640 1 -- 172.21.15.133:0/3137480657 shutdown_connections 2024-03-07T06:52:39.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.935+0000 7f8578c00640 1 -- 172.21.15.133:0/3137480657 >> 172.21.15.133:0/3137480657 conn(0x7f85740b21a0 msgr2=0x7f85740b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:39.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.935+0000 7f8578c00640 1 -- 172.21.15.133:0/3137480657 shutdown_connections 2024-03-07T06:52:39.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.935+0000 7f8578c00640 1 -- 172.21.15.133:0/3137480657 wait complete. 2024-03-07T06:52:39.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.937+0000 7f8578c00640 1 Processor -- start 2024-03-07T06:52:39.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.937+0000 7f8578c00640 1 -- start start 2024-03-07T06:52:39.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.938+0000 7f8578c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8558004d40 0x7f8574155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.938+0000 7f8578c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8574153ec0 0x7f85741478f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.938+0000 7f8578c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8574154340 0x7f8574147e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.938+0000 7f8578c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857406de20 con 0x7f8574153ec0 2024-03-07T06:52:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.938+0000 7f8578c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857406dca0 con 0x7f8574154340 2024-03-07T06:52:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.938+0000 7f8578c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8574072170 con 0x7f8558004d40 2024-03-07T06:52:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.938+0000 7f8569400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8574153ec0 0x7f85741478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.939+0000 7f8569400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8574153ec0 0x7f85741478f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43732/0 (socket says 172.21.15.133:43732) 2024-03-07T06:52:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.939+0000 7f8569400640 1 -- 172.21.15.133:0/3009721938 learned_addr learned my addr 172.21.15.133:0/3009721938 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.939+0000 7f8571e00640 1 --2- 172.21.15.133:0/3009721938 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8558004d40 0x7f8574155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.939+0000 7f8572800640 1 --2- 172.21.15.133:0/3009721938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8574154340 0x7f8574147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.939+0000 7f8569400640 1 -- 172.21.15.133:0/3009721938 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8558004d40 msgr2=0x7f8574155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.939+0000 7f8569400640 1 --2- 172.21.15.133:0/3009721938 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8558004d40 0x7f8574155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.939+0000 7f8569400640 1 -- 172.21.15.133:0/3009721938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8574154340 msgr2=0x7f8574147e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.939+0000 7f8569400640 1 --2- 172.21.15.133:0/3009721938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8574154340 0x7f8574147e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.939+0000 7f8569400640 1 -- 172.21.15.133:0/3009721938 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8574155e50 con 0x7f8574153ec0 2024-03-07T06:52:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.939+0000 7f8569400640 1 --2- 172.21.15.133:0/3009721938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8574153ec0 0x7f85741478f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f855c00e7f0 tx=0x7f855c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.940+0000 7f856aa00640 1 -- 172.21.15.133:0/3009721938 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c017070 con 0x7f8574153ec0 2024-03-07T06:52:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.940+0000 7f856aa00640 1 -- 172.21.15.133:0/3009721938 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855c00c940 con 0x7f8574153ec0 2024-03-07T06:52:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.940+0000 7f856aa00640 1 -- 172.21.15.133:0/3009721938 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c00eea0 con 0x7f8574153ec0 2024-03-07T06:52:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.940+0000 7f8578c00640 1 -- 172.21.15.133:0/3009721938 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85741f7f40 con 0x7f8574153ec0 2024-03-07T06:52:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.940+0000 7f8578c00640 1 -- 172.21.15.133:0/3009721938 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8574151ee0 con 0x7f8574153ec0 2024-03-07T06:52:39.941 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.941+0000 7f8578c00640 1 -- 172.21.15.133:0/3009721938 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8574062c90 con 0x7f8574153ec0 2024-03-07T06:52:39.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.942+0000 7f856aa00640 1 -- 172.21.15.133:0/3009721938 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f855c01a070 con 0x7f8574153ec0 2024-03-07T06:52:39.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.943+0000 7f856aa00640 1 --2- 172.21.15.133:0/3009721938 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f855006b4d0 0x7f855006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:39.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.943+0000 7f856aa00640 1 -- 172.21.15.133:0/3009721938 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f855c08a130 con 0x7f8574153ec0 2024-03-07T06:52:39.943 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.943+0000 7f8571e00640 1 --2- 172.21.15.133:0/3009721938 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f855006b4d0 0x7f855006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:39.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.944+0000 7f8571e00640 1 --2- 172.21.15.133:0/3009721938 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f855006b4d0 0x7f855006d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8564001860 tx=0x7f85640016f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:39.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:39.952+0000 7f856aa00640 1 -- 172.21.15.133:0/3009721938 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f855c058f10 con 0x7f8574153ec0 2024-03-07T06:52:40.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.144+0000 7f8578c00640 1 -- 172.21.15.133:0/3009721938 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85740b01a0 con 0x7f855006b4d0 2024-03-07T06:52:40.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.164+0000 7f856aa00640 1 -- 172.21.15.133:0/3009721938 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237085 (secure 0 0 0) 0x7f85740b01a0 con 0x7f855006b4d0 2024-03-07T06:52:40.167 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:40.169 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:52:40.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.168+0000 7f8578c00640 1 -- 172.21.15.133:0/3009721938 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f855006b4d0 msgr2=0x7f855006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:40.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.168+0000 7f8578c00640 1 --2- 172.21.15.133:0/3009721938 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f855006b4d0 0x7f855006d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8564001860 tx=0x7f85640016f0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.168+0000 7f8578c00640 1 -- 172.21.15.133:0/3009721938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8574153ec0 msgr2=0x7f85741478f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:40.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.168+0000 7f8578c00640 1 --2- 172.21.15.133:0/3009721938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8574153ec0 0x7f85741478f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f855c00e7f0 tx=0x7f855c00ecb0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.168+0000 7f8578c00640 1 -- 172.21.15.133:0/3009721938 shutdown_connections 2024-03-07T06:52:40.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.168+0000 7f8578c00640 1 --2- 172.21.15.133:0/3009721938 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f855006b4d0 0x7f855006d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.168+0000 7f8578c00640 1 --2- 172.21.15.133:0/3009721938 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8574154340 0x7f8574147e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.168+0000 7f8578c00640 1 --2- 172.21.15.133:0/3009721938 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8574153ec0 0x7f85741478f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.168+0000 7f8578c00640 1 --2- 172.21.15.133:0/3009721938 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8558004d40 0x7f8574155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.169+0000 7f8578c00640 1 -- 172.21.15.133:0/3009721938 >> 172.21.15.133:0/3009721938 conn(0x7f85740b21a0 msgr2=0x7f85740b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:40.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.169+0000 7f8578c00640 1 -- 172.21.15.133:0/3009721938 shutdown_connections 2024-03-07T06:52:40.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.169+0000 7f8578c00640 1 -- 172.21.15.133:0/3009721938 wait complete. 2024-03-07T06:52:40.185 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-03-07T06:52:38.499233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":220060,"kb_used_data":3880,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754660,"statfs":{"total":773094113280,"available":772868771840,"internally_reserved":0,"allocated":3973120,"data_stored":2232280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.787032"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:34.708921+0000","last_change":"2024-03-07T06:52:34.708921+0000","last_active":"2024-03-07T06:52:34.708921+0000","last_peered":"2024-03-07T06:52:34.708921+0000","last_clean":"2024-03-07T06:52:34.708921+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:34.708921+0000","last_undegraded":"2024-03-07T06:52:34.708921+0000","last_fullsized":"2024-03-07T06:52:34.708921+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:52:33.695462+0000","last_change":"2024-03-07T06:52:33.695462+0000","last_active":"2024-03-07T06:52:33.695462+0000","last_peered":"2024-03-07T06:52:33.695462+0000","last_clean":"2024-03-07T06:52:33.695462+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:52:33.695462+0000","last_undegraded":"2024-03-07T06:52:33.695462+0000","last_fullsized":"2024-03-07T06:52:33.695462+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":189,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-07T06:52:27.880623+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:52:27.880623+0000","last_peered":"2024-03-07T06:52:27.880623+0000","last_clean":"2024-03-07T06:52:27.880623+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:52:27.880623+0000","last_undegraded":"2024-03-07T06:52:27.880623+0000","last_fullsized":"2024-03-07T06:52:27.880623+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":942080,"data_stored":721811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":131443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":131443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":131443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":131443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":721811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:52:40.190 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:52:40.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.438+0000 7f9ea8a00640 1 Processor -- start 2024-03-07T06:52:40.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.438+0000 7f9ea8a00640 1 -- start start 2024-03-07T06:52:40.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.440+0000 7f9ea8a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9ea415aaf0 0x7f9ea406e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.440+0000 7f9ea8a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9ea40cd4c0 0x7f9ea415aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.440+0000 7f9ea8a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ea4072830 0x7f9ea4072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.440+0000 7f9ea8a00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea40731e0 con 0x7f9ea415aaf0 2024-03-07T06:52:40.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.440+0000 7f9ea8a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea406e850 con 0x7f9ea40cd4c0 2024-03-07T06:52:40.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.440+0000 7f9ea8a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea406eb30 con 0x7f9ea4072830 2024-03-07T06:52:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.440+0000 7f9ea3400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9ea415aaf0 0x7f9ea406e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.440+0000 7f9ea3400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9ea415aaf0 0x7f9ea406e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43734/0 (socket says 172.21.15.133:43734) 2024-03-07T06:52:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.440+0000 7f9ea3400640 1 -- 172.21.15.133:0/269279377 learned_addr learned my addr 172.21.15.133:0/269279377 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.441+0000 7f9ea2a00640 1 --2- 172.21.15.133:0/269279377 >> v2:172.21.15.161:3300/0 conn(0x7f9ea40cd4c0 0x7f9ea415aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.441+0000 7f9ea3e00640 1 --2- 172.21.15.133:0/269279377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ea4072830 0x7f9ea4072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.441+0000 7f9ea3400640 1 -- 172.21.15.133:0/269279377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ea4072830 msgr2=0x7f9ea4072ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.441+0000 7f9ea3400640 1 --2- 172.21.15.133:0/269279377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ea4072830 0x7f9ea4072ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.441+0000 7f9ea3400640 1 -- 172.21.15.133:0/269279377 >> v2:172.21.15.161:3300/0 conn(0x7f9ea40cd4c0 msgr2=0x7f9ea415aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.441+0000 7f9ea3400640 1 --2- 172.21.15.133:0/269279377 >> v2:172.21.15.161:3300/0 conn(0x7f9ea40cd4c0 0x7f9ea415aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.441+0000 7f9ea3400640 1 -- 172.21.15.133:0/269279377 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea406ee10 con 0x7f9ea415aaf0 2024-03-07T06:52:40.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.442+0000 7f9ea3400640 1 --2- 172.21.15.133:0/269279377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea415aaf0 0x7f9ea406e020 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4068ac0 tx=0x7f9e9402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd91a2ca526a8445 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:40.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.443+0000 7f9ea2000640 1 -- 172.21.15.133:0/269279377 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9402e070 con 0x7f9ea415aaf0 2024-03-07T06:52:40.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.443+0000 7f9ea2000640 1 -- 172.21.15.133:0/269279377 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e94037070 con 0x7f9ea415aaf0 2024-03-07T06:52:40.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.443+0000 7f9ea2000640 1 -- 172.21.15.133:0/269279377 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e94031a00 con 0x7f9ea415aaf0 2024-03-07T06:52:40.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.443+0000 7f9ea8a00640 1 -- 172.21.15.133:0/269279377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea415aaf0 msgr2=0x7f9ea406e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:40.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.443+0000 7f9ea8a00640 1 --2- 172.21.15.133:0/269279377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea415aaf0 0x7f9ea406e020 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4068ac0 tx=0x7f9e9402b880 comp rx=0 tx=0).stop 2024-03-07T06:52:40.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.443+0000 7f9ea8a00640 1 -- 172.21.15.133:0/269279377 shutdown_connections 2024-03-07T06:52:40.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.443+0000 7f9ea8a00640 1 --2- 172.21.15.133:0/269279377 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ea4072830 0x7f9ea4072ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.443+0000 7f9ea8a00640 1 --2- 172.21.15.133:0/269279377 >> v2:172.21.15.161:3300/0 conn(0x7f9ea40cd4c0 0x7f9ea415aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.443+0000 7f9ea8a00640 1 --2- 172.21.15.133:0/269279377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea415aaf0 0x7f9ea406e020 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.443+0000 7f9ea8a00640 1 -- 172.21.15.133:0/269279377 >> 172.21.15.133:0/269279377 conn(0x7f9ea40b2e90 msgr2=0x7f9ea40b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:40.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.443+0000 7f9ea8a00640 1 -- 172.21.15.133:0/269279377 shutdown_connections 2024-03-07T06:52:40.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.444+0000 7f9ea8a00640 1 -- 172.21.15.133:0/269279377 wait complete. 2024-03-07T06:52:40.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.445+0000 7f9ea8a00640 1 Processor -- start 2024-03-07T06:52:40.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.445+0000 7f9ea8a00640 1 -- start start 2024-03-07T06:52:40.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.446+0000 7f9ea8a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ea4072830 0x7f9ea41465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.446+0000 7f9ea8a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ea40cd4c0 0x7f9ea40c2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.446+0000 7f9ea8a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea415aaf0 0x7f9ea40b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.446+0000 7f9ea8a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea406f0f0 con 0x7f9ea415aaf0 2024-03-07T06:52:40.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.446+0000 7f9ea8a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea406e560 con 0x7f9ea4072830 2024-03-07T06:52:40.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.446+0000 7f9ea8a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea415b410 con 0x7f9ea40cd4c0 2024-03-07T06:52:40.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.447+0000 7f9ea3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea415aaf0 0x7f9ea40b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:40.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.447+0000 7f9ea3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea415aaf0 0x7f9ea40b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43744/0 (socket says 172.21.15.133:43744) 2024-03-07T06:52:40.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.447+0000 7f9ea3e00640 1 -- 172.21.15.133:0/241643101 learned_addr learned my addr 172.21.15.133:0/241643101 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:40.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.447+0000 7f9ea2a00640 1 --2- 172.21.15.133:0/241643101 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ea40cd4c0 0x7f9ea40c2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:40.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.447+0000 7f9ea3400640 1 --2- 172.21.15.133:0/241643101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ea4072830 0x7f9ea41465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:40.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.447+0000 7f9ea3e00640 1 -- 172.21.15.133:0/241643101 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ea40cd4c0 msgr2=0x7f9ea40c2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:40.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.447+0000 7f9ea3e00640 1 --2- 172.21.15.133:0/241643101 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ea40cd4c0 0x7f9ea40c2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.447+0000 7f9ea3e00640 1 -- 172.21.15.133:0/241643101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ea4072830 msgr2=0x7f9ea41465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:40.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.447+0000 7f9ea3e00640 1 --2- 172.21.15.133:0/241643101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ea4072830 0x7f9ea41465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.447+0000 7f9ea3e00640 1 -- 172.21.15.133:0/241643101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea415b8d0 con 0x7f9ea415aaf0 2024-03-07T06:52:40.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.447+0000 7f9ea3e00640 1 --2- 172.21.15.133:0/241643101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea415aaf0 0x7f9ea40b03d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c007e40 tx=0x7f9e9c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:40.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.448+0000 7f9e9b400640 1 -- 172.21.15.133:0/241643101 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c014620 con 0x7f9ea415aaf0 2024-03-07T06:52:40.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.448+0000 7f9e9b400640 1 -- 172.21.15.133:0/241643101 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c019070 con 0x7f9ea415aaf0 2024-03-07T06:52:40.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.448+0000 7f9ea8a00640 1 -- 172.21.15.133:0/241643101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea415bbb0 con 0x7f9ea415aaf0 2024-03-07T06:52:40.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.448+0000 7f9e9b400640 1 -- 172.21.15.133:0/241643101 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c014a20 con 0x7f9ea415aaf0 2024-03-07T06:52:40.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.448+0000 7f9ea8a00640 1 -- 172.21.15.133:0/241643101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea415c170 con 0x7f9ea415aaf0 2024-03-07T06:52:40.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.449+0000 7f9ea8a00640 1 -- 172.21.15.133:0/241643101 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea4062c90 con 0x7f9ea415aaf0 2024-03-07T06:52:40.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.450+0000 7f9e9b400640 1 -- 172.21.15.133:0/241643101 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9e9c014bc0 con 0x7f9ea415aaf0 2024-03-07T06:52:40.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.451+0000 7f9e9b400640 1 --2- 172.21.15.133:0/241643101 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9e8406b4d0 0x7f9e8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.452+0000 7f9e9b400640 1 -- 172.21.15.133:0/241643101 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e9c08aa40 con 0x7f9ea415aaf0 2024-03-07T06:52:40.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.452+0000 7f9ea3400640 1 --2- 172.21.15.133:0/241643101 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9e8406b4d0 0x7f9e8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:40.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.453+0000 7f9ea3400640 1 --2- 172.21.15.133:0/241643101 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9e8406b4d0 0x7f9e8406d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e94036000 tx=0x7f9e9403d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:40.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.459+0000 7f9e9b400640 1 -- 172.21.15.133:0/241643101 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9e9c059950 con 0x7f9ea415aaf0 2024-03-07T06:52:40.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.644+0000 7f9ea8a00640 1 -- 172.21.15.133:0/241643101 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ea415c3b0 con 0x7f9e8406b4d0 2024-03-07T06:52:40.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.662+0000 7f9e9b400640 1 -- 172.21.15.133:0/241643101 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251949 (secure 0 0 0) 0x7f9e94037040 con 0x7f9e8406b4d0 2024-03-07T06:52:40.666 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:40.667 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:52:40.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.668+0000 7f9ea8a00640 1 -- 172.21.15.133:0/241643101 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9e8406b4d0 msgr2=0x7f9e8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:40.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.668+0000 7f9ea8a00640 1 --2- 172.21.15.133:0/241643101 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9e8406b4d0 0x7f9e8406d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e94036000 tx=0x7f9e9403d040 comp rx=0 tx=0).stop 2024-03-07T06:52:40.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.668+0000 7f9ea8a00640 1 -- 172.21.15.133:0/241643101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea415aaf0 msgr2=0x7f9ea40b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:40.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.668+0000 7f9ea8a00640 1 --2- 172.21.15.133:0/241643101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea415aaf0 0x7f9ea40b03d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c007e40 tx=0x7f9e9c00e4b0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.668+0000 7f9ea8a00640 1 -- 172.21.15.133:0/241643101 shutdown_connections 2024-03-07T06:52:40.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.668+0000 7f9ea8a00640 1 --2- 172.21.15.133:0/241643101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ea415aaf0 0x7f9ea40b03d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.668+0000 7f9ea8a00640 1 --2- 172.21.15.133:0/241643101 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ea40cd4c0 0x7f9ea40c2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.668+0000 7f9ea8a00640 1 --2- 172.21.15.133:0/241643101 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9e8406b4d0 0x7f9e8406d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.668+0000 7f9ea8a00640 1 --2- 172.21.15.133:0/241643101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ea4072830 0x7f9ea41465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.668+0000 7f9ea8a00640 1 -- 172.21.15.133:0/241643101 >> 172.21.15.133:0/241643101 conn(0x7f9ea40b2e90 msgr2=0x7f9ea4153bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:40.669 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.668+0000 7f9ea8a00640 1 -- 172.21.15.133:0/241643101 shutdown_connections 2024-03-07T06:52:40.670 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.668+0000 7f9ea8a00640 1 -- 172.21.15.133:0/241643101 wait complete. 2024-03-07T06:52:40.693 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-03-07T06:52:40.499810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220500,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754220,"statfs":{"total":773094113280,"available":772868321280,"internally_reserved":0,"allocated":4431872,"data_stored":2438372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002052"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.029045+0000","last_change":"2024-03-07T06:52:35.774560+0000","last_active":"2024-03-07T06:52:37.029045+0000","last_peered":"2024-03-07T06:52:37.029045+0000","last_clean":"2024-03-07T06:52:37.029045+0000","last_became_active":"2024-03-07T06:52:35.774270+0000","last_became_peered":"2024-03-07T06:52:35.774270+0000","last_unstale":"2024-03-07T06:52:37.029045+0000","last_undegraded":"2024-03-07T06:52:37.029045+0000","last_fullsized":"2024-03-07T06:52:37.029045+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:54:12.441504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.026326+0000","last_change":"2024-03-07T06:52:35.774993+0000","last_active":"2024-03-07T06:52:37.026326+0000","last_peered":"2024-03-07T06:52:37.026326+0000","last_clean":"2024-03-07T06:52:37.026326+0000","last_became_active":"2024-03-07T06:52:35.773786+0000","last_became_peered":"2024-03-07T06:52:35.773786+0000","last_unstale":"2024-03-07T06:52:37.026326+0000","last_undegraded":"2024-03-07T06:52:37.026326+0000","last_fullsized":"2024-03-07T06:52:37.026326+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:12:11.845856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.029294+0000","last_change":"2024-03-07T06:52:35.776206+0000","last_active":"2024-03-07T06:52:37.029294+0000","last_peered":"2024-03-07T06:52:37.029294+0000","last_clean":"2024-03-07T06:52:37.029294+0000","last_became_active":"2024-03-07T06:52:35.775567+0000","last_became_peered":"2024-03-07T06:52:35.775567+0000","last_unstale":"2024-03-07T06:52:37.029294+0000","last_undegraded":"2024-03-07T06:52:37.029294+0000","last_fullsized":"2024-03-07T06:52:37.029294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:19:49.590912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.795778+0000","last_change":"2024-03-07T06:52:35.781269+0000","last_active":"2024-03-07T06:52:36.795778+0000","last_peered":"2024-03-07T06:52:36.795778+0000","last_clean":"2024-03-07T06:52:36.795778+0000","last_became_active":"2024-03-07T06:52:35.780733+0000","last_became_peered":"2024-03-07T06:52:35.780733+0000","last_unstale":"2024-03-07T06:52:36.795778+0000","last_undegraded":"2024-03-07T06:52:36.795778+0000","last_fullsized":"2024-03-07T06:52:36.795778+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:00:07.948174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.775095+0000","last_change":"2024-03-07T06:52:35.786140+0000","last_active":"2024-03-07T06:52:36.775095+0000","last_peered":"2024-03-07T06:52:36.775095+0000","last_clean":"2024-03-07T06:52:36.775095+0000","last_became_active":"2024-03-07T06:52:35.782367+0000","last_became_peered":"2024-03-07T06:52:35.782367+0000","last_unstale":"2024-03-07T06:52:36.775095+0000","last_undegraded":"2024-03-07T06:52:36.775095+0000","last_fullsized":"2024-03-07T06:52:36.775095+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:53:25.897973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.025568+0000","last_change":"2024-03-07T06:52:35.773517+0000","last_active":"2024-03-07T06:52:37.025568+0000","last_peered":"2024-03-07T06:52:37.025568+0000","last_clean":"2024-03-07T06:52:37.025568+0000","last_became_active":"2024-03-07T06:52:35.772842+0000","last_became_peered":"2024-03-07T06:52:35.772842+0000","last_unstale":"2024-03-07T06:52:37.025568+0000","last_undegraded":"2024-03-07T06:52:37.025568+0000","last_fullsized":"2024-03-07T06:52:37.025568+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:19:49.590912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.777685+0000","last_change":"2024-03-07T06:52:35.764938+0000","last_active":"2024-03-07T06:52:36.777685+0000","last_peered":"2024-03-07T06:52:36.777685+0000","last_clean":"2024-03-07T06:52:36.777685+0000","last_became_active":"2024-03-07T06:52:35.764513+0000","last_became_peered":"2024-03-07T06:52:35.764513+0000","last_unstale":"2024-03-07T06:52:36.777685+0000","last_undegraded":"2024-03-07T06:52:36.777685+0000","last_fullsized":"2024-03-07T06:52:36.777685+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:30:42.942864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.776712+0000","last_change":"2024-03-07T06:52:35.770402+0000","last_active":"2024-03-07T06:52:36.776712+0000","last_peered":"2024-03-07T06:52:36.776712+0000","last_clean":"2024-03-07T06:52:36.776712+0000","last_became_active":"2024-03-07T06:52:35.769993+0000","last_became_peered":"2024-03-07T06:52:35.769993+0000","last_unstale":"2024-03-07T06:52:36.776712+0000","last_undegraded":"2024-03-07T06:52:36.776712+0000","last_fullsized":"2024-03-07T06:52:36.776712+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:42:30.998629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.781499+0000","last_change":"2024-03-07T06:52:35.780873+0000","last_active":"2024-03-07T06:52:36.781499+0000","last_peered":"2024-03-07T06:52:36.781499+0000","last_clean":"2024-03-07T06:52:36.781499+0000","last_became_active":"2024-03-07T06:52:35.780572+0000","last_became_peered":"2024-03-07T06:52:35.780572+0000","last_unstale":"2024-03-07T06:52:36.781499+0000","last_undegraded":"2024-03-07T06:52:36.781499+0000","last_fullsized":"2024-03-07T06:52:36.781499+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:44:54.054891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.796088+0000","last_change":"2024-03-07T06:52:35.775438+0000","last_active":"2024-03-07T06:52:36.796088+0000","last_peered":"2024-03-07T06:52:36.796088+0000","last_clean":"2024-03-07T06:52:36.796088+0000","last_became_active":"2024-03-07T06:52:35.775163+0000","last_became_peered":"2024-03-07T06:52:35.775163+0000","last_unstale":"2024-03-07T06:52:36.796088+0000","last_undegraded":"2024-03-07T06:52:36.796088+0000","last_fullsized":"2024-03-07T06:52:36.796088+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:44:54.054891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.779248+0000","last_change":"2024-03-07T06:52:35.780694+0000","last_active":"2024-03-07T06:52:36.779248+0000","last_peered":"2024-03-07T06:52:36.779248+0000","last_clean":"2024-03-07T06:52:36.779248+0000","last_became_active":"2024-03-07T06:52:35.780352+0000","last_became_peered":"2024-03-07T06:52:35.780352+0000","last_unstale":"2024-03-07T06:52:36.779248+0000","last_undegraded":"2024-03-07T06:52:36.779248+0000","last_fullsized":"2024-03-07T06:52:36.779248+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:12:11.845856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.030005+0000","last_change":"2024-03-07T06:52:35.779561+0000","last_active":"2024-03-07T06:52:37.030005+0000","last_peered":"2024-03-07T06:52:37.030005+0000","last_clean":"2024-03-07T06:52:37.030005+0000","last_became_active":"2024-03-07T06:52:35.779133+0000","last_became_peered":"2024-03-07T06:52:35.779133+0000","last_unstale":"2024-03-07T06:52:37.030005+0000","last_undegraded":"2024-03-07T06:52:37.030005+0000","last_fullsized":"2024-03-07T06:52:37.030005+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:29:51.753832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.775125+0000","last_change":"2024-03-07T06:52:35.786840+0000","last_active":"2024-03-07T06:52:36.775125+0000","last_peered":"2024-03-07T06:52:36.775125+0000","last_clean":"2024-03-07T06:52:36.775125+0000","last_became_active":"2024-03-07T06:52:35.786437+0000","last_became_peered":"2024-03-07T06:52:35.786437+0000","last_unstale":"2024-03-07T06:52:36.775125+0000","last_undegraded":"2024-03-07T06:52:36.775125+0000","last_fullsized":"2024-03-07T06:52:36.775125+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:07:26.279786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.025591+0000","last_change":"2024-03-07T06:52:35.772476+0000","last_active":"2024-03-07T06:52:37.025591+0000","last_peered":"2024-03-07T06:52:37.025591+0000","last_clean":"2024-03-07T06:52:37.025591+0000","last_became_active":"2024-03-07T06:52:35.772023+0000","last_became_peered":"2024-03-07T06:52:35.772023+0000","last_unstale":"2024-03-07T06:52:37.025591+0000","last_undegraded":"2024-03-07T06:52:37.025591+0000","last_fullsized":"2024-03-07T06:52:37.025591+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:29:51.753832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.028641+0000","last_change":"2024-03-07T06:52:35.773827+0000","last_active":"2024-03-07T06:52:37.028641+0000","last_peered":"2024-03-07T06:52:37.028641+0000","last_clean":"2024-03-07T06:52:37.028641+0000","last_became_active":"2024-03-07T06:52:35.773510+0000","last_became_peered":"2024-03-07T06:52:35.773510+0000","last_unstale":"2024-03-07T06:52:37.028641+0000","last_undegraded":"2024-03-07T06:52:37.028641+0000","last_fullsized":"2024-03-07T06:52:37.028641+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:50:23.484049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.777536+0000","last_change":"2024-03-07T06:52:35.772530+0000","last_active":"2024-03-07T06:52:36.777536+0000","last_peered":"2024-03-07T06:52:36.777536+0000","last_clean":"2024-03-07T06:52:36.777536+0000","last_became_active":"2024-03-07T06:52:35.771678+0000","last_became_peered":"2024-03-07T06:52:35.771678+0000","last_unstale":"2024-03-07T06:52:36.777536+0000","last_undegraded":"2024-03-07T06:52:36.777536+0000","last_fullsized":"2024-03-07T06:52:36.777536+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:17:39.072164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.778195+0000","last_change":"2024-03-07T06:52:35.766223+0000","last_active":"2024-03-07T06:52:36.778195+0000","last_peered":"2024-03-07T06:52:36.778195+0000","last_clean":"2024-03-07T06:52:36.778195+0000","last_became_active":"2024-03-07T06:52:35.765858+0000","last_became_peered":"2024-03-07T06:52:35.765858+0000","last_unstale":"2024-03-07T06:52:36.778195+0000","last_undegraded":"2024-03-07T06:52:36.778195+0000","last_fullsized":"2024-03-07T06:52:36.778195+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:02:19.945266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.026353+0000","last_change":"2024-03-07T06:52:35.778466+0000","last_active":"2024-03-07T06:52:37.026353+0000","last_peered":"2024-03-07T06:52:37.026353+0000","last_clean":"2024-03-07T06:52:37.026353+0000","last_became_active":"2024-03-07T06:52:35.775073+0000","last_became_peered":"2024-03-07T06:52:35.775073+0000","last_unstale":"2024-03-07T06:52:37.026353+0000","last_undegraded":"2024-03-07T06:52:37.026353+0000","last_fullsized":"2024-03-07T06:52:37.026353+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:19:48.935882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.026204+0000","last_change":"2024-03-07T06:52:35.774663+0000","last_active":"2024-03-07T06:52:37.026204+0000","last_peered":"2024-03-07T06:52:37.026204+0000","last_clean":"2024-03-07T06:52:37.026204+0000","last_became_active":"2024-03-07T06:52:35.774390+0000","last_became_peered":"2024-03-07T06:52:35.774390+0000","last_unstale":"2024-03-07T06:52:37.026204+0000","last_undegraded":"2024-03-07T06:52:37.026204+0000","last_fullsized":"2024-03-07T06:52:37.026204+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:02:19.945266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.775617+0000","last_change":"2024-03-07T06:52:35.785685+0000","last_active":"2024-03-07T06:52:36.775617+0000","last_peered":"2024-03-07T06:52:36.775617+0000","last_clean":"2024-03-07T06:52:36.775617+0000","last_became_active":"2024-03-07T06:52:35.780947+0000","last_became_peered":"2024-03-07T06:52:35.780947+0000","last_unstale":"2024-03-07T06:52:36.775617+0000","last_undegraded":"2024-03-07T06:52:36.775617+0000","last_fullsized":"2024-03-07T06:52:36.775617+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:50.061653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.780398+0000","last_change":"2024-03-07T06:52:35.769136+0000","last_active":"2024-03-07T06:52:36.780398+0000","last_peered":"2024-03-07T06:52:36.780398+0000","last_clean":"2024-03-07T06:52:36.780398+0000","last_became_active":"2024-03-07T06:52:35.768843+0000","last_became_peered":"2024-03-07T06:52:35.768843+0000","last_unstale":"2024-03-07T06:52:36.780398+0000","last_undegraded":"2024-03-07T06:52:36.780398+0000","last_fullsized":"2024-03-07T06:52:36.780398+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:40:08.616833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.780365+0000","last_change":"2024-03-07T06:52:35.779664+0000","last_active":"2024-03-07T06:52:36.780365+0000","last_peered":"2024-03-07T06:52:36.780365+0000","last_clean":"2024-03-07T06:52:36.780365+0000","last_became_active":"2024-03-07T06:52:35.779373+0000","last_became_peered":"2024-03-07T06:52:35.779373+0000","last_unstale":"2024-03-07T06:52:36.780365+0000","last_undegraded":"2024-03-07T06:52:36.780365+0000","last_fullsized":"2024-03-07T06:52:36.780365+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:59:33.474982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.792171+0000","last_change":"2024-03-07T06:52:35.764085+0000","last_active":"2024-03-07T06:52:36.792171+0000","last_peered":"2024-03-07T06:52:36.792171+0000","last_clean":"2024-03-07T06:52:36.792171+0000","last_became_active":"2024-03-07T06:52:35.763042+0000","last_became_peered":"2024-03-07T06:52:35.763042+0000","last_unstale":"2024-03-07T06:52:36.792171+0000","last_undegraded":"2024-03-07T06:52:36.792171+0000","last_fullsized":"2024-03-07T06:52:36.792171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:54:12.441504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.028937+0000","last_change":"2024-03-07T06:52:35.770908+0000","last_active":"2024-03-07T06:52:37.028937+0000","last_peered":"2024-03-07T06:52:37.028937+0000","last_clean":"2024-03-07T06:52:37.028937+0000","last_became_active":"2024-03-07T06:52:35.770201+0000","last_became_peered":"2024-03-07T06:52:35.770201+0000","last_unstale":"2024-03-07T06:52:37.028937+0000","last_undegraded":"2024-03-07T06:52:37.028937+0000","last_fullsized":"2024-03-07T06:52:37.028937+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:44:54.054891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.796335+0000","last_change":"2024-03-07T06:52:35.768889+0000","last_active":"2024-03-07T06:52:36.796335+0000","last_peered":"2024-03-07T06:52:36.796335+0000","last_clean":"2024-03-07T06:52:36.796335+0000","last_became_active":"2024-03-07T06:52:35.768627+0000","last_became_peered":"2024-03-07T06:52:35.768627+0000","last_unstale":"2024-03-07T06:52:36.796335+0000","last_undegraded":"2024-03-07T06:52:36.796335+0000","last_fullsized":"2024-03-07T06:52:36.796335+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:42:30.998629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.026478+0000","last_change":"2024-03-07T06:52:35.771040+0000","last_active":"2024-03-07T06:52:37.026478+0000","last_peered":"2024-03-07T06:52:37.026478+0000","last_clean":"2024-03-07T06:52:37.026478+0000","last_became_active":"2024-03-07T06:52:35.770686+0000","last_became_peered":"2024-03-07T06:52:35.770686+0000","last_unstale":"2024-03-07T06:52:37.026478+0000","last_undegraded":"2024-03-07T06:52:37.026478+0000","last_fullsized":"2024-03-07T06:52:37.026478+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:27:07.019632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.796199+0000","last_change":"2024-03-07T06:52:35.778816+0000","last_active":"2024-03-07T06:52:36.796199+0000","last_peered":"2024-03-07T06:52:36.796199+0000","last_clean":"2024-03-07T06:52:36.796199+0000","last_became_active":"2024-03-07T06:52:35.778215+0000","last_became_peered":"2024-03-07T06:52:35.778215+0000","last_unstale":"2024-03-07T06:52:36.796199+0000","last_undegraded":"2024-03-07T06:52:36.796199+0000","last_fullsized":"2024-03-07T06:52:36.796199+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:50:23.484049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.030164+0000","last_change":"2024-03-07T06:52:35.773696+0000","last_active":"2024-03-07T06:52:37.030164+0000","last_peered":"2024-03-07T06:52:37.030164+0000","last_clean":"2024-03-07T06:52:37.030164+0000","last_became_active":"2024-03-07T06:52:35.773037+0000","last_became_peered":"2024-03-07T06:52:35.773037+0000","last_unstale":"2024-03-07T06:52:37.030164+0000","last_undegraded":"2024-03-07T06:52:37.030164+0000","last_fullsized":"2024-03-07T06:52:37.030164+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:45:42.012304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.780546+0000","last_change":"2024-03-07T06:52:35.774521+0000","last_active":"2024-03-07T06:52:36.780546+0000","last_peered":"2024-03-07T06:52:36.780546+0000","last_clean":"2024-03-07T06:52:36.780546+0000","last_became_active":"2024-03-07T06:52:35.773401+0000","last_became_peered":"2024-03-07T06:52:35.773401+0000","last_unstale":"2024-03-07T06:52:36.780546+0000","last_undegraded":"2024-03-07T06:52:36.780546+0000","last_fullsized":"2024-03-07T06:52:36.780546+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:36:54.923461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.777021+0000","last_change":"2024-03-07T06:52:35.764398+0000","last_active":"2024-03-07T06:52:36.777021+0000","last_peered":"2024-03-07T06:52:36.777021+0000","last_clean":"2024-03-07T06:52:36.777021+0000","last_became_active":"2024-03-07T06:52:35.764061+0000","last_became_peered":"2024-03-07T06:52:35.764061+0000","last_unstale":"2024-03-07T06:52:36.777021+0000","last_undegraded":"2024-03-07T06:52:36.777021+0000","last_fullsized":"2024-03-07T06:52:36.777021+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:35:34.325832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.028778+0000","last_change":"2024-03-07T06:52:35.776775+0000","last_active":"2024-03-07T06:52:37.028778+0000","last_peered":"2024-03-07T06:52:37.028778+0000","last_clean":"2024-03-07T06:52:37.028778+0000","last_became_active":"2024-03-07T06:52:35.776402+0000","last_became_peered":"2024-03-07T06:52:35.776402+0000","last_unstale":"2024-03-07T06:52:37.028778+0000","last_undegraded":"2024-03-07T06:52:37.028778+0000","last_fullsized":"2024-03-07T06:52:37.028778+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:02:28.976557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.776674+0000","last_change":"2024-03-07T06:52:35.761702+0000","last_active":"2024-03-07T06:52:36.776674+0000","last_peered":"2024-03-07T06:52:36.776674+0000","last_clean":"2024-03-07T06:52:36.776674+0000","last_became_active":"2024-03-07T06:52:35.761389+0000","last_became_peered":"2024-03-07T06:52:35.761389+0000","last_unstale":"2024-03-07T06:52:36.776674+0000","last_undegraded":"2024-03-07T06:52:36.776674+0000","last_fullsized":"2024-03-07T06:52:36.776674+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:34:15.040691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.792531+0000","last_change":"2024-03-07T06:52:35.771767+0000","last_active":"2024-03-07T06:52:36.792531+0000","last_peered":"2024-03-07T06:52:36.792531+0000","last_clean":"2024-03-07T06:52:36.792531+0000","last_became_active":"2024-03-07T06:52:35.768548+0000","last_became_peered":"2024-03-07T06:52:35.768548+0000","last_unstale":"2024-03-07T06:52:36.792531+0000","last_undegraded":"2024-03-07T06:52:36.792531+0000","last_fullsized":"2024-03-07T06:52:36.792531+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:55:13.780810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.029786+0000","last_change":"2024-03-07T06:52:34.752299+0000","last_active":"2024-03-07T06:52:37.029786+0000","last_peered":"2024-03-07T06:52:37.029786+0000","last_clean":"2024-03-07T06:52:37.029786+0000","last_became_active":"2024-03-07T06:52:34.751982+0000","last_became_peered":"2024-03-07T06:52:34.751982+0000","last_unstale":"2024-03-07T06:52:37.029786+0000","last_undegraded":"2024-03-07T06:52:37.029786+0000","last_fullsized":"2024-03-07T06:52:37.029786+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:17:20.769829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.029454+0000","last_change":"2024-03-07T06:52:35.760949+0000","last_active":"2024-03-07T06:52:37.029454+0000","last_peered":"2024-03-07T06:52:37.029454+0000","last_clean":"2024-03-07T06:52:37.029454+0000","last_became_active":"2024-03-07T06:52:35.760565+0000","last_became_peered":"2024-03-07T06:52:35.760565+0000","last_unstale":"2024-03-07T06:52:37.029454+0000","last_undegraded":"2024-03-07T06:52:37.029454+0000","last_fullsized":"2024-03-07T06:52:37.029454+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:24:05.939079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.792441+0000","last_change":"2024-03-07T06:52:34.754841+0000","last_active":"2024-03-07T06:52:36.792441+0000","last_peered":"2024-03-07T06:52:36.792441+0000","last_clean":"2024-03-07T06:52:36.792441+0000","last_became_active":"2024-03-07T06:52:34.754552+0000","last_became_peered":"2024-03-07T06:52:34.754552+0000","last_unstale":"2024-03-07T06:52:36.792441+0000","last_undegraded":"2024-03-07T06:52:36.792441+0000","last_fullsized":"2024-03-07T06:52:36.792441+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:00:02.046038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.781082+0000","last_change":"2024-03-07T06:52:35.773789+0000","last_active":"2024-03-07T06:52:36.781082+0000","last_peered":"2024-03-07T06:52:36.781082+0000","last_clean":"2024-03-07T06:52:36.781082+0000","last_became_active":"2024-03-07T06:52:35.773488+0000","last_became_peered":"2024-03-07T06:52:35.773488+0000","last_unstale":"2024-03-07T06:52:36.781082+0000","last_undegraded":"2024-03-07T06:52:36.781082+0000","last_fullsized":"2024-03-07T06:52:36.781082+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:21:43.753697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.026353+0000","last_change":"2024-03-07T06:52:34.748439+0000","last_active":"2024-03-07T06:52:37.026353+0000","last_peered":"2024-03-07T06:52:37.026353+0000","last_clean":"2024-03-07T06:52:37.026353+0000","last_became_active":"2024-03-07T06:52:34.746976+0000","last_became_peered":"2024-03-07T06:52:34.746976+0000","last_unstale":"2024-03-07T06:52:37.026353+0000","last_undegraded":"2024-03-07T06:52:37.026353+0000","last_fullsized":"2024-03-07T06:52:37.026353+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:43:41.756416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.796264+0000","last_change":"2024-03-07T06:52:35.779071+0000","last_active":"2024-03-07T06:52:36.796264+0000","last_peered":"2024-03-07T06:52:36.796264+0000","last_clean":"2024-03-07T06:52:36.796264+0000","last_became_active":"2024-03-07T06:52:35.778566+0000","last_became_peered":"2024-03-07T06:52:35.778566+0000","last_unstale":"2024-03-07T06:52:36.796264+0000","last_undegraded":"2024-03-07T06:52:36.796264+0000","last_fullsized":"2024-03-07T06:52:36.796264+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:34:15.040691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.775848+0000","last_change":"2024-03-07T06:52:34.735703+0000","last_active":"2024-03-07T06:52:36.775848+0000","last_peered":"2024-03-07T06:52:36.775848+0000","last_clean":"2024-03-07T06:52:36.775848+0000","last_became_active":"2024-03-07T06:52:34.735447+0000","last_became_peered":"2024-03-07T06:52:34.735447+0000","last_unstale":"2024-03-07T06:52:36.775848+0000","last_undegraded":"2024-03-07T06:52:36.775848+0000","last_fullsized":"2024-03-07T06:52:36.775848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:08:36.373216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.777522+0000","last_change":"2024-03-07T06:52:35.767652+0000","last_active":"2024-03-07T06:52:36.777522+0000","last_peered":"2024-03-07T06:52:36.777522+0000","last_clean":"2024-03-07T06:52:36.777522+0000","last_became_active":"2024-03-07T06:52:35.767329+0000","last_became_peered":"2024-03-07T06:52:35.767329+0000","last_unstale":"2024-03-07T06:52:36.777522+0000","last_undegraded":"2024-03-07T06:52:36.777522+0000","last_fullsized":"2024-03-07T06:52:36.777522+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:29:51.753832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.780650+0000","last_change":"2024-03-07T06:52:34.745923+0000","last_active":"2024-03-07T06:52:36.780650+0000","last_peered":"2024-03-07T06:52:36.780650+0000","last_clean":"2024-03-07T06:52:36.780650+0000","last_became_active":"2024-03-07T06:52:34.744741+0000","last_became_peered":"2024-03-07T06:52:34.744741+0000","last_unstale":"2024-03-07T06:52:36.780650+0000","last_undegraded":"2024-03-07T06:52:36.780650+0000","last_fullsized":"2024-03-07T06:52:36.780650+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:07:25.266328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.029268+0000","last_change":"2024-03-07T06:52:35.772424+0000","last_active":"2024-03-07T06:52:37.029268+0000","last_peered":"2024-03-07T06:52:37.029268+0000","last_clean":"2024-03-07T06:52:37.029268+0000","last_became_active":"2024-03-07T06:52:35.772047+0000","last_became_peered":"2024-03-07T06:52:35.772047+0000","last_unstale":"2024-03-07T06:52:37.029268+0000","last_undegraded":"2024-03-07T06:52:37.029268+0000","last_fullsized":"2024-03-07T06:52:37.029268+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:35:34.325832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.780357+0000","last_change":"2024-03-07T06:52:34.743229+0000","last_active":"2024-03-07T06:52:36.780357+0000","last_peered":"2024-03-07T06:52:36.780357+0000","last_clean":"2024-03-07T06:52:36.780357+0000","last_became_active":"2024-03-07T06:52:34.742866+0000","last_became_peered":"2024-03-07T06:52:34.742866+0000","last_unstale":"2024-03-07T06:52:36.780357+0000","last_undegraded":"2024-03-07T06:52:36.780357+0000","last_fullsized":"2024-03-07T06:52:36.780357+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:49.048195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.779662+0000","last_change":"2024-03-07T06:52:35.765297+0000","last_active":"2024-03-07T06:52:36.779662+0000","last_peered":"2024-03-07T06:52:36.779662+0000","last_clean":"2024-03-07T06:52:36.779662+0000","last_became_active":"2024-03-07T06:52:35.764988+0000","last_became_peered":"2024-03-07T06:52:35.764988+0000","last_unstale":"2024-03-07T06:52:36.779662+0000","last_undegraded":"2024-03-07T06:52:36.779662+0000","last_fullsized":"2024-03-07T06:52:36.779662+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:44:52.964545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.025661+0000","last_change":"2024-03-07T06:52:34.748499+0000","last_active":"2024-03-07T06:52:37.025661+0000","last_peered":"2024-03-07T06:52:37.025661+0000","last_clean":"2024-03-07T06:52:37.025661+0000","last_became_active":"2024-03-07T06:52:34.748176+0000","last_became_peered":"2024-03-07T06:52:34.748176+0000","last_unstale":"2024-03-07T06:52:37.025661+0000","last_undegraded":"2024-03-07T06:52:37.025661+0000","last_fullsized":"2024-03-07T06:52:37.025661+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:55:12.767352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.795544+0000","last_change":"2024-03-07T06:52:35.777482+0000","last_active":"2024-03-07T06:52:36.795544+0000","last_peered":"2024-03-07T06:52:36.795544+0000","last_clean":"2024-03-07T06:52:36.795544+0000","last_became_active":"2024-03-07T06:52:35.776261+0000","last_became_peered":"2024-03-07T06:52:35.776261+0000","last_unstale":"2024-03-07T06:52:36.795544+0000","last_undegraded":"2024-03-07T06:52:36.795544+0000","last_fullsized":"2024-03-07T06:52:36.795544+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:40:08.616833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.779821+0000","last_change":"2024-03-07T06:52:34.746834+0000","last_active":"2024-03-07T06:52:36.779821+0000","last_peered":"2024-03-07T06:52:36.779821+0000","last_clean":"2024-03-07T06:52:36.779821+0000","last_became_active":"2024-03-07T06:52:34.746511+0000","last_became_peered":"2024-03-07T06:52:34.746511+0000","last_unstale":"2024-03-07T06:52:36.779821+0000","last_undegraded":"2024-03-07T06:52:36.779821+0000","last_fullsized":"2024-03-07T06:52:36.779821+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:53:24.884515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.029194+0000","last_change":"2024-03-07T06:52:35.779896+0000","last_active":"2024-03-07T06:52:37.029194+0000","last_peered":"2024-03-07T06:52:37.029194+0000","last_clean":"2024-03-07T06:52:37.029194+0000","last_became_active":"2024-03-07T06:52:35.779506+0000","last_became_peered":"2024-03-07T06:52:35.779506+0000","last_unstale":"2024-03-07T06:52:37.029194+0000","last_undegraded":"2024-03-07T06:52:37.029194+0000","last_fullsized":"2024-03-07T06:52:37.029194+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:34:15.040691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.026037+0000","last_change":"2024-03-07T06:52:34.754791+0000","last_active":"2024-03-07T06:52:37.026037+0000","last_peered":"2024-03-07T06:52:37.026037+0000","last_clean":"2024-03-07T06:52:37.026037+0000","last_became_active":"2024-03-07T06:52:34.754498+0000","last_became_peered":"2024-03-07T06:52:34.754498+0000","last_unstale":"2024-03-07T06:52:37.026037+0000","last_undegraded":"2024-03-07T06:52:37.026037+0000","last_fullsized":"2024-03-07T06:52:37.026037+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:54:11.428046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.026083+0000","last_change":"2024-03-07T06:52:35.785889+0000","last_active":"2024-03-07T06:52:37.026083+0000","last_peered":"2024-03-07T06:52:37.026083+0000","last_clean":"2024-03-07T06:52:37.026083+0000","last_became_active":"2024-03-07T06:52:35.785574+0000","last_became_peered":"2024-03-07T06:52:35.785574+0000","last_unstale":"2024-03-07T06:52:37.026083+0000","last_undegraded":"2024-03-07T06:52:37.026083+0000","last_fullsized":"2024-03-07T06:52:37.026083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:30:42.942864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.792532+0000","last_change":"2024-03-07T06:52:34.742372+0000","last_active":"2024-03-07T06:52:36.792532+0000","last_peered":"2024-03-07T06:52:36.792532+0000","last_clean":"2024-03-07T06:52:36.792532+0000","last_became_active":"2024-03-07T06:52:34.741945+0000","last_became_peered":"2024-03-07T06:52:34.741945+0000","last_unstale":"2024-03-07T06:52:36.792532+0000","last_undegraded":"2024-03-07T06:52:36.792532+0000","last_fullsized":"2024-03-07T06:52:36.792532+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:55:15.290484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.026150+0000","last_change":"2024-03-07T06:52:35.769829+0000","last_active":"2024-03-07T06:52:37.026150+0000","last_peered":"2024-03-07T06:52:37.026150+0000","last_clean":"2024-03-07T06:52:37.026150+0000","last_became_active":"2024-03-07T06:52:35.769136+0000","last_became_peered":"2024-03-07T06:52:35.769136+0000","last_unstale":"2024-03-07T06:52:37.026150+0000","last_undegraded":"2024-03-07T06:52:37.026150+0000","last_fullsized":"2024-03-07T06:52:37.026150+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:00:33.583149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.029537+0000","last_change":"2024-03-07T06:52:34.755202+0000","last_active":"2024-03-07T06:52:37.029537+0000","last_peered":"2024-03-07T06:52:37.029537+0000","last_clean":"2024-03-07T06:52:37.029537+0000","last_became_active":"2024-03-07T06:52:34.754739+0000","last_became_peered":"2024-03-07T06:52:34.754739+0000","last_unstale":"2024-03-07T06:52:37.029537+0000","last_undegraded":"2024-03-07T06:52:37.029537+0000","last_fullsized":"2024-03-07T06:52:37.029537+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:02:27.963099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.029211+0000","last_change":"2024-03-07T06:52:35.784462+0000","last_active":"2024-03-07T06:52:37.029211+0000","last_peered":"2024-03-07T06:52:37.029211+0000","last_clean":"2024-03-07T06:52:37.029211+0000","last_became_active":"2024-03-07T06:52:35.784151+0000","last_became_peered":"2024-03-07T06:52:35.784151+0000","last_unstale":"2024-03-07T06:52:37.029211+0000","last_undegraded":"2024-03-07T06:52:37.029211+0000","last_fullsized":"2024-03-07T06:52:37.029211+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:42:30.998629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.026277+0000","last_change":"2024-03-07T06:52:34.745792+0000","last_active":"2024-03-07T06:52:37.026277+0000","last_peered":"2024-03-07T06:52:37.026277+0000","last_clean":"2024-03-07T06:52:37.026277+0000","last_became_active":"2024-03-07T06:52:34.745428+0000","last_became_peered":"2024-03-07T06:52:34.745428+0000","last_unstale":"2024-03-07T06:52:37.026277+0000","last_undegraded":"2024-03-07T06:52:37.026277+0000","last_fullsized":"2024-03-07T06:52:37.026277+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:02:27.963099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.795693+0000","last_change":"2024-03-07T06:52:35.773211+0000","last_active":"2024-03-07T06:52:36.795693+0000","last_peered":"2024-03-07T06:52:36.795693+0000","last_clean":"2024-03-07T06:52:36.795693+0000","last_became_active":"2024-03-07T06:52:35.772942+0000","last_became_peered":"2024-03-07T06:52:35.772942+0000","last_unstale":"2024-03-07T06:52:36.795693+0000","last_undegraded":"2024-03-07T06:52:36.795693+0000","last_fullsized":"2024-03-07T06:52:36.795693+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:36:54.923461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.780832+0000","last_change":"2024-03-07T06:52:34.747541+0000","last_active":"2024-03-07T06:52:36.780832+0000","last_peered":"2024-03-07T06:52:36.780832+0000","last_clean":"2024-03-07T06:52:36.780832+0000","last_became_active":"2024-03-07T06:52:34.745121+0000","last_became_peered":"2024-03-07T06:52:34.745121+0000","last_unstale":"2024-03-07T06:52:36.780832+0000","last_undegraded":"2024-03-07T06:52:36.780832+0000","last_fullsized":"2024-03-07T06:52:36.780832+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:37:14.568564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.026010+0000","last_change":"2024-03-07T06:52:35.780525+0000","last_active":"2024-03-07T06:52:37.026010+0000","last_peered":"2024-03-07T06:52:37.026010+0000","last_clean":"2024-03-07T06:52:37.026010+0000","last_became_active":"2024-03-07T06:52:35.778332+0000","last_became_peered":"2024-03-07T06:52:35.778332+0000","last_unstale":"2024-03-07T06:52:37.026010+0000","last_undegraded":"2024-03-07T06:52:37.026010+0000","last_fullsized":"2024-03-07T06:52:37.026010+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:17:39.072164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.795502+0000","last_change":"2024-03-07T06:52:34.747654+0000","last_active":"2024-03-07T06:52:36.795502+0000","last_peered":"2024-03-07T06:52:36.795502+0000","last_clean":"2024-03-07T06:52:36.795502+0000","last_became_active":"2024-03-07T06:52:34.747391+0000","last_became_peered":"2024-03-07T06:52:34.747391+0000","last_unstale":"2024-03-07T06:52:36.795502+0000","last_undegraded":"2024-03-07T06:52:36.795502+0000","last_fullsized":"2024-03-07T06:52:36.795502+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:17:20.769829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.777886+0000","last_change":"2024-03-07T06:52:35.761719+0000","last_active":"2024-03-07T06:52:36.777886+0000","last_peered":"2024-03-07T06:52:36.777886+0000","last_clean":"2024-03-07T06:52:36.777886+0000","last_became_active":"2024-03-07T06:52:35.761407+0000","last_became_peered":"2024-03-07T06:52:35.761407+0000","last_unstale":"2024-03-07T06:52:36.777886+0000","last_undegraded":"2024-03-07T06:52:36.777886+0000","last_fullsized":"2024-03-07T06:52:36.777886+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:00:33.583149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.025846+0000","last_change":"2024-03-07T06:52:34.754774+0000","last_active":"2024-03-07T06:52:37.025846+0000","last_peered":"2024-03-07T06:52:37.025846+0000","last_clean":"2024-03-07T06:52:37.025846+0000","last_became_active":"2024-03-07T06:52:34.754485+0000","last_became_peered":"2024-03-07T06:52:34.754485+0000","last_unstale":"2024-03-07T06:52:37.025846+0000","last_undegraded":"2024-03-07T06:52:37.025846+0000","last_fullsized":"2024-03-07T06:52:37.025846+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:24:04.925621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.029226+0000","last_change":"2024-03-07T06:52:35.756908+0000","last_active":"2024-03-07T06:52:37.029226+0000","last_peered":"2024-03-07T06:52:37.029226+0000","last_clean":"2024-03-07T06:52:37.029226+0000","last_became_active":"2024-03-07T06:52:35.756161+0000","last_became_peered":"2024-03-07T06:52:35.756161+0000","last_unstale":"2024-03-07T06:52:37.029226+0000","last_undegraded":"2024-03-07T06:52:37.029226+0000","last_fullsized":"2024-03-07T06:52:37.029226+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:55:13.780810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":22,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.025477+0000","last_change":"2024-03-07T06:52:34.745809+0000","last_active":"2024-03-07T06:52:37.025477+0000","last_peered":"2024-03-07T06:52:37.025477+0000","last_clean":"2024-03-07T06:52:37.025477+0000","last_became_active":"2024-03-07T06:52:34.745422+0000","last_became_peered":"2024-03-07T06:52:34.745422+0000","last_unstale":"2024-03-07T06:52:37.025477+0000","last_undegraded":"2024-03-07T06:52:37.025477+0000","last_fullsized":"2024-03-07T06:52:37.025477+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:42:29.584003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.781260+0000","last_change":"2024-03-07T06:52:35.768659+0000","last_active":"2024-03-07T06:52:36.781260+0000","last_peered":"2024-03-07T06:52:36.781260+0000","last_clean":"2024-03-07T06:52:36.781260+0000","last_became_active":"2024-03-07T06:52:35.768322+0000","last_became_peered":"2024-03-07T06:52:35.768322+0000","last_unstale":"2024-03-07T06:52:36.781260+0000","last_undegraded":"2024-03-07T06:52:36.781260+0000","last_fullsized":"2024-03-07T06:52:36.781260+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:00:07.948174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.796378+0000","last_change":"2024-03-07T06:52:34.743112+0000","last_active":"2024-03-07T06:52:36.796378+0000","last_peered":"2024-03-07T06:52:36.796378+0000","last_clean":"2024-03-07T06:52:36.796378+0000","last_became_active":"2024-03-07T06:52:34.742808+0000","last_became_peered":"2024-03-07T06:52:34.742808+0000","last_unstale":"2024-03-07T06:52:36.796378+0000","last_undegraded":"2024-03-07T06:52:36.796378+0000","last_fullsized":"2024-03-07T06:52:36.796378+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:55:15.290484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.030280+0000","last_change":"2024-03-07T06:52:35.772385+0000","last_active":"2024-03-07T06:52:37.030280+0000","last_peered":"2024-03-07T06:52:37.030280+0000","last_clean":"2024-03-07T06:52:37.030280+0000","last_became_active":"2024-03-07T06:52:35.771998+0000","last_became_peered":"2024-03-07T06:52:35.771998+0000","last_unstale":"2024-03-07T06:52:37.030280+0000","last_undegraded":"2024-03-07T06:52:37.030280+0000","last_fullsized":"2024-03-07T06:52:37.030280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:17:39.072164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.781237+0000","last_change":"2024-03-07T06:52:34.748591+0000","last_active":"2024-03-07T06:52:36.781237+0000","last_peered":"2024-03-07T06:52:36.781237+0000","last_clean":"2024-03-07T06:52:36.781237+0000","last_became_active":"2024-03-07T06:52:34.748136+0000","last_became_peered":"2024-03-07T06:52:34.748136+0000","last_unstale":"2024-03-07T06:52:36.781237+0000","last_undegraded":"2024-03-07T06:52:36.781237+0000","last_fullsized":"2024-03-07T06:52:36.781237+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:50:23.176714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.795776+0000","last_change":"2024-03-07T06:52:35.787055+0000","last_active":"2024-03-07T06:52:36.795776+0000","last_peered":"2024-03-07T06:52:36.795776+0000","last_clean":"2024-03-07T06:52:36.795776+0000","last_became_active":"2024-03-07T06:52:35.782946+0000","last_became_peered":"2024-03-07T06:52:35.782946+0000","last_unstale":"2024-03-07T06:52:36.795776+0000","last_undegraded":"2024-03-07T06:52:36.795776+0000","last_fullsized":"2024-03-07T06:52:36.795776+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:21:43.753697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.781284+0000","last_change":"2024-03-07T06:52:34.738445+0000","last_active":"2024-03-07T06:52:36.781284+0000","last_peered":"2024-03-07T06:52:36.781284+0000","last_clean":"2024-03-07T06:52:36.781284+0000","last_became_active":"2024-03-07T06:52:34.736790+0000","last_became_peered":"2024-03-07T06:52:34.736790+0000","last_unstale":"2024-03-07T06:52:36.781284+0000","last_undegraded":"2024-03-07T06:52:36.781284+0000","last_fullsized":"2024-03-07T06:52:36.781284+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:34:02.598454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.774937+0000","last_change":"2024-03-07T06:52:35.784087+0000","last_active":"2024-03-07T06:52:36.774937+0000","last_peered":"2024-03-07T06:52:36.774937+0000","last_clean":"2024-03-07T06:52:36.774937+0000","last_became_active":"2024-03-07T06:52:35.783788+0000","last_became_peered":"2024-03-07T06:52:35.783788+0000","last_unstale":"2024-03-07T06:52:36.774937+0000","last_undegraded":"2024-03-07T06:52:36.774937+0000","last_fullsized":"2024-03-07T06:52:36.774937+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:50:24.190172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.781022+0000","last_change":"2024-03-07T06:52:34.744204+0000","last_active":"2024-03-07T06:52:36.781022+0000","last_peered":"2024-03-07T06:52:36.781022+0000","last_clean":"2024-03-07T06:52:36.781022+0000","last_became_active":"2024-03-07T06:52:34.743605+0000","last_became_peered":"2024-03-07T06:52:34.743605+0000","last_unstale":"2024-03-07T06:52:36.781022+0000","last_undegraded":"2024-03-07T06:52:36.781022+0000","last_fullsized":"2024-03-07T06:52:36.781022+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:17:20.769829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.774864+0000","last_change":"2024-03-07T06:52:35.770702+0000","last_active":"2024-03-07T06:52:36.774864+0000","last_peered":"2024-03-07T06:52:36.774864+0000","last_clean":"2024-03-07T06:52:36.774864+0000","last_became_active":"2024-03-07T06:52:35.770033+0000","last_became_peered":"2024-03-07T06:52:35.770033+0000","last_unstale":"2024-03-07T06:52:36.774864+0000","last_undegraded":"2024-03-07T06:52:36.774864+0000","last_fullsized":"2024-03-07T06:52:36.774864+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:02:42.395412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.792735+0000","last_change":"2024-03-07T06:52:34.735817+0000","last_active":"2024-03-07T06:52:36.792735+0000","last_peered":"2024-03-07T06:52:36.792735+0000","last_clean":"2024-03-07T06:52:36.792735+0000","last_became_active":"2024-03-07T06:52:34.735406+0000","last_became_peered":"2024-03-07T06:52:34.735406+0000","last_unstale":"2024-03-07T06:52:36.792735+0000","last_undegraded":"2024-03-07T06:52:36.792735+0000","last_fullsized":"2024-03-07T06:52:36.792735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:49.048195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.775250+0000","last_change":"2024-03-07T06:52:35.776994+0000","last_active":"2024-03-07T06:52:36.775250+0000","last_peered":"2024-03-07T06:52:36.775250+0000","last_clean":"2024-03-07T06:52:36.775250+0000","last_became_active":"2024-03-07T06:52:35.776700+0000","last_became_peered":"2024-03-07T06:52:35.776700+0000","last_unstale":"2024-03-07T06:52:36.775250+0000","last_undegraded":"2024-03-07T06:52:36.775250+0000","last_fullsized":"2024-03-07T06:52:36.775250+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:37:15.582022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.028705+0000","last_change":"2024-03-07T06:52:34.750315+0000","last_active":"2024-03-07T06:52:37.028705+0000","last_peered":"2024-03-07T06:52:37.028705+0000","last_clean":"2024-03-07T06:52:37.028705+0000","last_became_active":"2024-03-07T06:52:34.749971+0000","last_became_peered":"2024-03-07T06:52:34.749971+0000","last_unstale":"2024-03-07T06:52:37.028705+0000","last_undegraded":"2024-03-07T06:52:37.028705+0000","last_fullsized":"2024-03-07T06:52:37.028705+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:49.048195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.774844+0000","last_change":"2024-03-07T06:52:35.776320+0000","last_active":"2024-03-07T06:52:36.774844+0000","last_peered":"2024-03-07T06:52:36.774844+0000","last_clean":"2024-03-07T06:52:36.774844+0000","last_became_active":"2024-03-07T06:52:35.776036+0000","last_became_peered":"2024-03-07T06:52:35.776036+0000","last_unstale":"2024-03-07T06:52:36.774844+0000","last_undegraded":"2024-03-07T06:52:36.774844+0000","last_fullsized":"2024-03-07T06:52:36.774844+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:39:55.464103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.779631+0000","last_change":"2024-03-07T06:52:34.738169+0000","last_active":"2024-03-07T06:52:36.779631+0000","last_peered":"2024-03-07T06:52:36.779631+0000","last_clean":"2024-03-07T06:52:36.779631+0000","last_became_active":"2024-03-07T06:52:34.737805+0000","last_became_peered":"2024-03-07T06:52:34.737805+0000","last_unstale":"2024-03-07T06:52:36.779631+0000","last_undegraded":"2024-03-07T06:52:36.779631+0000","last_fullsized":"2024-03-07T06:52:36.779631+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:39:54.450645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.777131+0000","last_change":"2024-03-07T06:52:35.759508+0000","last_active":"2024-03-07T06:52:36.777131+0000","last_peered":"2024-03-07T06:52:36.777131+0000","last_clean":"2024-03-07T06:52:36.777131+0000","last_became_active":"2024-03-07T06:52:35.759072+0000","last_became_peered":"2024-03-07T06:52:35.759072+0000","last_unstale":"2024-03-07T06:52:36.777131+0000","last_undegraded":"2024-03-07T06:52:36.777131+0000","last_fullsized":"2024-03-07T06:52:36.777131+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:19:49.590912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.028126+0000","last_change":"2024-03-07T06:52:34.755208+0000","last_active":"2024-03-07T06:52:37.028126+0000","last_peered":"2024-03-07T06:52:37.028126+0000","last_clean":"2024-03-07T06:52:37.028126+0000","last_became_active":"2024-03-07T06:52:34.754924+0000","last_became_peered":"2024-03-07T06:52:34.754924+0000","last_unstale":"2024-03-07T06:52:37.028126+0000","last_undegraded":"2024-03-07T06:52:37.028126+0000","last_fullsized":"2024-03-07T06:52:37.028126+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:42:29.584003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.775443+0000","last_change":"2024-03-07T06:52:35.779406+0000","last_active":"2024-03-07T06:52:36.775443+0000","last_peered":"2024-03-07T06:52:36.775443+0000","last_clean":"2024-03-07T06:52:36.775443+0000","last_became_active":"2024-03-07T06:52:35.779016+0000","last_became_peered":"2024-03-07T06:52:35.779016+0000","last_unstale":"2024-03-07T06:52:36.775443+0000","last_undegraded":"2024-03-07T06:52:36.775443+0000","last_fullsized":"2024-03-07T06:52:36.775443+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:34:03.611912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.796437+0000","last_change":"2024-03-07T06:52:34.750291+0000","last_active":"2024-03-07T06:52:36.796437+0000","last_peered":"2024-03-07T06:52:36.796437+0000","last_clean":"2024-03-07T06:52:36.796437+0000","last_became_active":"2024-03-07T06:52:34.749954+0000","last_became_peered":"2024-03-07T06:52:34.749954+0000","last_unstale":"2024-03-07T06:52:36.796437+0000","last_undegraded":"2024-03-07T06:52:36.796437+0000","last_fullsized":"2024-03-07T06:52:36.796437+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:48:45.620261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.030475+0000","last_change":"2024-03-07T06:52:35.774387+0000","last_active":"2024-03-07T06:52:37.030475+0000","last_peered":"2024-03-07T06:52:37.030475+0000","last_clean":"2024-03-07T06:52:37.030475+0000","last_became_active":"2024-03-07T06:52:35.774114+0000","last_became_peered":"2024-03-07T06:52:35.774114+0000","last_unstale":"2024-03-07T06:52:37.030475+0000","last_undegraded":"2024-03-07T06:52:37.030475+0000","last_fullsized":"2024-03-07T06:52:37.030475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:30:42.942864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.794875+0000","last_change":"2024-03-07T06:52:34.735580+0000","last_active":"2024-03-07T06:52:36.794875+0000","last_peered":"2024-03-07T06:52:36.794875+0000","last_clean":"2024-03-07T06:52:36.794875+0000","last_became_active":"2024-03-07T06:52:34.735018+0000","last_became_peered":"2024-03-07T06:52:34.735018+0000","last_unstale":"2024-03-07T06:52:36.794875+0000","last_undegraded":"2024-03-07T06:52:36.794875+0000","last_fullsized":"2024-03-07T06:52:36.794875+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:48:45.620261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.792640+0000","last_change":"2024-03-07T06:52:35.780308+0000","last_active":"2024-03-07T06:52:36.792640+0000","last_peered":"2024-03-07T06:52:36.792640+0000","last_clean":"2024-03-07T06:52:36.792640+0000","last_became_active":"2024-03-07T06:52:35.780013+0000","last_became_peered":"2024-03-07T06:52:35.780013+0000","last_unstale":"2024-03-07T06:52:36.792640+0000","last_undegraded":"2024-03-07T06:52:36.792640+0000","last_fullsized":"2024-03-07T06:52:36.792640+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:44:52.964545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.795910+0000","last_change":"2024-03-07T06:52:34.740833+0000","last_active":"2024-03-07T06:52:36.795910+0000","last_peered":"2024-03-07T06:52:36.795910+0000","last_clean":"2024-03-07T06:52:36.795910+0000","last_became_active":"2024-03-07T06:52:34.740517+0000","last_became_peered":"2024-03-07T06:52:34.740517+0000","last_unstale":"2024-03-07T06:52:36.795910+0000","last_undegraded":"2024-03-07T06:52:36.795910+0000","last_fullsized":"2024-03-07T06:52:36.795910+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:50:32.577677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.028831+0000","last_change":"2024-03-07T06:52:35.766737+0000","last_active":"2024-03-07T06:52:37.028831+0000","last_peered":"2024-03-07T06:52:37.028831+0000","last_clean":"2024-03-07T06:52:37.028831+0000","last_became_active":"2024-03-07T06:52:35.766382+0000","last_became_peered":"2024-03-07T06:52:35.766382+0000","last_unstale":"2024-03-07T06:52:37.028831+0000","last_undegraded":"2024-03-07T06:52:37.028831+0000","last_fullsized":"2024-03-07T06:52:37.028831+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:43:42.769874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.792700+0000","last_change":"2024-03-07T06:52:34.742820+0000","last_active":"2024-03-07T06:52:36.792700+0000","last_peered":"2024-03-07T06:52:36.792700+0000","last_clean":"2024-03-07T06:52:36.792700+0000","last_became_active":"2024-03-07T06:52:34.742498+0000","last_became_peered":"2024-03-07T06:52:34.742498+0000","last_unstale":"2024-03-07T06:52:36.792700+0000","last_undegraded":"2024-03-07T06:52:36.792700+0000","last_fullsized":"2024-03-07T06:52:36.792700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:17:20.769829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.792534+0000","last_change":"2024-03-07T06:52:35.769079+0000","last_active":"2024-03-07T06:52:36.792534+0000","last_peered":"2024-03-07T06:52:36.792534+0000","last_clean":"2024-03-07T06:52:36.792534+0000","last_became_active":"2024-03-07T06:52:35.765919+0000","last_became_peered":"2024-03-07T06:52:35.765919+0000","last_unstale":"2024-03-07T06:52:36.792534+0000","last_undegraded":"2024-03-07T06:52:36.792534+0000","last_fullsized":"2024-03-07T06:52:36.792534+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:24:05.939079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.795556+0000","last_change":"2024-03-07T06:52:34.750183+0000","last_active":"2024-03-07T06:52:36.795556+0000","last_peered":"2024-03-07T06:52:36.795556+0000","last_clean":"2024-03-07T06:52:36.795556+0000","last_became_active":"2024-03-07T06:52:34.749869+0000","last_became_peered":"2024-03-07T06:52:34.749869+0000","last_unstale":"2024-03-07T06:52:36.795556+0000","last_undegraded":"2024-03-07T06:52:36.795556+0000","last_fullsized":"2024-03-07T06:52:36.795556+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:00:02.046038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.025631+0000","last_change":"2024-03-07T06:52:35.774495+0000","last_active":"2024-03-07T06:52:37.025631+0000","last_peered":"2024-03-07T06:52:37.025631+0000","last_clean":"2024-03-07T06:52:37.025631+0000","last_became_active":"2024-03-07T06:52:35.774201+0000","last_became_peered":"2024-03-07T06:52:35.774201+0000","last_unstale":"2024-03-07T06:52:37.025631+0000","last_undegraded":"2024-03-07T06:52:37.025631+0000","last_fullsized":"2024-03-07T06:52:37.025631+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:45:42.012304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.780020+0000","last_change":"2024-03-07T06:52:34.751775+0000","last_active":"2024-03-07T06:52:36.780020+0000","last_peered":"2024-03-07T06:52:36.780020+0000","last_clean":"2024-03-07T06:52:36.780020+0000","last_became_active":"2024-03-07T06:52:34.750831+0000","last_became_peered":"2024-03-07T06:52:34.750831+0000","last_unstale":"2024-03-07T06:52:36.780020+0000","last_undegraded":"2024-03-07T06:52:36.780020+0000","last_fullsized":"2024-03-07T06:52:36.780020+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:02:41.381954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.779589+0000","last_change":"2024-03-07T06:52:35.771308+0000","last_active":"2024-03-07T06:52:36.779589+0000","last_peered":"2024-03-07T06:52:36.779589+0000","last_clean":"2024-03-07T06:52:36.779589+0000","last_became_active":"2024-03-07T06:52:35.770976+0000","last_became_peered":"2024-03-07T06:52:35.770976+0000","last_unstale":"2024-03-07T06:52:36.779589+0000","last_undegraded":"2024-03-07T06:52:36.779589+0000","last_fullsized":"2024-03-07T06:52:36.779589+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:24:05.939079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.025571+0000","last_change":"2024-03-07T06:52:34.753693+0000","last_active":"2024-03-07T06:52:37.025571+0000","last_peered":"2024-03-07T06:52:37.025571+0000","last_clean":"2024-03-07T06:52:37.025571+0000","last_became_active":"2024-03-07T06:52:34.753379+0000","last_became_peered":"2024-03-07T06:52:34.753379+0000","last_unstale":"2024-03-07T06:52:37.025571+0000","last_undegraded":"2024-03-07T06:52:37.025571+0000","last_fullsized":"2024-03-07T06:52:37.025571+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:44:51.951086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.777516+0000","last_change":"2024-03-07T06:52:35.784186+0000","last_active":"2024-03-07T06:52:36.777516+0000","last_peered":"2024-03-07T06:52:36.777516+0000","last_clean":"2024-03-07T06:52:36.777516+0000","last_became_active":"2024-03-07T06:52:35.783898+0000","last_became_peered":"2024-03-07T06:52:35.783898+0000","last_unstale":"2024-03-07T06:52:36.777516+0000","last_undegraded":"2024-03-07T06:52:36.777516+0000","last_fullsized":"2024-03-07T06:52:36.777516+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:34.708921+0000","last_clean_scrub_stamp":"2024-03-07T06:52:34.708921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:45:42.012304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:37.029402+0000","last_change":"2024-03-07T06:52:34.753726+0000","last_active":"2024-03-07T06:52:37.029402+0000","last_peered":"2024-03-07T06:52:37.029402+0000","last_clean":"2024-03-07T06:52:37.029402+0000","last_became_active":"2024-03-07T06:52:34.753406+0000","last_became_peered":"2024-03-07T06:52:34.753406+0000","last_unstale":"2024-03-07T06:52:37.029402+0000","last_undegraded":"2024-03-07T06:52:37.029402+0000","last_fullsized":"2024-03-07T06:52:37.029402+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:52:33.695462+0000","last_clean_scrub_stamp":"2024-03-07T06:52:33.695462+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:34:02.598454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":197,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-07T06:52:36.777685+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:52:36.777685+0000","last_peered":"2024-03-07T06:52:36.777685+0000","last_clean":"2024-03-07T06:52:36.777685+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:52:36.777685+0000","last_undegraded":"2024-03-07T06:52:36.777685+0000","last_fullsized":"2024-03-07T06:52:36.777685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":747060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":157218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":157274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":405504,"data_stored":156706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":158236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":156672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":158236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":991232,"data_stored":746970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:52:40.697 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-07T06:52:40.697 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-07T06:52:40.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.944+0000 7fb003600640 1 Processor -- start 2024-03-07T06:52:40.944 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.944+0000 7fb003600640 1 -- start start 2024-03-07T06:52:40.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.946+0000 7fb003600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faffc15e6f0 0x7faffc0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.946+0000 7fb003600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faffc161de0 con 0x7faffc141200 2024-03-07T06:52:40.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.946+0000 7fb003600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faffc141ed0 con 0x7faffc140e20 2024-03-07T06:52:40.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.946+0000 7fb003600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faffc15eb40 con 0x7faffc15e6f0 2024-03-07T06:52:40.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.946+0000 7faffbe00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7faffc141200 0x7faffc1417c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33862/0 (socket says 172.21.15.133:33862) 2024-03-07T06:52:40.946 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.946+0000 7faffbe00640 1 -- 172.21.15.133:0/1431405441 learned_addr learned my addr 172.21.15.133:0/1431405441 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.946+0000 7fb001600640 1 --2- 172.21.15.133:0/1431405441 >> v2:172.21.15.161:3300/0 conn(0x7faffc15e6f0 0x7faffc0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.946+0000 7faffb400640 1 -- 172.21.15.133:0/1431405441 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504129447 0 0) 0x7faffc161de0 con 0x7faffc141200 2024-03-07T06:52:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7faffb400640 1 -- 172.21.15.133:0/1431405441 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafe4003520 con 0x7faffc141200 2024-03-07T06:52:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7faffb400640 1 -- 172.21.15.133:0/1431405441 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3493395785 0 0) 0x7fafe4003520 con 0x7faffc141200 2024-03-07T06:52:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7faffb400640 1 -- 172.21.15.133:0/1431405441 >> v2:172.21.15.161:3300/0 conn(0x7faffc15e6f0 msgr2=0x7faffc0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7faffb400640 1 --2- 172.21.15.133:0/1431405441 >> v2:172.21.15.161:3300/0 conn(0x7faffc15e6f0 0x7faffc0b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7faffb400640 1 -- 172.21.15.133:0/1431405441 >> v1:172.21.15.161:6789/0 conn(0x7faffc140e20 legacy=0x7faffc0ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7faffb400640 1 -- 172.21.15.133:0/1431405441 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc15eda0 con 0x7faffc141200 2024-03-07T06:52:40.948 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7faffb400640 1 -- 172.21.15.133:0/1431405441 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fafe8001670 con 0x7faffc141200 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7faffb400640 1 -- 172.21.15.133:0/1431405441 >> v1:172.21.15.133:6789/0 conn(0x7faffc141200 legacy=0x7faffc1417c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7faffb400640 1 --2- 172.21.15.133:0/1431405441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafe40039b0 0x7fafe4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7faffb400640 1 --2- 172.21.15.133:0/1431405441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafe4004800 0x7fafe4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7faffb400640 1 --2- 172.21.15.133:0/1431405441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fafe40051c0 0x7fafe400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7faffb400640 1 -- 172.21.15.133:0/1431405441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc15eb40 con 0x7fafe40039b0 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7faffb400640 1 -- 172.21.15.133:0/1431405441 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe400c160 con 0x7fafe4004800 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7faffb400640 1 -- 172.21.15.133:0/1431405441 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe400c360 con 0x7fafe40051c0 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.947+0000 7fb000c00640 1 --2- 172.21.15.133:0/1431405441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafe40039b0 0x7fafe4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.948+0000 7fb001600640 1 --2- 172.21.15.133:0/1431405441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fafe40051c0 0x7fafe400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.948+0000 7fb000c00640 1 -- 172.21.15.133:0/1431405441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fafe40051c0 msgr2=0x7fafe400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.948+0000 7fb000c00640 1 --2- 172.21.15.133:0/1431405441 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fafe40051c0 0x7fafe400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.948+0000 7fb000c00640 1 -- 172.21.15.133:0/1431405441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafe4004800 msgr2=0x7fafe4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.948+0000 7faffbe00640 1 --2- 172.21.15.133:0/1431405441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafe4004800 0x7fafe4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.948+0000 7fb000c00640 1 --2- 172.21.15.133:0/1431405441 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fafe4004800 0x7fafe4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.948+0000 7fb000c00640 1 -- 172.21.15.133:0/1431405441 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe4003520 con 0x7fafe40039b0 2024-03-07T06:52:40.950 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.948+0000 7fb001600640 1 -- 172.21.15.133:0/1431405441 reap_dead start 2024-03-07T06:52:40.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.949+0000 7fb000c00640 1 --2- 172.21.15.133:0/1431405441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafe40039b0 0x7fafe4003dd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faff0001750 tx=0x7fafe4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed442454744d8370 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:40.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.949+0000 7faffb400640 1 -- 172.21.15.133:0/1431405441 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff0001370 con 0x7fafe40039b0 2024-03-07T06:52:40.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.949+0000 7faffb400640 1 -- 172.21.15.133:0/1431405441 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff00052d0 con 0x7fafe40039b0 2024-03-07T06:52:40.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.949+0000 7faffb400640 1 -- 172.21.15.133:0/1431405441 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff0003830 con 0x7fafe40039b0 2024-03-07T06:52:40.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.950+0000 7fb003600640 1 -- 172.21.15.133:0/1431405441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafe40039b0 msgr2=0x7fafe4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:40.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.950+0000 7fb003600640 1 --2- 172.21.15.133:0/1431405441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafe40039b0 0x7fafe4003dd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faff0001750 tx=0x7fafe4005830 comp rx=0 tx=0).stop 2024-03-07T06:52:40.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.950+0000 7fb003600640 1 -- 172.21.15.133:0/1431405441 shutdown_connections 2024-03-07T06:52:40.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.950+0000 7fb003600640 1 --2- 172.21.15.133:0/1431405441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fafe40039b0 0x7fafe4003dd0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.950+0000 7fb003600640 1 -- 172.21.15.133:0/1431405441 >> 172.21.15.133:0/1431405441 conn(0x7faffc072bd0 msgr2=0x7faffc072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:40.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.950+0000 7fb003600640 1 -- 172.21.15.133:0/1431405441 shutdown_connections 2024-03-07T06:52:40.951 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.950+0000 7fb003600640 1 -- 172.21.15.133:0/1431405441 wait complete. 2024-03-07T06:52:40.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.952+0000 7fb003600640 1 Processor -- start 2024-03-07T06:52:40.952 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.952+0000 7fb003600640 1 -- start start 2024-03-07T06:52:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.953+0000 7fb003600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fafe40039b0 0x7faffc134140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.953+0000 7fb003600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faffc0ad840 0x7faffc0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.953+0000 7fb003600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faffc0b1450 0x7faffc0b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.953+0000 7fb003600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc15f640 con 0x7faffc0ad840 2024-03-07T06:52:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.953+0000 7fb003600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc15f360 con 0x7faffc0b1450 2024-03-07T06:52:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.953+0000 7fb003600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faffc1417c0 con 0x7fafe40039b0 2024-03-07T06:52:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.953+0000 7faffbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faffc0ad840 0x7faffc0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.953+0000 7faffbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faffc0ad840 0x7faffc0ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47324/0 (socket says 172.21.15.133:47324) 2024-03-07T06:52:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.953+0000 7faffbe00640 1 -- 172.21.15.133:0/224302085 learned_addr learned my addr 172.21.15.133:0/224302085 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.954+0000 7fb000c00640 1 --2- 172.21.15.133:0/224302085 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fafe40039b0 0x7faffc134140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.954+0000 7faffbe00640 1 -- 172.21.15.133:0/224302085 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fafe40039b0 msgr2=0x7faffc134140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:40.954 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.954+0000 7faffbe00640 1 --2- 172.21.15.133:0/224302085 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fafe40039b0 0x7faffc134140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.954+0000 7faffbe00640 1 -- 172.21.15.133:0/224302085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faffc0b1450 msgr2=0x7faffc0b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.954+0000 7faffbe00640 1 --2- 172.21.15.133:0/224302085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faffc0b1450 0x7faffc0b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.954+0000 7faffbe00640 1 -- 172.21.15.133:0/224302085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc134680 con 0x7faffc0ad840 2024-03-07T06:52:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.954+0000 7faffbe00640 1 --2- 172.21.15.133:0/224302085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faffc0ad840 0x7faffc0ccde0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fafe800d7a0 tx=0x7fafe800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.954+0000 7faff8c00640 1 -- 172.21.15.133:0/224302085 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8019070 con 0x7faffc0ad840 2024-03-07T06:52:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.954+0000 7faff8c00640 1 -- 172.21.15.133:0/224302085 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe80072d0 con 0x7faffc0ad840 2024-03-07T06:52:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.954+0000 7faff8c00640 1 -- 172.21.15.133:0/224302085 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe80047e0 con 0x7faffc0ad840 2024-03-07T06:52:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.954+0000 7fb003600640 1 -- 172.21.15.133:0/224302085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faffc0b1e10 con 0x7faffc0ad840 2024-03-07T06:52:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.954+0000 7fb003600640 1 -- 172.21.15.133:0/224302085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faffc0ae0c0 con 0x7faffc0ad840 2024-03-07T06:52:40.955 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.956+0000 7fb003600640 1 -- 172.21.15.133:0/224302085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faffc062c90 con 0x7faffc0ad840 2024-03-07T06:52:40.957 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.958+0000 7faff8c00640 1 -- 172.21.15.133:0/224302085 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fafe8004980 con 0x7faffc0ad840 2024-03-07T06:52:40.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.959+0000 7faff8c00640 1 --2- 172.21.15.133:0/224302085 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fafdc06b4d0 0x7fafdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:40.958 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.959+0000 7faff8c00640 1 -- 172.21.15.133:0/224302085 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fafe80898a0 con 0x7faffc0ad840 2024-03-07T06:52:40.959 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.960+0000 7fb000c00640 1 --2- 172.21.15.133:0/224302085 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fafdc06b4d0 0x7fafdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:40.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.961+0000 7fb000c00640 1 --2- 172.21.15.133:0/224302085 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fafdc06b4d0 0x7fafdc06d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faff0033000 tx=0x7faff0009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:40.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:40.964+0000 7faff8c00640 1 -- 172.21.15.133:0/224302085 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faffc134680 con 0x7faffc0ad840 2024-03-07T06:52:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.229+0000 7fb003600640 1 -- 172.21.15.133:0/224302085 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7faffc06f250 con 0x7faffc0ad840 2024-03-07T06:52:41.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.233+0000 7faff8c00640 1 -- 172.21.15.133:0/224302085 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7fafe80580d0 con 0x7faffc0ad840 2024-03-07T06:52:41.233 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-03-07T06:52:41.234 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCbY+ll6xWENRAArKbw8ID7PLMFVCO3UlKDOg== 2024-03-07T06:52:41.234 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-03-07T06:52:41.234 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-03-07T06:52:41.234 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-07T06:52:41.234 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-03-07T06:52:41.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.236+0000 7fb003600640 1 -- 172.21.15.133:0/224302085 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fafdc06b4d0 msgr2=0x7fafdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:41.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.236+0000 7fb003600640 1 --2- 172.21.15.133:0/224302085 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fafdc06b4d0 0x7fafdc06d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faff0033000 tx=0x7faff0009030 comp rx=0 tx=0).stop 2024-03-07T06:52:41.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.236+0000 7fb003600640 1 -- 172.21.15.133:0/224302085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faffc0ad840 msgr2=0x7faffc0ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:41.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.236+0000 7fb003600640 1 --2- 172.21.15.133:0/224302085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faffc0ad840 0x7faffc0ccde0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fafe800d7a0 tx=0x7fafe800dc60 comp rx=0 tx=0).stop 2024-03-07T06:52:41.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.236+0000 7fb003600640 1 -- 172.21.15.133:0/224302085 shutdown_connections 2024-03-07T06:52:41.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.236+0000 7fb003600640 1 --2- 172.21.15.133:0/224302085 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fafdc06b4d0 0x7fafdc06d990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:41.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.236+0000 7fb003600640 1 --2- 172.21.15.133:0/224302085 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faffc0b1450 0x7faffc0b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:41.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.236+0000 7fb003600640 1 --2- 172.21.15.133:0/224302085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faffc0ad840 0x7faffc0ccde0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:41.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.236+0000 7fb003600640 1 --2- 172.21.15.133:0/224302085 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fafe40039b0 0x7faffc134140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:41.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.236+0000 7fb003600640 1 -- 172.21.15.133:0/224302085 >> 172.21.15.133:0/224302085 conn(0x7faffc072bd0 msgr2=0x7faffc0d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:41.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.237+0000 7fb003600640 1 -- 172.21.15.133:0/224302085 shutdown_connections 2024-03-07T06:52:41.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.237+0000 7fb003600640 1 -- 172.21.15.133:0/224302085 wait complete. 2024-03-07T06:52:41.257 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:52:41.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.454+0000 7f725a600640 1 Processor -- start 2024-03-07T06:52:41.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.454+0000 7f725a600640 1 -- start start 2024-03-07T06:52:41.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.456+0000 7f725a600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f725415fbb0 0x7f72540cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:41.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.456+0000 7f725a600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f72540cdee0 0x7f725415c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:41.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.456+0000 7f725a600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72540731b0 con 0x7f7254072950 2024-03-07T06:52:41.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.456+0000 7f725a600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7254072f60 con 0x7f72540cdee0 2024-03-07T06:52:41.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.456+0000 7f725a600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7254073490 con 0x7f725415fbb0 2024-03-07T06:52:41.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.456+0000 7f7253e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f72540cdee0 0x7f725415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:41.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.456+0000 7f724be00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7254072950 0x7f725415ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33872/0 (socket says 172.21.15.133:33872) 2024-03-07T06:52:41.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.456+0000 7f724be00640 1 -- 172.21.15.133:0/4125868602 learned_addr learned my addr 172.21.15.133:0/4125868602 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:41.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.457+0000 7f7253400640 1 --2- 172.21.15.133:0/4125868602 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f725415fbb0 0x7f72540cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:41.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.457+0000 7f7253e00640 1 -- 172.21.15.133:0/4125868602 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f725415fbb0 msgr2=0x7f72540cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:41.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.457+0000 7f7253e00640 1 --2- 172.21.15.133:0/4125868602 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f725415fbb0 0x7f72540cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:41.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.457+0000 7f7252a00640 1 -- 172.21.15.133:0/4125868602 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46212273 0 0) 0x7f72540731b0 con 0x7f7254072950 2024-03-07T06:52:41.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.457+0000 7f7253e00640 1 -- 172.21.15.133:0/4125868602 >> v1:172.21.15.133:6789/0 conn(0x7f7254072950 legacy=0x7f725415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:41.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.457+0000 7f7253e00640 1 -- 172.21.15.133:0/4125868602 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7254072150 con 0x7f72540cdee0 2024-03-07T06:52:41.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.458+0000 7f7253e00640 1 --2- 172.21.15.133:0/4125868602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72540cdee0 0x7f725415c550 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f72540730e0 tx=0x7f724c02be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da5880a66786b40 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:41.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.458+0000 7f7252a00640 1 -- 172.21.15.133:0/4125868602 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c039070 con 0x7f72540cdee0 2024-03-07T06:52:41.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.458+0000 7f7252a00640 1 -- 172.21.15.133:0/4125868602 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c02cbb0 con 0x7f72540cdee0 2024-03-07T06:52:41.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.459+0000 7f7252a00640 1 -- 172.21.15.133:0/4125868602 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c034440 con 0x7f72540cdee0 2024-03-07T06:52:41.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.459+0000 7f725a600640 1 -- 172.21.15.133:0/4125868602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72540cdee0 msgr2=0x7f725415c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:41.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.459+0000 7f725a600640 1 --2- 172.21.15.133:0/4125868602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72540cdee0 0x7f725415c550 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f72540730e0 tx=0x7f724c02be00 comp rx=0 tx=0).stop 2024-03-07T06:52:41.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.459+0000 7f725a600640 1 -- 172.21.15.133:0/4125868602 shutdown_connections 2024-03-07T06:52:41.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.459+0000 7f725a600640 1 --2- 172.21.15.133:0/4125868602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72540cdee0 0x7f725415c550 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:41.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.459+0000 7f725a600640 1 --2- 172.21.15.133:0/4125868602 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f725415fbb0 0x7f72540cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:41.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.459+0000 7f725a600640 1 -- 172.21.15.133:0/4125868602 >> 172.21.15.133:0/4125868602 conn(0x7f72540b3120 msgr2=0x7f72540b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:41.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.459+0000 7f725a600640 1 -- 172.21.15.133:0/4125868602 shutdown_connections 2024-03-07T06:52:41.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.459+0000 7f725a600640 1 -- 172.21.15.133:0/4125868602 wait complete. 2024-03-07T06:52:41.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.462+0000 7f725a600640 1 Processor -- start 2024-03-07T06:52:41.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.462+0000 7f725a600640 1 -- start start 2024-03-07T06:52:41.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.463+0000 7f725a600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72540cdee0 0x7f725409a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:41.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.463+0000 7f725a600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f725415fbb0 0x7f725409ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.463+0000 7f725a600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72541f5820 0x7f725409b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.463+0000 7f725a600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7254160980 con 0x7f72540cdee0 2024-03-07T06:52:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.463+0000 7f725a600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72540ce2f0 con 0x7f725415fbb0 2024-03-07T06:52:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.463+0000 7f725a600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72540b1a00 con 0x7f72541f5820 2024-03-07T06:52:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.464+0000 7f7253400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72540cdee0 0x7f725409a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.464+0000 7f7253400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72540cdee0 0x7f725409a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47338/0 (socket says 172.21.15.133:47338) 2024-03-07T06:52:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.464+0000 7f7253400640 1 -- 172.21.15.133:0/555975277 learned_addr learned my addr 172.21.15.133:0/555975277 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.464+0000 7f724be00640 1 --2- 172.21.15.133:0/555975277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f725415fbb0 0x7f725409ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.464+0000 7f7253400640 1 -- 172.21.15.133:0/555975277 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72541f5820 msgr2=0x7f725409b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.464+0000 7f7253400640 1 --2- 172.21.15.133:0/555975277 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72541f5820 0x7f725409b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.464+0000 7f7253400640 1 -- 172.21.15.133:0/555975277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f725415fbb0 msgr2=0x7f725409ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.464+0000 7f7253400640 1 --2- 172.21.15.133:0/555975277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f725415fbb0 0x7f725409ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.464+0000 7f7253400640 1 -- 172.21.15.133:0/555975277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725409cf60 con 0x7f72540cdee0 2024-03-07T06:52:41.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.464+0000 7f7253400640 1 --2- 172.21.15.133:0/555975277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72540cdee0 0x7f725409a9d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f724400d620 tx=0x7f724400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:41.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.465+0000 7f724b400640 1 -- 172.21.15.133:0/555975277 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724400b620 con 0x7f72540cdee0 2024-03-07T06:52:41.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.465+0000 7f724b400640 1 -- 172.21.15.133:0/555975277 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7244019070 con 0x7f72540cdee0 2024-03-07T06:52:41.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.465+0000 7f724b400640 1 -- 172.21.15.133:0/555975277 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724400ba00 con 0x7f72540cdee0 2024-03-07T06:52:41.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.465+0000 7f725a600640 1 -- 172.21.15.133:0/555975277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72541f5d20 con 0x7f72540cdee0 2024-03-07T06:52:41.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.465+0000 7f725a600640 1 -- 172.21.15.133:0/555975277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72541f62e0 con 0x7f72540cdee0 2024-03-07T06:52:41.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.466+0000 7f725a600640 1 -- 172.21.15.133:0/555975277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7254062c90 con 0x7f72540cdee0 2024-03-07T06:52:41.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.467+0000 7f724b400640 1 -- 172.21.15.133:0/555975277 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f724400bba0 con 0x7f72540cdee0 2024-03-07T06:52:41.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.468+0000 7f724b400640 1 --2- 172.21.15.133:0/555975277 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f723006b4d0 0x7f723006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:41.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.468+0000 7f724b400640 1 -- 172.21.15.133:0/555975277 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f724408a630 con 0x7f72540cdee0 2024-03-07T06:52:41.468 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.468+0000 7f724be00640 1 --2- 172.21.15.133:0/555975277 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f723006b4d0 0x7f723006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:41.469 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.469+0000 7f724be00640 1 --2- 172.21.15.133:0/555975277 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f723006b4d0 0x7f723006d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f723c007fd0 tx=0x7f723c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:41.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.477+0000 7f724b400640 1 -- 172.21.15.133:0/555975277 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f72440594a0 con 0x7f72540cdee0 2024-03-07T06:52:41.641 INFO:tasks.ceph.mds.a.smithi133.stderr:2024-03-07T06:52:41.641+0000 7f7faf000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-07T06:52:41.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.710+0000 7f725a600640 1 -- 172.21.15.133:0/555975277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72540ce470 con 0x7f72540cdee0 2024-03-07T06:52:41.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.711+0000 7f724b400640 1 -- 172.21.15.133:0/555975277 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4983 (secure 0 0 0) 0x7f7244058da0 con 0x7f72540cdee0 2024-03-07T06:52:41.712 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:41.712 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3269394125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3269394125},{"type":"v1","addr":"172.21.15.161:6835","nonce":3269394125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/72243172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":72243172},{"type":"v1","addr":"172.21.15.161:6837","nonce":72243172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3714291615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3714291615},{"type":"v1","addr":"172.21.15.133:6834","nonce":3714291615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:52:35.182567+0000","modified":"2024-03-07T06:52:38.268724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6835/4067860709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":4067860709},{"type":"v1","addr":"172.21.15.133:6835","nonce":4067860709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5100,"qdb_cluster":[5100]},"id":2}]} 2024-03-07T06:52:41.713 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 18 2024-03-07T06:52:41.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.713+0000 7f725a600640 1 -- 172.21.15.133:0/555975277 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f723006b4d0 msgr2=0x7f723006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:41.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.713+0000 7f725a600640 1 --2- 172.21.15.133:0/555975277 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f723006b4d0 0x7f723006d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f723c007fd0 tx=0x7f723c009040 comp rx=0 tx=0).stop 2024-03-07T06:52:41.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.713+0000 7f725a600640 1 -- 172.21.15.133:0/555975277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72540cdee0 msgr2=0x7f725409a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:41.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.713+0000 7f725a600640 1 --2- 172.21.15.133:0/555975277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72540cdee0 0x7f725409a9d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f724400d620 tx=0x7f724400dae0 comp rx=0 tx=0).stop 2024-03-07T06:52:41.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.713+0000 7f725a600640 1 -- 172.21.15.133:0/555975277 shutdown_connections 2024-03-07T06:52:41.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.713+0000 7f725a600640 1 --2- 172.21.15.133:0/555975277 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72541f5820 0x7f725409b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:41.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.713+0000 7f725a600640 1 --2- 172.21.15.133:0/555975277 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f723006b4d0 0x7f723006d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:41.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.713+0000 7f725a600640 1 --2- 172.21.15.133:0/555975277 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f725415fbb0 0x7f725409ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:41.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.713+0000 7f725a600640 1 --2- 172.21.15.133:0/555975277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72540cdee0 0x7f725409a9d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:41.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.713+0000 7f725a600640 1 -- 172.21.15.133:0/555975277 >> 172.21.15.133:0/555975277 conn(0x7f72540b3120 msgr2=0x7f7254074150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:41.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.714+0000 7f725a600640 1 -- 172.21.15.133:0/555975277 shutdown_connections 2024-03-07T06:52:41.714 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:41.714+0000 7f725a600640 1 -- 172.21.15.133:0/555975277 wait complete. 2024-03-07T06:52:42.134 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-07T06:52:35.182567+0000', 'modified': '2024-03-07T06:52:38.268724+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5100}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5100': {'gid': 5100, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.133:6835/4067860709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6833', 'nonce': 4067860709}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 4067860709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5100, 'qdb_cluster': [5100]} 2024-03-07T06:52:42.134 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-07T06:52:42.134 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-07T06:52:42.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.348+0000 7f058e800640 1 Processor -- start 2024-03-07T06:52:42.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.348+0000 7f058e800640 1 -- start start 2024-03-07T06:52:42.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.350+0000 7f058e800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f05880cf5d0 0x7f0588072830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.351+0000 7f058e800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0588072d70 0x7f058815c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.351+0000 7f058e800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0588160760 con 0x7f058815bd30 2024-03-07T06:52:42.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.351+0000 7f058e800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05880731c0 con 0x7f05880cf5d0 2024-03-07T06:52:42.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.351+0000 7f058e800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058815f9a0 con 0x7f0588072d70 2024-03-07T06:52:42.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.351+0000 7f058ca00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f05880cf5d0 0x7f0588072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.351+0000 7f058ca00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f05880cf5d0 0x7f0588072830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47350/0 (socket says 172.21.15.133:47350) 2024-03-07T06:52:42.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.351+0000 7f058ca00640 1 -- 172.21.15.133:0/488705818 learned_addr learned my addr 172.21.15.133:0/488705818 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:42.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.351+0000 7f058ca00640 1 -- 172.21.15.133:0/488705818 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0588072d70 msgr2=0x7f058815c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:42.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.351+0000 7f058ca00640 1 --2- 172.21.15.133:0/488705818 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0588072d70 0x7f058815c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.351+0000 7f058ca00640 1 -- 172.21.15.133:0/488705818 >> v1:172.21.15.161:6789/0 conn(0x7f058815bd30 legacy=0x7f058806e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:42.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.351+0000 7f058ca00640 1 -- 172.21.15.133:0/488705818 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058815fc80 con 0x7f05880cf5d0 2024-03-07T06:52:42.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.352+0000 7f058ca00640 1 --2- 172.21.15.133:0/488705818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05880cf5d0 0x7f0588072830 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0574009870 tx=0x7f057402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e263830cdcd70177 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:42.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.353+0000 7f0587e00640 1 -- 172.21.15.133:0/488705818 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0574035020 con 0x7f05880cf5d0 2024-03-07T06:52:42.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.353+0000 7f0587e00640 1 -- 172.21.15.133:0/488705818 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0574031640 con 0x7f05880cf5d0 2024-03-07T06:52:42.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.353+0000 7f0587e00640 1 -- 172.21.15.133:0/488705818 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057402d040 con 0x7f05880cf5d0 2024-03-07T06:52:42.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.353+0000 7f058e800640 1 -- 172.21.15.133:0/488705818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05880cf5d0 msgr2=0x7f0588072830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.353+0000 7f058e800640 1 --2- 172.21.15.133:0/488705818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05880cf5d0 0x7f0588072830 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0574009870 tx=0x7f057402b8f0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.353+0000 7f058e800640 1 -- 172.21.15.133:0/488705818 shutdown_connections 2024-03-07T06:52:42.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.353+0000 7f058e800640 1 --2- 172.21.15.133:0/488705818 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0588072d70 0x7f058815c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.353+0000 7f058e800640 1 --2- 172.21.15.133:0/488705818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f05880cf5d0 0x7f0588072830 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.353+0000 7f058e800640 1 -- 172.21.15.133:0/488705818 >> 172.21.15.133:0/488705818 conn(0x7f05880b2e90 msgr2=0x7f05880b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:42.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.353+0000 7f058e800640 1 -- 172.21.15.133:0/488705818 shutdown_connections 2024-03-07T06:52:42.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.353+0000 7f058e800640 1 -- 172.21.15.133:0/488705818 wait complete. 2024-03-07T06:52:42.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.355+0000 7f058e800640 1 Processor -- start 2024-03-07T06:52:42.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.355+0000 7f058e800640 1 -- start start 2024-03-07T06:52:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.356+0000 7f058e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0588072d70 0x7f05881537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.356+0000 7f058e800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05880cf5d0 0x7f058815e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.356+0000 7f058e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f058815bd30 0x7f058815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.356+0000 7f058e800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058806f130 con 0x7f058815bd30 2024-03-07T06:52:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.356+0000 7f058e800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0588160240 con 0x7f0588072d70 2024-03-07T06:52:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.356+0000 7f058e800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058815c650 con 0x7f05880cf5d0 2024-03-07T06:52:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.356+0000 7f058de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f058815bd30 0x7f058815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.356+0000 7f058de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f058815bd30 0x7f058815ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47358/0 (socket says 172.21.15.133:47358) 2024-03-07T06:52:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.356+0000 7f058de00640 1 -- 172.21.15.133:0/3344204618 learned_addr learned my addr 172.21.15.133:0/3344204618 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.357+0000 7f058ca00640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05880cf5d0 0x7f058815e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.357+0000 7f058d400640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0588072d70 0x7f05881537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.357+0000 7f058de00640 1 -- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05880cf5d0 msgr2=0x7f058815e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.357+0000 7f058de00640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05880cf5d0 0x7f058815e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.357+0000 7f058de00640 1 -- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0588072d70 msgr2=0x7f05881537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.357+0000 7f058de00640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0588072d70 0x7f05881537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.357+0000 7f058de00640 1 -- 172.21.15.133:0/3344204618 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05881f76c0 con 0x7f058815bd30 2024-03-07T06:52:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.357+0000 7f058d400640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0588072d70 0x7f05881537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.357+0000 7f058de00640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f058815bd30 0x7f058815ece0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f058000b970 tx=0x7f058000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.357+0000 7f0585600640 1 -- 172.21.15.133:0/3344204618 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580019070 con 0x7f058815bd30 2024-03-07T06:52:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.357+0000 7f0585600640 1 -- 172.21.15.133:0/3344204618 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058000cb60 con 0x7f058815bd30 2024-03-07T06:52:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.357+0000 7f0585600640 1 -- 172.21.15.133:0/3344204618 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580014440 con 0x7f058815bd30 2024-03-07T06:52:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.357+0000 7f058e800640 1 -- 172.21.15.133:0/3344204618 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05881f79a0 con 0x7f058815bd30 2024-03-07T06:52:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.358+0000 7f058e800640 1 -- 172.21.15.133:0/3344204618 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05881f7f60 con 0x7f058815bd30 2024-03-07T06:52:42.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.360+0000 7f0585600640 1 -- 172.21.15.133:0/3344204618 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f058001c070 con 0x7f058815bd30 2024-03-07T06:52:42.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.361+0000 7f0585600640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f056806b4d0 0x7f056806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.361+0000 7f0585600640 1 -- 172.21.15.133:0/3344204618 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0580089f50 con 0x7f058815bd30 2024-03-07T06:52:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.361+0000 7f058d400640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f056806b4d0 0x7f056806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.362+0000 7f058d400640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f056806b4d0 0x7f056806d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f057c00a810 tx=0x7f057c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.363+0000 7f0592e2c740 1 Processor -- start 2024-03-07T06:52:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.364+0000 7f0592e2c740 1 -- start start 2024-03-07T06:52:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.365+0000 7f0592e2c740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df293b47e0 0x55df29387780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.365+0000 7f0592e2c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b4bc0 0x55df293b7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.365+0000 7f0592e2c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df29387cc0 0x55df293b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.365+0000 7f0592e2c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df292f16c0 con 0x55df29387cc0 2024-03-07T06:52:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.365+0000 7f0592e2c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2929a9b0 con 0x55df293b4bc0 2024-03-07T06:52:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.365+0000 7f0592e2c740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df292ed760 con 0x55df293b47e0 2024-03-07T06:52:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.365+0000 7f058de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df29387cc0 0x55df293b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.365+0000 7f058de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df29387cc0 0x55df293b7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47376/0 (socket says 172.21.15.133:47376) 2024-03-07T06:52:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.365+0000 7f058de00640 1 -- 172.21.15.133:0/163142676 learned_addr learned my addr 172.21.15.133:0/163142676 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:42.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.366+0000 7f058d400640 1 --2- 172.21.15.133:0/163142676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df293b47e0 0x55df29387780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.366+0000 7f058ca00640 1 --2- 172.21.15.133:0/163142676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b4bc0 0x55df293b7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.366+0000 7f058de00640 1 -- 172.21.15.133:0/163142676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df293b47e0 msgr2=0x55df29387780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.366+0000 7f058de00640 1 --2- 172.21.15.133:0/163142676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df293b47e0 0x55df29387780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.366+0000 7f058de00640 1 -- 172.21.15.133:0/163142676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b4bc0 msgr2=0x55df293b7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.366+0000 7f058de00640 1 --2- 172.21.15.133:0/163142676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b4bc0 0x55df293b7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.366+0000 7f058de00640 1 -- 172.21.15.133:0/163142676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df2928e6f0 con 0x55df29387cc0 2024-03-07T06:52:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.366+0000 7f058ca00640 1 --2- 172.21.15.133:0/163142676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b4bc0 0x55df293b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.366+0000 7f058de00640 1 --2- 172.21.15.133:0/163142676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df29387cc0 0x55df293b7ab0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f05800608a0 tx=0x7f05800586c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4087e12e2f82ea16 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.366+0000 7f0579c00640 1 -- 172.21.15.133:0/163142676 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05800569c0 con 0x55df29387cc0 2024-03-07T06:52:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.366+0000 7f0579c00640 1 -- 172.21.15.133:0/163142676 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0580070170 con 0x55df29387cc0 2024-03-07T06:52:42.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.366+0000 7f0579c00640 1 -- 172.21.15.133:0/163142676 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580060940 con 0x55df29387cc0 2024-03-07T06:52:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.367+0000 7f0592e2c740 1 -- 172.21.15.133:0/163142676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df29387cc0 msgr2=0x55df293b7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.367+0000 7f0592e2c740 1 --2- 172.21.15.133:0/163142676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df29387cc0 0x55df293b7ab0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f05800608a0 tx=0x7f05800586c0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.367+0000 7f0592e2c740 1 -- 172.21.15.133:0/163142676 shutdown_connections 2024-03-07T06:52:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.367+0000 7f0592e2c740 1 --2- 172.21.15.133:0/163142676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df29387cc0 0x55df293b7ab0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.367+0000 7f0592e2c740 1 --2- 172.21.15.133:0/163142676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b4bc0 0x55df293b7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.367+0000 7f0592e2c740 1 --2- 172.21.15.133:0/163142676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df293b47e0 0x55df29387780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.367+0000 7f0592e2c740 1 -- 172.21.15.133:0/163142676 >> 172.21.15.133:0/163142676 conn(0x55df293899f0 msgr2=0x55df293d2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.367+0000 7f0592e2c740 1 -- 172.21.15.133:0/163142676 shutdown_connections 2024-03-07T06:52:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.367+0000 7f0592e2c740 1 -- 172.21.15.133:0/163142676 wait complete. 2024-03-07T06:52:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.369+0000 7f0592e2c740 1 -- start start 2024-03-07T06:52:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.369+0000 7f0592e2c740 1 Processor -- start 2024-03-07T06:52:42.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.370+0000 7f0592e2c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b47e0 0x55df293a1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f0592e2c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df2939bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f0592e2c740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df2939e990 0x55df293ba8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f0592e2c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2938bf90 con 0x55df293b4bc0 2024-03-07T06:52:42.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f0592e2c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df292addf0 con 0x55df293b47e0 2024-03-07T06:52:42.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f0592e2c740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df2938b380 con 0x55df2939e990 2024-03-07T06:52:42.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f0592e2c740 10 client.0 _ll_register_callbacks cb 0x55df292e88a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T06:52:42.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f058ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df2939bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f058ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df2939bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47378/0 (socket says 172.21.15.133:47378) 2024-03-07T06:52:42.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f058ca00640 1 -- 172.21.15.133:0/1565747469 learned_addr learned my addr 172.21.15.133:0/1565747469 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f058d400640 1 --2- 172.21.15.133:0/1565747469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b47e0 0x55df293a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f058de00640 1 --2- 172.21.15.133:0/1565747469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df2939e990 0x55df293ba8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f058ca00640 1 -- 172.21.15.133:0/1565747469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df2939e990 msgr2=0x55df293ba8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f058ca00640 1 --2- 172.21.15.133:0/1565747469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df2939e990 0x55df293ba8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f058ca00640 1 -- 172.21.15.133:0/1565747469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b47e0 msgr2=0x55df293a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f058ca00640 1 --2- 172.21.15.133:0/1565747469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b47e0 0x55df293a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.371+0000 7f058ca00640 1 -- 172.21.15.133:0/1565747469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df290062d0 con 0x55df293b4bc0 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.372+0000 7f058ca00640 1 --2- 172.21.15.133:0/1565747469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df2939bcd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0574000c00 tx=0x7f057402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1d4f5ede17e636 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.372+0000 7f056e000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.372+0000 7f056e000640 1 -- 172.21.15.133:0/1565747469 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057404a060 con 0x55df293b4bc0 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.372+0000 7f056e000640 1 -- 172.21.15.133:0/1565747469 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0574045670 con 0x55df293b4bc0 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.372+0000 7f0592e2c740 1 -- 172.21.15.133:0/1565747469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0580070170 con 0x55df293b4bc0 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.372+0000 7f056e000640 1 -- 172.21.15.133:0/1565747469 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0574050050 con 0x55df293b4bc0 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.372+0000 7f0592e2c740 10 client.5298 fetch_fsmap learned FSMap version 18 2024-03-07T06:52:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.372+0000 7f0592e2c740 1 -- 172.21.15.133:0/1565747469 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55df290d06b0 con 0x55df293b4bc0 2024-03-07T06:52:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.373+0000 7f056e000640 1 -- 172.21.15.133:0/1565747469 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0580070170 con 0x55df293b4bc0 2024-03-07T06:52:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.373+0000 7f056e000640 1 -- 172.21.15.133:0/1565747469 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 18) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f0574053020 con 0x55df293b4bc0 2024-03-07T06:52:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.373+0000 7f0592e2c740 10 client.5298 fetch_fsmap finished waiting for FSMap version 18 2024-03-07T06:52:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.373+0000 7f0592e2c740 10 client.5298 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-07T06:52:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.373+0000 7f0592e2c740 20 client.5298 populate_metadata read hostname 'smithi133' 2024-03-07T06:52:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.373+0000 7f0592e2c740 1 --2- 172.21.15.133:0/1565747469 >> [v2:172.21.15.133:6833/4067860709,v1:172.21.15.133:6835/4067860709] conn(0x55df293b0080 0x55df293f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.373+0000 7f0592e2c740 4 client.5298 mds_command: new command op to 5100 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T06:52:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.373+0000 7f0592e2c740 1 -- 172.21.15.133:0/1565747469 --> [v2:172.21.15.133:6833/4067860709,v1:172.21.15.133:6835/4067860709] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df292fec00 con 0x55df293b0080 2024-03-07T06:52:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.373+0000 7f058d400640 1 --2- 172.21.15.133:0/1565747469 >> [v2:172.21.15.133:6833/4067860709,v1:172.21.15.133:6835/4067860709] conn(0x55df293b0080 0x55df293f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.374+0000 7f058d400640 1 --2- 172.21.15.133:0/1565747469 >> [v2:172.21.15.133:6833/4067860709,v1:172.21.15.133:6835/4067860709] conn(0x55df293b0080 0x55df293f0d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b24cbda2cf62758 server_cookie=6f0ce3c332485985 in_seq=0 out_seq=0 2024-03-07T06:52:42.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.374+0000 7f056e000640 10 client.5298 ms_handle_connect on v2:172.21.15.133:6833/4067860709 2024-03-07T06:52:42.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f056e000640 1 -- 172.21.15.133:0/1565747469 <== mds.0 v2:172.21.15.133:6833/4067860709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55df292fec00 con 0x55df293b0080 2024-03-07T06:52:42.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f056e000640 10 client.5298 handle_command_reply: tid=0 2024-03-07T06:52:42.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f0592e2c740 1 client.5298 shutdown 2024-03-07T06:52:42.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f0592e2c740 1 -- 172.21.15.133:0/1565747469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 msgr2=0x55df2939bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f0592e2c740 1 --2- 172.21.15.133:0/1565747469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df2939bcd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0574000c00 tx=0x7f057402e000 comp rx=0 tx=0).stop 2024-03-07T06:52:42.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f0592e2c740 1 -- 172.21.15.133:0/1565747469 shutdown_connections 2024-03-07T06:52:42.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f0592e2c740 1 --2- 172.21.15.133:0/1565747469 >> [v2:172.21.15.133:6833/4067860709,v1:172.21.15.133:6835/4067860709] conn(0x55df293b0080 0x55df293f0d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f0592e2c740 1 --2- 172.21.15.133:0/1565747469 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df2939e990 0x55df293ba8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f0592e2c740 1 --2- 172.21.15.133:0/1565747469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df2939bcd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f0592e2c740 1 --2- 172.21.15.133:0/1565747469 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b47e0 0x55df293a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f056e000640 0 client.5298 ms_handle_reset on v2:172.21.15.133:6833/4067860709 2024-03-07T06:52:42.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f0592e2c740 1 -- 172.21.15.133:0/1565747469 >> 172.21.15.133:0/1565747469 conn(0x55df29387cc0 msgr2=0x55df293d5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:42.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f0592e2c740 1 -- 172.21.15.133:0/1565747469 shutdown_connections 2024-03-07T06:52:42.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f0592e2c740 1 -- 172.21.15.133:0/1565747469 wait complete. 2024-03-07T06:52:42.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.375+0000 7f0592e2c740 20 client.5298 trim_cache size 0 max 16384 2024-03-07T06:52:42.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.393+0000 7f0592e2c740 1 Processor -- start 2024-03-07T06:52:42.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.393+0000 7f0592e2c740 1 -- start start 2024-03-07T06:52:42.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.394+0000 7f0592e2c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df2939e990 0x55df293b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.394+0000 7f0592e2c740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df293b47e0 0x55df2938b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.394+0000 7f0592e2c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df2938b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.394+0000 7f0592e2c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05882003b0 con 0x55df293b4bc0 2024-03-07T06:52:42.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.394+0000 7f0592e2c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058815dfc0 con 0x55df2939e990 2024-03-07T06:52:42.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.394+0000 7f0592e2c740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05880cd650 con 0x55df293b47e0 2024-03-07T06:52:42.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.395+0000 7f058de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df2938b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.395+0000 7f058de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df2938b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47390/0 (socket says 172.21.15.133:47390) 2024-03-07T06:52:42.394 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.395+0000 7f058de00640 1 -- 172.21.15.133:0/598272108 learned_addr learned my addr 172.21.15.133:0/598272108 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:42.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.395+0000 7f058de00640 1 -- 172.21.15.133:0/598272108 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df293b47e0 msgr2=0x55df2938b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:42.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.395+0000 7f058de00640 1 --2- 172.21.15.133:0/598272108 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df293b47e0 0x55df2938b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.395+0000 7f058de00640 1 -- 172.21.15.133:0/598272108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df2939e990 msgr2=0x55df293b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:42.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.395+0000 7f058de00640 1 --2- 172.21.15.133:0/598272108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df2939e990 0x55df293b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.395+0000 7f058de00640 1 -- 172.21.15.133:0/598272108 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df2938eba0 con 0x55df293b4bc0 2024-03-07T06:52:42.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.395+0000 7f058de00640 1 --2- 172.21.15.133:0/598272108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df2938b820 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0580088470 tx=0x7f058001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52bfb67e05fab17 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:42.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.395+0000 7f0579200640 1 -- 172.21.15.133:0/598272108 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580054960 con 0x55df293b4bc0 2024-03-07T06:52:42.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.396+0000 7f0579200640 1 -- 172.21.15.133:0/598272108 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0580085020 con 0x55df293b4bc0 2024-03-07T06:52:42.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.396+0000 7f0579200640 1 -- 172.21.15.133:0/598272108 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05800569c0 con 0x55df293b4bc0 2024-03-07T06:52:42.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.396+0000 7f0592e2c740 1 -- 172.21.15.133:0/598272108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 msgr2=0x55df2938b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.396+0000 7f0592e2c740 1 --2- 172.21.15.133:0/598272108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df2938b820 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0580088470 tx=0x7f058001f000 comp rx=0 tx=0).stop 2024-03-07T06:52:42.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.396+0000 7f0592e2c740 1 -- 172.21.15.133:0/598272108 shutdown_connections 2024-03-07T06:52:42.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.396+0000 7f0592e2c740 1 --2- 172.21.15.133:0/598272108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df2938b820 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.396+0000 7f0592e2c740 1 --2- 172.21.15.133:0/598272108 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df293b47e0 0x55df2938b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.396+0000 7f0592e2c740 1 --2- 172.21.15.133:0/598272108 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df2939e990 0x55df293b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.396+0000 7f0592e2c740 1 -- 172.21.15.133:0/598272108 >> 172.21.15.133:0/598272108 conn(0x55df29387cc0 msgr2=0x55df2938d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:42.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.396+0000 7f0592e2c740 1 -- 172.21.15.133:0/598272108 shutdown_connections 2024-03-07T06:52:42.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.396+0000 7f0592e2c740 1 -- 172.21.15.133:0/598272108 wait complete. 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.397+0000 7f0592e2c740 1 -- start start 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.397+0000 7f0592e2c740 1 Processor -- start 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.398+0000 7f0592e2c740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df2939e990 0x55df29441800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.398+0000 7f0592e2c740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b47e0 0x55df293b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.398+0000 7f0592e2c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df293b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.398+0000 7f0592e2c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df293b9ae0 con 0x55df293b4bc0 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.398+0000 7f0592e2c740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f058815d9c0 con 0x55df293b47e0 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.398+0000 7f0592e2c740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df293ba140 con 0x55df2939e990 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.398+0000 7f0592e2c740 10 client.0 _ll_register_callbacks cb 0x7f05740096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.398+0000 7f058de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df293b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.398+0000 7f058de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df293b74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47406/0 (socket says 172.21.15.133:47406) 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.398+0000 7f058de00640 1 -- 172.21.15.133:0/793685262 learned_addr learned my addr 172.21.15.133:0/793685262 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.399+0000 7f058de00640 1 -- 172.21.15.133:0/793685262 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df2939e990 msgr2=0x55df29441800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.399+0000 7f058de00640 1 --2- 172.21.15.133:0/793685262 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df2939e990 0x55df29441800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.399+0000 7f058ca00640 1 --2- 172.21.15.133:0/793685262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b47e0 0x55df293b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.399+0000 7f058de00640 1 -- 172.21.15.133:0/793685262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b47e0 msgr2=0x55df293b6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.399+0000 7f058de00640 1 --2- 172.21.15.133:0/793685262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b47e0 0x55df293b6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.399+0000 7f058de00640 1 -- 172.21.15.133:0/793685262 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df293f0d00 con 0x55df293b4bc0 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.399+0000 7f058de00640 1 --2- 172.21.15.133:0/793685262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df293b74f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f058000b940 tx=0x7f0580004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa9b7150dc11542 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.399+0000 7f056e000640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.399+0000 7f056e000640 1 -- 172.21.15.133:0/793685262 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0580075060 con 0x55df293b4bc0 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.399+0000 7f056e000640 1 -- 172.21.15.133:0/793685262 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0580070170 con 0x55df293b4bc0 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.399+0000 7f0592e2c740 1 -- 172.21.15.133:0/793685262 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0580085020 con 0x55df293b4bc0 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.400+0000 7f056e000640 1 -- 172.21.15.133:0/793685262 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f058007c050 con 0x55df293b4bc0 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.400+0000 7f0592e2c740 10 client.5304 fetch_fsmap learned FSMap version 18 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.400+0000 7f0592e2c740 1 -- 172.21.15.133:0/793685262 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55df293f0f10 con 0x55df293b4bc0 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.400+0000 7f056e000640 1 -- 172.21.15.133:0/793685262 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0580085020 con 0x55df293b4bc0 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.400+0000 7f056e000640 1 -- 172.21.15.133:0/793685262 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 18) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f0580060940 con 0x55df293b4bc0 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.400+0000 7f0592e2c740 10 client.5304 fetch_fsmap finished waiting for FSMap version 18 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.400+0000 7f0592e2c740 10 client.5304 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.400+0000 7f0592e2c740 20 client.5304 populate_metadata read hostname 'smithi133' 2024-03-07T06:52:42.401 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.400+0000 7f0592e2c740 1 --2- 172.21.15.133:0/793685262 >> [v2:172.21.15.133:6833/4067860709,v1:172.21.15.133:6835/4067860709] conn(0x55df293e1ca0 0x55df293b8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.400+0000 7f0592e2c740 4 client.5304 mds_command: new command op to 5100 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-07T06:52:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.400+0000 7f0592e2c740 1 -- 172.21.15.133:0/793685262 --> [v2:172.21.15.133:6833/4067860709,v1:172.21.15.133:6835/4067860709] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55df292f8f50 con 0x55df293e1ca0 2024-03-07T06:52:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.400+0000 7f058d400640 1 --2- 172.21.15.133:0/793685262 >> [v2:172.21.15.133:6833/4067860709,v1:172.21.15.133:6835/4067860709] conn(0x55df293e1ca0 0x55df293b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f058d400640 1 --2- 172.21.15.133:0/793685262 >> [v2:172.21.15.133:6833/4067860709,v1:172.21.15.133:6835/4067860709] conn(0x55df293e1ca0 0x55df293b8530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c16303019132e65 server_cookie=10d49a11a84ee45 in_seq=0 out_seq=0 2024-03-07T06:52:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f056e000640 10 client.5304 ms_handle_connect on v2:172.21.15.133:6833/4067860709 2024-03-07T06:52:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f056e000640 1 -- 172.21.15.133:0/793685262 <== mds.0 v2:172.21.15.133:6833/4067860709 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55df292f8f50 con 0x55df293e1ca0 2024-03-07T06:52:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f056e000640 10 client.5304 handle_command_reply: tid=0 2024-03-07T06:52:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f0592e2c740 1 client.5304 shutdown 2024-03-07T06:52:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f0592e2c740 1 -- 172.21.15.133:0/793685262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 msgr2=0x55df293b74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f0592e2c740 1 --2- 172.21.15.133:0/793685262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df293b74f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f058000b940 tx=0x7f0580004a40 comp rx=0 tx=0).stop 2024-03-07T06:52:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f0592e2c740 1 -- 172.21.15.133:0/793685262 shutdown_connections 2024-03-07T06:52:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f0592e2c740 1 --2- 172.21.15.133:0/793685262 >> [v2:172.21.15.133:6833/4067860709,v1:172.21.15.133:6835/4067860709] conn(0x55df293e1ca0 0x55df293b8530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f0592e2c740 1 --2- 172.21.15.133:0/793685262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df293b4bc0 0x55df293b74f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f0592e2c740 1 --2- 172.21.15.133:0/793685262 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55df293b47e0 0x55df293b6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f0592e2c740 1 --2- 172.21.15.133:0/793685262 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55df2939e990 0x55df29441800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f0592e2c740 1 -- 172.21.15.133:0/793685262 >> 172.21.15.133:0/793685262 conn(0x55df29387cc0 msgr2=0x55df293ac280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:42.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f056e000640 0 client.5304 ms_handle_reset on v2:172.21.15.133:6833/4067860709 2024-03-07T06:52:42.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.401+0000 7f0592e2c740 1 -- 172.21.15.133:0/793685262 shutdown_connections 2024-03-07T06:52:42.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.402+0000 7f0592e2c740 1 -- 172.21.15.133:0/793685262 wait complete. 2024-03-07T06:52:42.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.402+0000 7f0592e2c740 20 client.5304 trim_cache size 0 max 16384 2024-03-07T06:52:42.404 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:52:42.404 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "1655279d-73c2-44d4-ba1b-3e8a15ae005a", 2024-03-07T06:52:42.404 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-03-07T06:52:42.404 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 5100, 2024-03-07T06:52:42.404 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-03-07T06:52:42.405 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-03-07T06:52:42.405 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-03-07T06:52:42.405 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 6.6394885730000004, 2024-03-07T06:52:42.405 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 18, 2024-03-07T06:52:42.405 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 35, 2024-03-07T06:52:42.405 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 35, 2024-03-07T06:52:42.405 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 22.551479289 2024-03-07T06:52:42.405 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:52:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.402+0000 7f058e800640 1 -- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f056806b4d0 msgr2=0x7f056806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.402+0000 7f058e800640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f056806b4d0 0x7f056806d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f057c00a810 tx=0x7f057c008040 comp rx=0 tx=0).stop 2024-03-07T06:52:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.402+0000 7f058e800640 1 -- 172.21.15.133:0/3344204618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f058815bd30 msgr2=0x7f058815ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.402+0000 7f058e800640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f058815bd30 0x7f058815ece0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f058000b970 tx=0x7f058000be30 comp rx=0 tx=0).stop 2024-03-07T06:52:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.402+0000 7f058e800640 1 -- 172.21.15.133:0/3344204618 shutdown_connections 2024-03-07T06:52:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.402+0000 7f058e800640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f058815bd30 0x7f058815ece0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.402+0000 7f058e800640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05880cf5d0 0x7f058815e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.402+0000 7f058e800640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f056806b4d0 0x7f056806d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.402+0000 7f058e800640 1 --2- 172.21.15.133:0/3344204618 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0588072d70 0x7f05881537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.402+0000 7f058e800640 1 -- 172.21.15.133:0/3344204618 >> 172.21.15.133:0/3344204618 conn(0x7f05880b2e90 msgr2=0x7f058806d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.402+0000 7f058e800640 1 -- 172.21.15.133:0/3344204618 shutdown_connections 2024-03-07T06:52:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.403+0000 7f058e800640 1 -- 172.21.15.133:0/3344204618 wait complete. 2024-03-07T06:52:42.419 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-07T06:52:42.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.613+0000 7f6e10e00640 1 Processor -- start 2024-03-07T06:52:42.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.613+0000 7f6e10e00640 1 -- start start 2024-03-07T06:52:42.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.614+0000 7f6e10e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6e0c071410 0x7f6e0c071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.614+0000 7f6e10e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0c162c80 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.614+0000 7f6e10e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0c0b4bd0 con 0x7f6e0c144540 2024-03-07T06:52:42.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.614+0000 7f6e10e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e0c141700 con 0x7f6e0c071410 2024-03-07T06:52:42.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.615+0000 7f6e0a800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6e0c071410 0x7f6e0c071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.615+0000 7f6e09e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6e0c0b5dd0 0x7f6e0c0b43f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33882/0 (socket says 172.21.15.133:33882) 2024-03-07T06:52:42.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.615+0000 7f6e09e00640 1 -- 172.21.15.133:0/1418193467 learned_addr learned my addr 172.21.15.133:0/1418193467 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:42.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.615+0000 7f6e0a800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6e0c071410 0x7f6e0c071880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47410/0 (socket says 172.21.15.133:47410) 2024-03-07T06:52:42.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.615+0000 7f6e0a800640 1 -- 172.21.15.133:0/1418193467 >> v1:172.21.15.161:6789/0 conn(0x7f6e0c144540 legacy=0x7f6e0c162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:42.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.615+0000 7f6e0a800640 1 -- 172.21.15.133:0/1418193467 >> v1:172.21.15.133:6789/0 conn(0x7f6e0c0b5dd0 legacy=0x7f6e0c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.615+0000 7f6e0a800640 1 -- 172.21.15.133:0/1418193467 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c0b4eb0 con 0x7f6e0c071410 2024-03-07T06:52:42.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.615+0000 7f6e08a00640 1 -- 172.21.15.133:0/1418193467 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888773287 0 0) 0x7f6e0c162c80 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.617+0000 7f6e0a800640 1 --2- 172.21.15.133:0/1418193467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e0c071410 0x7f6e0c071880 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009d90 tx=0x7f6e0402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3574c0823276b2c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:42.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.617+0000 7f6e08a00640 1 -- 172.21.15.133:0/1418193467 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04039070 con 0x7f6e0c071410 2024-03-07T06:52:42.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.617+0000 7f6e08a00640 1 -- 172.21.15.133:0/1418193467 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0402cc60 con 0x7f6e0c071410 2024-03-07T06:52:42.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.617+0000 7f6e08a00640 1 -- 172.21.15.133:0/1418193467 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04034660 con 0x7f6e0c071410 2024-03-07T06:52:42.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.617+0000 7f6e10e00640 1 -- 172.21.15.133:0/1418193467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e0c071410 msgr2=0x7f6e0c071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.617+0000 7f6e10e00640 1 --2- 172.21.15.133:0/1418193467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e0c071410 0x7f6e0c071880 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6e04009d90 tx=0x7f6e0402beb0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.618+0000 7f6e10e00640 1 -- 172.21.15.133:0/1418193467 shutdown_connections 2024-03-07T06:52:42.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.618+0000 7f6e10e00640 1 --2- 172.21.15.133:0/1418193467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e0c071410 0x7f6e0c071880 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.618+0000 7f6e10e00640 1 -- 172.21.15.133:0/1418193467 >> 172.21.15.133:0/1418193467 conn(0x7f6e0c0d0e70 msgr2=0x7f6e0c0d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:42.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.618+0000 7f6e10e00640 1 -- 172.21.15.133:0/1418193467 shutdown_connections 2024-03-07T06:52:42.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.618+0000 7f6e10e00640 1 -- 172.21.15.133:0/1418193467 wait complete. 2024-03-07T06:52:42.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.619+0000 7f6e10e00640 1 Processor -- start 2024-03-07T06:52:42.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.619+0000 7f6e10e00640 1 -- start start 2024-03-07T06:52:42.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.620+0000 7f6e10e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6e0c071410 0x7f6e0c152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.620+0000 7f6e10e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e0c0b5dd0 0x7f6e0c157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.620+0000 7f6e10e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e0c144540 0x7f6e0c0b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.620+0000 7f6e10e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c161e00 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.620+0000 7f6e10e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c071dc0 con 0x7f6e0c144540 2024-03-07T06:52:42.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.620+0000 7f6e10e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c162050 con 0x7f6e0c071410 2024-03-07T06:52:42.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.621+0000 7f6e09400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e0c0b5dd0 0x7f6e0c157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.621+0000 7f6e09400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e0c0b5dd0 0x7f6e0c157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47420/0 (socket says 172.21.15.133:47420) 2024-03-07T06:52:42.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.621+0000 7f6e09400640 1 -- 172.21.15.133:0/1598924070 learned_addr learned my addr 172.21.15.133:0/1598924070 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:42.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.621+0000 7f6e09400640 1 -- 172.21.15.133:0/1598924070 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6e0c071410 msgr2=0x7f6e0c152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.621+0000 7f6e09400640 1 --2- 172.21.15.133:0/1598924070 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6e0c071410 0x7f6e0c152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.621+0000 7f6e09400640 1 -- 172.21.15.133:0/1598924070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e0c144540 msgr2=0x7f6e0c0b8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.621+0000 7f6e09400640 1 --2- 172.21.15.133:0/1598924070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e0c144540 0x7f6e0c0b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.621+0000 7f6e09400640 1 -- 172.21.15.133:0/1598924070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c13dfd0 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.621+0000 7f6e09400640 1 --2- 172.21.15.133:0/1598924070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e0c0b5dd0 0x7f6e0c157e90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6df800c430 tx=0x7f6df800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:42.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.621+0000 7f6e02000640 1 -- 172.21.15.133:0/1598924070 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8017070 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.622+0000 7f6e02000640 1 -- 172.21.15.133:0/1598924070 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df80052d0 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.622+0000 7f6e02000640 1 -- 172.21.15.133:0/1598924070 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8003830 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.622+0000 7f6e10e00640 1 -- 172.21.15.133:0/1598924070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0c1569f0 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.622+0000 7f6e10e00640 1 -- 172.21.15.133:0/1598924070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0c156710 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.623+0000 7f6e10e00640 1 -- 172.21.15.133:0/1598924070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0c062c90 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.624+0000 7f6e02000640 1 -- 172.21.15.133:0/1598924070 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6df801a070 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.624+0000 7f6e02000640 1 --2- 172.21.15.133:0/1598924070 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6dec06b4d0 0x7f6dec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:42.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.625+0000 7f6e02000640 1 -- 172.21.15.133:0/1598924070 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6df8089580 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.625+0000 7f6e09e00640 1 --2- 172.21.15.133:0/1598924070 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6dec06b4d0 0x7f6dec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:42.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.626+0000 7f6e09e00640 1 --2- 172.21.15.133:0/1598924070 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6dec06b4d0 0x7f6dec06d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc007fd0 tx=0x7f6dfc007900 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:42.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.629+0000 7f6e02000640 1 -- 172.21.15.133:0/1598924070 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6df8058400 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.898 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.898+0000 7f6e10e00640 1 -- 172.21.15.133:0/1598924070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6e0c1611c0 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.899+0000 7f6e02000640 1 -- 172.21.15.133:0/1598924070 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6df8057d00 con 0x7f6e0c0b5dd0 2024-03-07T06:52:42.899 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-07T06:52:42.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.901+0000 7f6e10e00640 1 -- 172.21.15.133:0/1598924070 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6dec06b4d0 msgr2=0x7f6dec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.901+0000 7f6e10e00640 1 --2- 172.21.15.133:0/1598924070 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6dec06b4d0 0x7f6dec06d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc007fd0 tx=0x7f6dfc007900 comp rx=0 tx=0).stop 2024-03-07T06:52:42.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.901+0000 7f6e10e00640 1 -- 172.21.15.133:0/1598924070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e0c0b5dd0 msgr2=0x7f6e0c157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:42.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.901+0000 7f6e10e00640 1 --2- 172.21.15.133:0/1598924070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e0c0b5dd0 0x7f6e0c157e90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6df800c430 tx=0x7f6df800c8f0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.902+0000 7f6e10e00640 1 -- 172.21.15.133:0/1598924070 shutdown_connections 2024-03-07T06:52:42.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.902+0000 7f6e10e00640 1 --2- 172.21.15.133:0/1598924070 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6dec06b4d0 0x7f6dec06d990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.902+0000 7f6e10e00640 1 --2- 172.21.15.133:0/1598924070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6e0c144540 0x7f6e0c0b8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.902+0000 7f6e10e00640 1 --2- 172.21.15.133:0/1598924070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e0c0b5dd0 0x7f6e0c157e90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.902+0000 7f6e10e00640 1 --2- 172.21.15.133:0/1598924070 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6e0c071410 0x7f6e0c152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:42.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.902+0000 7f6e10e00640 1 -- 172.21.15.133:0/1598924070 >> 172.21.15.133:0/1598924070 conn(0x7f6e0c0d0e70 msgr2=0x7f6e0c0d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:42.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.902+0000 7f6e10e00640 1 -- 172.21.15.133:0/1598924070 shutdown_connections 2024-03-07T06:52:42.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:42.902+0000 7f6e10e00640 1 -- 172.21.15.133:0/1598924070 wait complete. 2024-03-07T06:52:42.925 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-07T06:52:42.925 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-07T06:52:42.925 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-03-07T06:52:42.925 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-07T06:52:42.925 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-07T06:52:42.925 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-07T06:52:42.925 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-07T06:52:42.925 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-07T06:52:42.925 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-07T06:52:42.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:42.926 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-03-07T06:52:42.975 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T06:52:42.976 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T06:52:42.976 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T06:52:42.976 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:52:42.976 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-03-07T06:52:42.976 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:52:42.976 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:52:42.976 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:52:42.976 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-03-07T06:52:42.976 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:52:42.976 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:52:42.976 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42458sec preferred_lft 42458sec 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 6e:50:ef:52:0c:c1 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::4c8:8aff:feed:a7f5/64 scope link 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:52:42.977 INFO:teuthology.orchestra.run.smithi133.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-07T06:52:42.978 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 6e:50:ef:52:0c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-07T06:52:42.978 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::6c50:efff:fe52:cc1/64 scope link 2024-03-07T06:52:42.978 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:52:42.978 INFO:teuthology.orchestra.run.smithi133.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-07T06:52:42.978 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether f2:f2:9f:72:0c:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-07T06:52:42.978 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::f0f2:9fff:fe72:c1e/64 scope link 2024-03-07T06:52:42.978 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:52:42.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:42.979 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-07T06:52:43.060 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-03-07T06:52:43.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:43.064 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-03-07T06:52:43.118 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-03-07T06:52:43.118 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-07T06:52:43.119 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-07T06:52:43.119 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-07T06:52:43.119 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-07T06:52:43.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:43.119 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-07T06:52:43.119 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-07T06:52:43.119 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-07T06:52:43.119 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-07T06:52:43.119 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-07T06:52:43.119 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-07T06:52:43.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:43.313 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-03-07T06:52:43.360 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-07T06:52:43.361 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-07T06:52:43.361 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-07T06:52:43.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:43.361 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-03-07T06:52:43.490 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T06:52:43.490 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-07T06:52:43.541 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-07T06:52:43.542 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T06:52:43.542 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-07T06:52:43.587 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-07T06:52:43.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:43.588 DEBUG:teuthology.orchestra.run.smithi133:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:52:43.763 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:52:43.763 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T06:52:43.796 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:52:43.796 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T06:52:43.796 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: resolved to: "172.21.15.133:3300,172.21.15.161:3300,172.21.15.161:3301" 2024-03-07T06:52:43.796 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: trying mount with new device syntax: 0@1655279d-73c2-44d4-ba1b-3e8a15ae005a.cephfs=/ 2024-03-07T06:52:43.796 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.133:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2024-03-07T06:52:43.798 INFO:teuthology.orchestra.run.smithi133.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-07T06:52:43.798 INFO:teuthology.orchestra.run.smithi133.stdout: You just mounted a file system that supports labels which does not 2024-03-07T06:52:43.798 INFO:teuthology.orchestra.run.smithi133.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-07T06:52:43.798 INFO:teuthology.orchestra.run.smithi133.stdout: applications will generate AVC messages and not be allowed access to 2024-03-07T06:52:43.798 INFO:teuthology.orchestra.run.smithi133.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-07T06:52:43.802 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-07T06:52:43.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:43.803 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-07T06:52:43.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:43.882 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T06:52:43.941 INFO:teuthology.orchestra.run.smithi133.stdout:client5316 2024-03-07T06:52:43.941 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:52:43.941 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:52:43.986 DEBUG:teuthology.orchestra.run.smithi133:> stdin-killer -- python3 -c ' 2024-03-07T06:52:43.986 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-03-07T06:52:43.986 DEBUG:teuthology.orchestra.run.smithi133:> 2024-03-07T06:52:43.986 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-07T06:52:43.986 DEBUG:teuthology.orchestra.run.smithi133:> subdir = "delete_me" 2024-03-07T06:52:43.986 DEBUG:teuthology.orchestra.run.smithi133:> size = 1024 2024-03-07T06:52:43.986 DEBUG:teuthology.orchestra.run.smithi133:> file_count = 1000 2024-03-07T06:52:43.986 DEBUG:teuthology.orchestra.run.smithi133:> os.mkdir(os.path.join(mountpoint, subdir)) 2024-03-07T06:52:43.986 DEBUG:teuthology.orchestra.run.smithi133:> for i in range(0, file_count): 2024-03-07T06:52:43.986 DEBUG:teuthology.orchestra.run.smithi133:> filename = "{0}_{1}.bin".format(i, size) 2024-03-07T06:52:43.986 DEBUG:teuthology.orchestra.run.smithi133:> with open(os.path.join(mountpoint, subdir, filename), '"'"'w'"'"') as f: 2024-03-07T06:52:43.986 DEBUG:teuthology.orchestra.run.smithi133:> f.write(size * '"'"'x'"'"') 2024-03-07T06:52:43.987 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-03-07T06:52:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-07T06:52:45.158 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:52:45.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.408+0000 7f4792800640 1 Processor -- start 2024-03-07T06:52:45.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.408+0000 7f4792800640 1 -- start start 2024-03-07T06:52:45.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.409+0000 7f4792800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f478c15b1b0 0x7f478c0b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:45.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.409+0000 7f4792800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f478c0af950 0x7f478c0cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:45.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.409+0000 7f4792800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478c06f140 con 0x7f478c0ce420 2024-03-07T06:52:45.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.409+0000 7f4792800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f478c0ce870 con 0x7f478c15b1b0 2024-03-07T06:52:45.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.409+0000 7f4792800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478c071610 con 0x7f478c0af950 2024-03-07T06:52:45.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.410+0000 7f478b400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f478c15b1b0 0x7f478c0b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:45.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.410+0000 7f478b400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f478c15b1b0 0x7f478c0b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47426/0 (socket says 172.21.15.133:47426) 2024-03-07T06:52:45.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.410+0000 7f478b400640 1 -- 172.21.15.133:0/3076130827 learned_addr learned my addr 172.21.15.133:0/3076130827 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:45.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.410+0000 7f4783e00640 1 --2- 172.21.15.133:0/3076130827 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f478c0af950 0x7f478c0cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:45.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.410+0000 7f478b400640 1 -- 172.21.15.133:0/3076130827 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f478c0af950 msgr2=0x7f478c0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:45.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.410+0000 7f478b400640 1 --2- 172.21.15.133:0/3076130827 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f478c0af950 0x7f478c0cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:45.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.411+0000 7f478b400640 1 -- 172.21.15.133:0/3076130827 >> v1:172.21.15.161:6789/0 conn(0x7f478c0ce420 legacy=0x7f478c06de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:45.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.411+0000 7f478b400640 1 -- 172.21.15.133:0/3076130827 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c06f420 con 0x7f478c15b1b0 2024-03-07T06:52:45.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.412+0000 7f478b400640 1 --2- 172.21.15.133:0/3076130827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478c15b1b0 0x7f478c0b3c10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f477c009870 tx=0x7f477c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5f473820fe459a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:45.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.412+0000 7f478aa00640 1 -- 172.21.15.133:0/3076130827 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c0317c0 con 0x7f478c15b1b0 2024-03-07T06:52:45.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.412+0000 7f478aa00640 1 -- 172.21.15.133:0/3076130827 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477c037070 con 0x7f478c15b1b0 2024-03-07T06:52:45.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.412+0000 7f478aa00640 1 -- 172.21.15.133:0/3076130827 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477c031b80 con 0x7f478c15b1b0 2024-03-07T06:52:45.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.412+0000 7f4792800640 1 -- 172.21.15.133:0/3076130827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478c15b1b0 msgr2=0x7f478c0b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.412+0000 7f4792800640 1 --2- 172.21.15.133:0/3076130827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478c15b1b0 0x7f478c0b3c10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f477c009870 tx=0x7f477c02b860 comp rx=0 tx=0).stop 2024-03-07T06:52:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.413+0000 7f4792800640 1 -- 172.21.15.133:0/3076130827 shutdown_connections 2024-03-07T06:52:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.413+0000 7f4792800640 1 --2- 172.21.15.133:0/3076130827 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f478c0af950 0x7f478c0cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.413+0000 7f4792800640 1 --2- 172.21.15.133:0/3076130827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478c15b1b0 0x7f478c0b3c10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.413+0000 7f4792800640 1 -- 172.21.15.133:0/3076130827 >> 172.21.15.133:0/3076130827 conn(0x7f478c0b3120 msgr2=0x7f478c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.413+0000 7f4792800640 1 -- 172.21.15.133:0/3076130827 shutdown_connections 2024-03-07T06:52:45.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.413+0000 7f4792800640 1 -- 172.21.15.133:0/3076130827 wait complete. 2024-03-07T06:52:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.415+0000 7f4792800640 1 Processor -- start 2024-03-07T06:52:45.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.415+0000 7f4792800640 1 -- start start 2024-03-07T06:52:45.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f4792800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f478c0af950 0x7f478c147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:45.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f4792800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478c0ce420 0x7f478c1f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:45.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f4792800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c15b1b0 0x7f478c09a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:45.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f4792800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c15b590 con 0x7f478c0ce420 2024-03-07T06:52:45.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f4792800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0cc880 con 0x7f478c15b1b0 2024-03-07T06:52:45.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f4792800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478c15b710 con 0x7f478c0af950 2024-03-07T06:52:45.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f4783e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478c0ce420 0x7f478c1f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:45.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f4783e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478c0ce420 0x7f478c1f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47438/0 (socket says 172.21.15.133:47438) 2024-03-07T06:52:45.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f4783e00640 1 -- 172.21.15.133:0/3397630841 learned_addr learned my addr 172.21.15.133:0/3397630841 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:45.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f478b400640 1 --2- 172.21.15.133:0/3397630841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f478c0af950 0x7f478c147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:45.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f4783e00640 1 -- 172.21.15.133:0/3397630841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f478c0af950 msgr2=0x7f478c147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:45.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f4783e00640 1 --2- 172.21.15.133:0/3397630841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f478c0af950 0x7f478c147be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:45.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f4783e00640 1 -- 172.21.15.133:0/3397630841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c15b1b0 msgr2=0x7f478c09a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:45.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f4783e00640 1 --2- 172.21.15.133:0/3397630841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c15b1b0 0x7f478c09a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:45.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f4783e00640 1 -- 172.21.15.133:0/3397630841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478c09cb40 con 0x7f478c0ce420 2024-03-07T06:52:45.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.416+0000 7f478b400640 1 --2- 172.21.15.133:0/3397630841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f478c0af950 0x7f478c147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:45.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.417+0000 7f4783e00640 1 --2- 172.21.15.133:0/3397630841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478c0ce420 0x7f478c1f6480 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f477400d800 tx=0x7f477400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:45.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.417+0000 7f4783400640 1 -- 172.21.15.133:0/3397630841 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477400b6c0 con 0x7f478c0ce420 2024-03-07T06:52:45.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.417+0000 7f4783400640 1 -- 172.21.15.133:0/3397630841 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4774019070 con 0x7f478c0ce420 2024-03-07T06:52:45.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.417+0000 7f4792800640 1 -- 172.21.15.133:0/3397630841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478c09cde0 con 0x7f478c0ce420 2024-03-07T06:52:45.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.417+0000 7f4783400640 1 -- 172.21.15.133:0/3397630841 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477400ba80 con 0x7f478c0ce420 2024-03-07T06:52:45.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.417+0000 7f4792800640 1 -- 172.21.15.133:0/3397630841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478c09d3a0 con 0x7f478c0ce420 2024-03-07T06:52:45.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.418+0000 7f4792800640 1 -- 172.21.15.133:0/3397630841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478c062c90 con 0x7f478c0ce420 2024-03-07T06:52:45.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.419+0000 7f4783400640 1 -- 172.21.15.133:0/3397630841 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f477400bc20 con 0x7f478c0ce420 2024-03-07T06:52:45.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.420+0000 7f4783400640 1 --2- 172.21.15.133:0/3397630841 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f476806b4d0 0x7f476806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:45.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.420+0000 7f4783400640 1 -- 172.21.15.133:0/3397630841 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f477408a6f0 con 0x7f478c0ce420 2024-03-07T06:52:45.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.421+0000 7f478b400640 1 --2- 172.21.15.133:0/3397630841 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f476806b4d0 0x7f476806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:45.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.422+0000 7f478b400640 1 --2- 172.21.15.133:0/3397630841 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f476806b4d0 0x7f476806d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f477c036000 tx=0x7f477c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:45.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.425+0000 7f4783400640 1 -- 172.21.15.133:0/3397630841 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f477401c360 con 0x7f478c0ce420 2024-03-07T06:52:45.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.662+0000 7f4792800640 1 -- 172.21.15.133:0/3397630841 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f478c06e0d0 con 0x7f478c0ce420 2024-03-07T06:52:45.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.663+0000 7f4783400640 1 -- 172.21.15.133:0/3397630841 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4983 (secure 0 0 0) 0x7f4774059560 con 0x7f478c0ce420 2024-03-07T06:52:45.663 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:45.663 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3269394125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3269394125},{"type":"v1","addr":"172.21.15.161:6835","nonce":3269394125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/72243172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":72243172},{"type":"v1","addr":"172.21.15.161:6837","nonce":72243172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3714291615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3714291615},{"type":"v1","addr":"172.21.15.133:6834","nonce":3714291615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:52:35.182567+0000","modified":"2024-03-07T06:52:38.268724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6835/4067860709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":4067860709},{"type":"v1","addr":"172.21.15.133:6835","nonce":4067860709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5100,"qdb_cluster":[5100]},"id":2}]} 2024-03-07T06:52:45.663 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 18 2024-03-07T06:52:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.666+0000 7f4792800640 1 -- 172.21.15.133:0/3397630841 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f476806b4d0 msgr2=0x7f476806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.666+0000 7f4792800640 1 --2- 172.21.15.133:0/3397630841 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f476806b4d0 0x7f476806d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f477c036000 tx=0x7f477c008030 comp rx=0 tx=0).stop 2024-03-07T06:52:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.666+0000 7f4792800640 1 -- 172.21.15.133:0/3397630841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478c0ce420 msgr2=0x7f478c1f6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.666+0000 7f4792800640 1 --2- 172.21.15.133:0/3397630841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478c0ce420 0x7f478c1f6480 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f477400d800 tx=0x7f477400dcc0 comp rx=0 tx=0).stop 2024-03-07T06:52:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.666+0000 7f4792800640 1 -- 172.21.15.133:0/3397630841 shutdown_connections 2024-03-07T06:52:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.666+0000 7f4792800640 1 --2- 172.21.15.133:0/3397630841 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f476806b4d0 0x7f476806d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.666+0000 7f4792800640 1 --2- 172.21.15.133:0/3397630841 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f478c15b1b0 0x7f478c09a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.666+0000 7f4792800640 1 --2- 172.21.15.133:0/3397630841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478c0ce420 0x7f478c1f6480 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.666+0000 7f4792800640 1 --2- 172.21.15.133:0/3397630841 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f478c0af950 0x7f478c147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.666+0000 7f4792800640 1 -- 172.21.15.133:0/3397630841 >> 172.21.15.133:0/3397630841 conn(0x7f478c0b3120 msgr2=0x7f478c072ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.666+0000 7f4792800640 1 -- 172.21.15.133:0/3397630841 shutdown_connections 2024-03-07T06:52:45.666 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:45.666+0000 7f4792800640 1 -- 172.21.15.133:0/3397630841 wait complete. 2024-03-07T06:52:45.685 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-03-07T06:52:46.289 INFO:teuthology.orchestra.run.smithi133.stdout:{"message":"","return_code":0} 2024-03-07T06:52:46.290 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-07T06:52:46.290 DEBUG:teuthology.orchestra.run.smithi133:> stdin-killer -- python3 -c ' 2024-03-07T06:52:46.290 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-03-07T06:52:46.290 DEBUG:teuthology.orchestra.run.smithi133:> import stat 2024-03-07T06:52:46.290 DEBUG:teuthology.orchestra.run.smithi133:> 2024-03-07T06:52:46.290 DEBUG:teuthology.orchestra.run.smithi133:> print(os.stat("/home/ubuntu/cephtest/mnt.0/delete_me").st_ino) 2024-03-07T06:52:46.290 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-03-07T06:52:46.396 INFO:teuthology.orchestra.run.smithi133.stdout:1099511627776 2024-03-07T06:52:46.400 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-07T06:52:46.408 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 10000000000.00000000 2024-03-07T06:52:46.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.553+0000 7f5fa5a40f40 1 Processor -- start 2024-03-07T06:52:46.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.553+0000 7f5fa5a40f40 1 -- start start 2024-03-07T06:52:46.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa5a40f40 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20e8f7800 0x55c20e8f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:46.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa5a40f40 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55c20e923040 0x55c20e927f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:46.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa5a40f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c20e8f85f0 0x55c20e926ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:46.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa5a40f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20e89fac0 con 0x55c20e8f85f0 2024-03-07T06:52:46.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa5a40f40 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20e8a3a20 con 0x55c20e8f7800 2024-03-07T06:52:46.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa5a40f40 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c20e941070 con 0x55c20e923040 2024-03-07T06:52:46.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa2c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c20e8f85f0 0x55c20e926ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:46.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa2c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c20e8f85f0 0x55c20e926ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47462/0 (socket says 172.21.15.133:47462) 2024-03-07T06:52:46.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa2c00640 1 -- 172.21.15.133:0/1132156999 learned_addr learned my addr 172.21.15.133:0/1132156999 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:46.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5f9be00640 1 --2- 172.21.15.133:0/1132156999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55c20e923040 0x55c20e927f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:46.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa2200640 1 --2- 172.21.15.133:0/1132156999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20e8f7800 0x55c20e8f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:46.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa2c00640 1 -- 172.21.15.133:0/1132156999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55c20e923040 msgr2=0x55c20e927f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:46.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa2c00640 1 --2- 172.21.15.133:0/1132156999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55c20e923040 0x55c20e927f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:46.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa2c00640 1 -- 172.21.15.133:0/1132156999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20e8f7800 msgr2=0x55c20e8f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:46.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa2c00640 1 --2- 172.21.15.133:0/1132156999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20e8f7800 0x55c20e8f7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:46.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa2c00640 1 -- 172.21.15.133:0/1132156999 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c20e8f8120 con 0x55c20e8f85f0 2024-03-07T06:52:46.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5f9be00640 1 --2- 172.21.15.133:0/1132156999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55c20e923040 0x55c20e927f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:46.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.554+0000 7f5fa2200640 1 --2- 172.21.15.133:0/1132156999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20e8f7800 0x55c20e8f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:46.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.555+0000 7f5fa2c00640 1 --2- 172.21.15.133:0/1132156999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c20e8f85f0 0x55c20e926ce0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c001740 tx=0x7f5f9c04c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2169955d2ff46974 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:46.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.555+0000 7f5fa0e00640 1 -- 172.21.15.133:0/1132156999 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c006d50 con 0x55c20e8f85f0 2024-03-07T06:52:46.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.555+0000 7f5fa0e00640 1 -- 172.21.15.133:0/1132156999 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c062070 con 0x55c20e8f85f0 2024-03-07T06:52:46.556 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.555+0000 7f5fa0e00640 1 -- 172.21.15.133:0/1132156999 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0066a0 con 0x55c20e8f85f0 2024-03-07T06:52:46.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.556+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/1132156999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c20e8f85f0 msgr2=0x55c20e926ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:46.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.556+0000 7f5fa5a40f40 1 --2- 172.21.15.133:0/1132156999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c20e8f85f0 0x55c20e926ce0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c001740 tx=0x7f5f9c04c070 comp rx=0 tx=0).stop 2024-03-07T06:52:46.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.556+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/1132156999 shutdown_connections 2024-03-07T06:52:46.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.556+0000 7f5fa5a40f40 1 --2- 172.21.15.133:0/1132156999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c20e8f85f0 0x55c20e926ce0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:46.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.556+0000 7f5fa5a40f40 1 --2- 172.21.15.133:0/1132156999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55c20e923040 0x55c20e927f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:46.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.556+0000 7f5fa5a40f40 1 --2- 172.21.15.133:0/1132156999 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20e8f7800 0x55c20e8f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:46.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.556+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/1132156999 >> 172.21.15.133:0/1132156999 conn(0x55c20e9268d0 msgr2=0x55c20e928700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:46.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.556+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/1132156999 shutdown_connections 2024-03-07T06:52:46.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.556+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/1132156999 wait complete. 2024-03-07T06:52:46.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.557+0000 7f5fa5a40f40 1 Processor -- start 2024-03-07T06:52:46.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.557+0000 7f5fa5a40f40 1 -- start start 2024-03-07T06:52:46.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5fa5a40f40 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55c20e8f7800 0x55c20e93d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:46.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5fa5a40f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c20e8f85f0 0x55c20ea616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:46.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5fa5a40f40 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20e923040 0x55c20ea61bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:46.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5fa5a40f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20e9284a0 con 0x55c20e8f85f0 2024-03-07T06:52:46.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5fa5a40f40 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20e938fc0 con 0x55c20e923040 2024-03-07T06:52:46.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5fa5a40f40 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c20e942ef0 con 0x55c20e8f7800 2024-03-07T06:52:46.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5f9be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c20e8f85f0 0x55c20ea616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:46.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5f9be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c20e8f85f0 0x55c20ea616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47466/0 (socket says 172.21.15.133:47466) 2024-03-07T06:52:46.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5f9be00640 1 -- 172.21.15.133:0/3207366186 learned_addr learned my addr 172.21.15.133:0/3207366186 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:46.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5fa2200640 1 --2- 172.21.15.133:0/3207366186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55c20e8f7800 0x55c20e93d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:46.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5f9be00640 1 -- 172.21.15.133:0/3207366186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55c20e8f7800 msgr2=0x55c20e93d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:46.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5f9be00640 1 --2- 172.21.15.133:0/3207366186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55c20e8f7800 0x55c20e93d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:46.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5f9be00640 1 -- 172.21.15.133:0/3207366186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20e923040 msgr2=0x55c20ea61bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:46.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5f9be00640 1 --2- 172.21.15.133:0/3207366186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55c20e923040 0x55c20ea61bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:46.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.559+0000 7f5f9be00640 1 -- 172.21.15.133:0/3207366186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c20e928000 con 0x55c20e8f85f0 2024-03-07T06:52:46.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.560+0000 7f5fa2200640 1 --2- 172.21.15.133:0/3207366186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55c20e8f7800 0x55c20e93d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:46.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.560+0000 7f5f9be00640 1 --2- 172.21.15.133:0/3207366186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c20e8f85f0 0x55c20ea616b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009ee0 tx=0x7f5f8c008400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:46.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.560+0000 7f5f9aa00640 1 -- 172.21.15.133:0/3207366186 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0143f0 con 0x55c20e8f85f0 2024-03-07T06:52:46.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.560+0000 7f5f9aa00640 1 -- 172.21.15.133:0/3207366186 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8c019070 con 0x55c20e8f85f0 2024-03-07T06:52:46.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.560+0000 7f5f9aa00640 1 -- 172.21.15.133:0/3207366186 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0147f0 con 0x55c20e8f85f0 2024-03-07T06:52:46.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.560+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/3207366186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c20e9282a0 con 0x55c20e8f85f0 2024-03-07T06:52:46.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.560+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/3207366186 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c20e93c940 con 0x55c20e8f85f0 2024-03-07T06:52:46.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.562+0000 7f5f9aa00640 1 -- 172.21.15.133:0/3207366186 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5f8c014990 con 0x55c20e8f85f0 2024-03-07T06:52:46.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.563+0000 7f5f9aa00640 1 --2- 172.21.15.133:0/3207366186 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5f80069eb0 0x7f5f8006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:46.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.563+0000 7f5f9aa00640 1 -- 172.21.15.133:0/3207366186 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f8c088df0 con 0x55c20e8f85f0 2024-03-07T06:52:46.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.563+0000 7f5fa5a40f40 1 --2- 172.21.15.133:0/3207366186 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x55c20e8f7d30 0x55c20e9c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:46.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.563+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/3207366186 --> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v9 -- 0x55c20e9c9aa0 con 0x55c20e8f7d30 2024-03-07T06:52:46.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.564+0000 7f5fa2200640 1 --2- 172.21.15.133:0/3207366186 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5f80069eb0 0x7f5f8006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:46.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.564+0000 7f5fa2c00640 1 --2- 172.21.15.133:0/3207366186 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x55c20e8f7d30 0x55c20e9c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:46.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.564+0000 7f5fa2c00640 1 --2- 172.21.15.133:0/3207366186 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x55c20e8f7d30 0x55c20e9c9560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:46.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.565+0000 7f5fa2200640 1 --2- 172.21.15.133:0/3207366186 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5f80069eb0 0x7f5f8006c370 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f94001920 tx=0x7f5f940014d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:46.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.565+0000 7f5fa2c00640 1 -- 172.21.15.133:0/3207366186 <== osd.3 v2:172.21.15.133:6816/2661528260 1 ==== osd_op_reply(1 10000000000.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x7f5f9c058040 con 0x55c20e8f7d30 2024-03-07T06:52:46.565 INFO:teuthology.orchestra.run.smithi133.stdout:cephfs_metadata/10000000000.00000000 mtime 2024-03-07T06:52:46.000000+0000, size 0 2024-03-07T06:52:46.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.565+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/3207366186 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x55c20e8f7d30 msgr2=0x55c20e9c9560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:46.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.565+0000 7f5fa5a40f40 1 --2- 172.21.15.133:0/3207366186 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x55c20e8f7d30 0x55c20e9c9560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:46.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.565+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/3207366186 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5f80069eb0 msgr2=0x7f5f8006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:46.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.565+0000 7f5fa5a40f40 1 --2- 172.21.15.133:0/3207366186 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5f80069eb0 0x7f5f8006c370 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f94001920 tx=0x7f5f940014d0 comp rx=0 tx=0).stop 2024-03-07T06:52:46.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.565+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/3207366186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c20e8f85f0 msgr2=0x55c20ea616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:46.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.565+0000 7f5fa5a40f40 1 --2- 172.21.15.133:0/3207366186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c20e8f85f0 0x55c20ea616b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009ee0 tx=0x7f5f8c008400 comp rx=0 tx=0).stop 2024-03-07T06:52:46.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.565+0000 7f5fa2c00640 1 -- 172.21.15.133:0/3207366186 reap_dead start 2024-03-07T06:52:46.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.565+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/3207366186 shutdown_connections 2024-03-07T06:52:46.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.565+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/3207366186 >> 172.21.15.133:0/3207366186 conn(0x55c20e9268d0 msgr2=0x55c20e939180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:46.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.565+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/3207366186 shutdown_connections 2024-03-07T06:52:46.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:46.565+0000 7f5fa5a40f40 1 -- 172.21.15.133:0/3207366186 wait complete. 2024-03-07T06:52:46.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:52:46.572 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -rf delete_me) 2024-03-07T06:52:49.212 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:52:49.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.404+0000 7fb954200640 1 Processor -- start 2024-03-07T06:52:49.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.404+0000 7fb954200640 1 -- start start 2024-03-07T06:52:49.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.406+0000 7fb954200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb94c0b5090 0x7fb94c072910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:49.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.406+0000 7fb954200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb94c0d1260 0x7fb94c072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:49.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.406+0000 7fb954200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94c15fc30 con 0x7fb94c147bb0 2024-03-07T06:52:49.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.406+0000 7fb954200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94c161ce0 con 0x7fb94c0d1260 2024-03-07T06:52:49.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.406+0000 7fb954200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c15ff10 con 0x7fb94c0b5090 2024-03-07T06:52:49.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.406+0000 7fb950e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb94c0b5090 0x7fb94c072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:49.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.406+0000 7fb950e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb94c0b5090 0x7fb94c072910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:54192/0 (socket says 172.21.15.133:54192) 2024-03-07T06:52:49.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.406+0000 7fb950e00640 1 -- 172.21.15.133:0/1877275697 learned_addr learned my addr 172.21.15.133:0/1877275697 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:49.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.407+0000 7fb952200640 1 --2- 172.21.15.133:0/1877275697 >> v2:172.21.15.161:3300/0 conn(0x7fb94c0d1260 0x7fb94c072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:49.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.407+0000 7fb950e00640 1 -- 172.21.15.133:0/1877275697 >> v2:172.21.15.161:3300/0 conn(0x7fb94c0d1260 msgr2=0x7fb94c072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:49.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.407+0000 7fb950e00640 1 --2- 172.21.15.133:0/1877275697 >> v2:172.21.15.161:3300/0 conn(0x7fb94c0d1260 0x7fb94c072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:49.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.407+0000 7fb950e00640 1 -- 172.21.15.133:0/1877275697 >> v1:172.21.15.161:6789/0 conn(0x7fb94c147bb0 legacy=0x7fb94c0d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:49.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.407+0000 7fb950e00640 1 -- 172.21.15.133:0/1877275697 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c1601f0 con 0x7fb94c0b5090 2024-03-07T06:52:49.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.408+0000 7fb950e00640 1 --2- 172.21.15.133:0/1877275697 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb94c0b5090 0x7fb94c072910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb938009870 tx=0x7fb93802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7e61ce5c9541b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:49.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.409+0000 7fb94be00640 1 -- 172.21.15.133:0/1877275697 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938034070 con 0x7fb94c0b5090 2024-03-07T06:52:49.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.409+0000 7fb94be00640 1 -- 172.21.15.133:0/1877275697 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb938030840 con 0x7fb94c0b5090 2024-03-07T06:52:49.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.409+0000 7fb94be00640 1 -- 172.21.15.133:0/1877275697 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938030c10 con 0x7fb94c0b5090 2024-03-07T06:52:49.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.409+0000 7fb954200640 1 -- 172.21.15.133:0/1877275697 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb94c0b5090 msgr2=0x7fb94c072910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:49.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.409+0000 7fb954200640 1 --2- 172.21.15.133:0/1877275697 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb94c0b5090 0x7fb94c072910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb938009870 tx=0x7fb93802b980 comp rx=0 tx=0).stop 2024-03-07T06:52:49.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.409+0000 7fb954200640 1 -- 172.21.15.133:0/1877275697 shutdown_connections 2024-03-07T06:52:49.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.409+0000 7fb954200640 1 --2- 172.21.15.133:0/1877275697 >> v2:172.21.15.161:3300/0 conn(0x7fb94c0d1260 0x7fb94c072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:49.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.409+0000 7fb954200640 1 --2- 172.21.15.133:0/1877275697 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb94c0b5090 0x7fb94c072910 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:49.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.409+0000 7fb954200640 1 -- 172.21.15.133:0/1877275697 >> 172.21.15.133:0/1877275697 conn(0x7fb94c06b480 msgr2=0x7fb94c06b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:49.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.410+0000 7fb954200640 1 -- 172.21.15.133:0/1877275697 shutdown_connections 2024-03-07T06:52:49.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.410+0000 7fb954200640 1 -- 172.21.15.133:0/1877275697 wait complete. 2024-03-07T06:52:49.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.412+0000 7fb954200640 1 Processor -- start 2024-03-07T06:52:49.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.412+0000 7fb954200640 1 -- start start 2024-03-07T06:52:49.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.414+0000 7fb954200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb94c0b5090 0x7fb94c13f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:49.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.414+0000 7fb954200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb94c0d1260 0x7fb94c15c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:49.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.414+0000 7fb954200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb94c147bb0 0x7fb94c140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:49.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.414+0000 7fb954200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c147f90 con 0x7fb94c0b5090 2024-03-07T06:52:49.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.414+0000 7fb954200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c1607b0 con 0x7fb94c147bb0 2024-03-07T06:52:49.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.414+0000 7fb954200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c160cd0 con 0x7fb94c0d1260 2024-03-07T06:52:49.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.414+0000 7fb951800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb94c0b5090 0x7fb94c13f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:49.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.414+0000 7fb951800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb94c0b5090 0x7fb94c13f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47476/0 (socket says 172.21.15.133:47476) 2024-03-07T06:52:49.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.414+0000 7fb951800640 1 -- 172.21.15.133:0/1621545639 learned_addr learned my addr 172.21.15.133:0/1621545639 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:49.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.415+0000 7fb950e00640 1 --2- 172.21.15.133:0/1621545639 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb94c0d1260 0x7fb94c15c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:49.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.415+0000 7fb951800640 1 -- 172.21.15.133:0/1621545639 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb94c0d1260 msgr2=0x7fb94c15c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:49.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.415+0000 7fb951800640 1 --2- 172.21.15.133:0/1621545639 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb94c0d1260 0x7fb94c15c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:49.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.415+0000 7fb951800640 1 -- 172.21.15.133:0/1621545639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb94c147bb0 msgr2=0x7fb94c140de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:49.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.415+0000 7fb951800640 1 --2- 172.21.15.133:0/1621545639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb94c147bb0 0x7fb94c140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:49.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.415+0000 7fb951800640 1 -- 172.21.15.133:0/1621545639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c13fd40 con 0x7fb94c0b5090 2024-03-07T06:52:49.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.415+0000 7fb951800640 1 --2- 172.21.15.133:0/1621545639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb94c0b5090 0x7fb94c13f6b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb94000c3c0 tx=0x7fb94000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:49.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.415+0000 7fb949600640 1 -- 172.21.15.133:0/1621545639 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9400052d0 con 0x7fb94c0b5090 2024-03-07T06:52:49.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.415+0000 7fb949600640 1 -- 172.21.15.133:0/1621545639 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb940005490 con 0x7fb94c0b5090 2024-03-07T06:52:49.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.415+0000 7fb954200640 1 -- 172.21.15.133:0/1621545639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c140020 con 0x7fb94c0b5090 2024-03-07T06:52:49.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.415+0000 7fb949600640 1 -- 172.21.15.133:0/1621545639 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940007260 con 0x7fb94c0b5090 2024-03-07T06:52:49.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.415+0000 7fb954200640 1 -- 172.21.15.133:0/1621545639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c1405e0 con 0x7fb94c0b5090 2024-03-07T06:52:49.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.416+0000 7fb954200640 1 -- 172.21.15.133:0/1621545639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94c062c90 con 0x7fb94c0b5090 2024-03-07T06:52:49.417 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.418+0000 7fb949600640 1 -- 172.21.15.133:0/1621545639 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb940007400 con 0x7fb94c0b5090 2024-03-07T06:52:49.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.419+0000 7fb949600640 1 --2- 172.21.15.133:0/1621545639 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb92c06b500 0x7fb92c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:49.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.419+0000 7fb949600640 1 -- 172.21.15.133:0/1621545639 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9400896a0 con 0x7fb94c0b5090 2024-03-07T06:52:49.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.420+0000 7fb950e00640 1 --2- 172.21.15.133:0/1621545639 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb92c06b500 0x7fb92c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:49.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.421+0000 7fb950e00640 1 --2- 172.21.15.133:0/1621545639 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb92c06b500 0x7fb92c06d9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb938036000 tx=0x7fb93803b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:49.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.423+0000 7fb949600640 1 -- 172.21.15.133:0/1621545639 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb940058510 con 0x7fb94c0b5090 2024-03-07T06:52:49.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.657+0000 7fb954200640 1 -- 172.21.15.133:0/1621545639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb94c0726f0 con 0x7fb94c0b5090 2024-03-07T06:52:49.657 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.658+0000 7fb949600640 1 -- 172.21.15.133:0/1621545639 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4983 (secure 0 0 0) 0x7fb940057e10 con 0x7fb94c0b5090 2024-03-07T06:52:49.658 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:49.658 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3269394125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3269394125},{"type":"v1","addr":"172.21.15.161:6835","nonce":3269394125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/72243172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":72243172},{"type":"v1","addr":"172.21.15.161:6837","nonce":72243172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3714291615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3714291615},{"type":"v1","addr":"172.21.15.133:6834","nonce":3714291615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:52:35.182567+0000","modified":"2024-03-07T06:52:38.268724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6835/4067860709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":4067860709},{"type":"v1","addr":"172.21.15.133:6835","nonce":4067860709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5100,"qdb_cluster":[5100]},"id":2}]} 2024-03-07T06:52:49.658 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 18 2024-03-07T06:52:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.661+0000 7fb954200640 1 -- 172.21.15.133:0/1621545639 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb92c06b500 msgr2=0x7fb92c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.661+0000 7fb954200640 1 --2- 172.21.15.133:0/1621545639 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb92c06b500 0x7fb92c06d9c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb938036000 tx=0x7fb93803b040 comp rx=0 tx=0).stop 2024-03-07T06:52:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.661+0000 7fb954200640 1 -- 172.21.15.133:0/1621545639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb94c0b5090 msgr2=0x7fb94c13f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.661+0000 7fb954200640 1 --2- 172.21.15.133:0/1621545639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb94c0b5090 0x7fb94c13f6b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb94000c3c0 tx=0x7fb94000c880 comp rx=0 tx=0).stop 2024-03-07T06:52:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.661+0000 7fb954200640 1 -- 172.21.15.133:0/1621545639 shutdown_connections 2024-03-07T06:52:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.661+0000 7fb954200640 1 --2- 172.21.15.133:0/1621545639 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb92c06b500 0x7fb92c06d9c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.661+0000 7fb954200640 1 --2- 172.21.15.133:0/1621545639 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb94c147bb0 0x7fb94c140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.661+0000 7fb954200640 1 --2- 172.21.15.133:0/1621545639 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb94c0d1260 0x7fb94c15c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.661+0000 7fb954200640 1 --2- 172.21.15.133:0/1621545639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb94c0b5090 0x7fb94c13f6b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:49.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.661+0000 7fb954200640 1 -- 172.21.15.133:0/1621545639 >> 172.21.15.133:0/1621545639 conn(0x7fb94c06b480 msgr2=0x7fb94c1461b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:49.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.661+0000 7fb954200640 1 -- 172.21.15.133:0/1621545639 shutdown_connections 2024-03-07T06:52:49.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:49.662+0000 7fb954200640 1 -- 172.21.15.133:0/1621545639 wait complete. 2024-03-07T06:52:49.693 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-03-07T06:52:55.096 INFO:teuthology.orchestra.run.smithi133.stdout:{"message":"","return_code":0} 2024-03-07T06:52:55.096 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-07T06:52:55.097 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:52:55.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.275+0000 7fd0aaa00640 1 Processor -- start 2024-03-07T06:52:55.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.275+0000 7fd0aaa00640 1 -- start start 2024-03-07T06:52:55.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.277+0000 7fd0aaa00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd0a415bd30 0x7fd0a406e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:55.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.277+0000 7fd0aaa00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0a4072f40 0x7fd0a415c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:55.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.277+0000 7fd0aaa00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a4160760 con 0x7fd0a40cf5d0 2024-03-07T06:52:55.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.277+0000 7fd0aaa00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a415f9a0 con 0x7fd0a415bd30 2024-03-07T06:52:55.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.277+0000 7fd0aaa00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a415fc80 con 0x7fd0a4072f40 2024-03-07T06:52:55.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.277+0000 7fd0a9600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd0a415bd30 0x7fd0a406e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:55.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.277+0000 7fd0a9600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd0a415bd30 0x7fd0a406e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36848/0 (socket says 172.21.15.133:36848) 2024-03-07T06:52:55.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.277+0000 7fd0a9600640 1 -- 172.21.15.133:0/321640131 learned_addr learned my addr 172.21.15.133:0/321640131 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:55.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.277+0000 7fd0aa000640 1 --2- 172.21.15.133:0/321640131 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0a4072f40 0x7fd0a415c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:55.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.278+0000 7fd0a9600640 1 -- 172.21.15.133:0/321640131 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0a4072f40 msgr2=0x7fd0a415c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:55.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.278+0000 7fd0a9600640 1 --2- 172.21.15.133:0/321640131 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0a4072f40 0x7fd0a415c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:55.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.278+0000 7fd0a9600640 1 -- 172.21.15.133:0/321640131 >> v1:172.21.15.161:6789/0 conn(0x7fd0a40cf5d0 legacy=0x7fd0a4072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:55.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.278+0000 7fd0a9600640 1 -- 172.21.15.133:0/321640131 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a415ff60 con 0x7fd0a415bd30 2024-03-07T06:52:55.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.279+0000 7fd0a9600640 1 --2- 172.21.15.133:0/321640131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a415bd30 0x7fd0a406e8e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd098009870 tx=0x7fd09802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10771d17b68e46e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:55.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.279+0000 7fd0a3e00640 1 -- 172.21.15.133:0/321640131 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd098034070 con 0x7fd0a415bd30 2024-03-07T06:52:55.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.279+0000 7fd0a3e00640 1 -- 172.21.15.133:0/321640131 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd098030840 con 0x7fd0a415bd30 2024-03-07T06:52:55.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.279+0000 7fd0a3e00640 1 -- 172.21.15.133:0/321640131 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd098030c10 con 0x7fd0a415bd30 2024-03-07T06:52:55.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.280+0000 7fd0aaa00640 1 -- 172.21.15.133:0/321640131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a415bd30 msgr2=0x7fd0a406e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:55.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.280+0000 7fd0aaa00640 1 --2- 172.21.15.133:0/321640131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a415bd30 0x7fd0a406e8e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd098009870 tx=0x7fd09802b980 comp rx=0 tx=0).stop 2024-03-07T06:52:55.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.280+0000 7fd0aaa00640 1 -- 172.21.15.133:0/321640131 shutdown_connections 2024-03-07T06:52:55.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.280+0000 7fd0aaa00640 1 --2- 172.21.15.133:0/321640131 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0a4072f40 0x7fd0a415c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:55.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.280+0000 7fd0aaa00640 1 --2- 172.21.15.133:0/321640131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a415bd30 0x7fd0a406e8e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:55.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.280+0000 7fd0aaa00640 1 -- 172.21.15.133:0/321640131 >> 172.21.15.133:0/321640131 conn(0x7fd0a40b2e90 msgr2=0x7fd0a40b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:55.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.280+0000 7fd0aaa00640 1 -- 172.21.15.133:0/321640131 shutdown_connections 2024-03-07T06:52:55.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.280+0000 7fd0aaa00640 1 -- 172.21.15.133:0/321640131 wait complete. 2024-03-07T06:52:55.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.283+0000 7fd0aaa00640 1 Processor -- start 2024-03-07T06:52:55.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.283+0000 7fd0aaa00640 1 -- start start 2024-03-07T06:52:55.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.284+0000 7fd0aaa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a4072f40 0x7fd0a41461f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:55.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0aaa00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0a40cf5d0 0x7fd0a40aa910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:55.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0aaa00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0a415bd30 0x7fd0a40aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:55.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0aaa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a406f130 con 0x7fd0a4072f40 2024-03-07T06:52:55.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0aaa00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a406ee20 con 0x7fd0a40cf5d0 2024-03-07T06:52:55.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0aaa00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a415c650 con 0x7fd0a415bd30 2024-03-07T06:52:55.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0a9600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a4072f40 0x7fd0a41461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:55.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0a9600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a4072f40 0x7fd0a41461f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36858/0 (socket says 172.21.15.133:36858) 2024-03-07T06:52:55.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0a9600640 1 -- 172.21.15.133:0/2227114683 learned_addr learned my addr 172.21.15.133:0/2227114683 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0a8c00640 1 --2- 172.21.15.133:0/2227114683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0a40cf5d0 0x7fd0a40aa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0a9600640 1 -- 172.21.15.133:0/2227114683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0a415bd30 msgr2=0x7fd0a40aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0a9600640 1 --2- 172.21.15.133:0/2227114683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0a415bd30 0x7fd0a40aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0a9600640 1 -- 172.21.15.133:0/2227114683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0a40cf5d0 msgr2=0x7fd0a40aa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:55.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0a9600640 1 --2- 172.21.15.133:0/2227114683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0a40cf5d0 0x7fd0a40aa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0a9600640 1 -- 172.21.15.133:0/2227114683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a40acd00 con 0x7fd0a4072f40 2024-03-07T06:52:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.285+0000 7fd0a8c00640 1 --2- 172.21.15.133:0/2227114683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0a40cf5d0 0x7fd0a40aa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.286+0000 7fd0a9600640 1 --2- 172.21.15.133:0/2227114683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a4072f40 0x7fd0a41461f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd098000c00 tx=0x7fd0980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.286+0000 7fd0a1600640 1 -- 172.21.15.133:0/2227114683 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd098034040 con 0x7fd0a4072f40 2024-03-07T06:52:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.286+0000 7fd0a1600640 1 -- 172.21.15.133:0/2227114683 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd098039d90 con 0x7fd0a4072f40 2024-03-07T06:52:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.286+0000 7fd0a1600640 1 -- 172.21.15.133:0/2227114683 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd098043610 con 0x7fd0a4072f40 2024-03-07T06:52:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.286+0000 7fd0aaa00640 1 -- 172.21.15.133:0/2227114683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a40acfe0 con 0x7fd0a4072f40 2024-03-07T06:52:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.286+0000 7fd0aaa00640 1 -- 172.21.15.133:0/2227114683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a40ad5a0 con 0x7fd0a4072f40 2024-03-07T06:52:55.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.287+0000 7fd0aaa00640 1 -- 172.21.15.133:0/2227114683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a4062c90 con 0x7fd0a4072f40 2024-03-07T06:52:55.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.289+0000 7fd0a1600640 1 -- 172.21.15.133:0/2227114683 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd09804a060 con 0x7fd0a4072f40 2024-03-07T06:52:55.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.290+0000 7fd0a1600640 1 --2- 172.21.15.133:0/2227114683 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd08406b4d0 0x7fd08406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:55.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.290+0000 7fd0a1600640 1 -- 172.21.15.133:0/2227114683 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0980ac250 con 0x7fd0a4072f40 2024-03-07T06:52:55.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.291+0000 7fd0a8c00640 1 --2- 172.21.15.133:0/2227114683 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd08406b4d0 0x7fd08406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:55.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.292+0000 7fd0a8c00640 1 --2- 172.21.15.133:0/2227114683 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd08406b4d0 0x7fd08406d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd0a406ea80 tx=0x7fd090009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:55.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.295+0000 7fd0a1600640 1 -- 172.21.15.133:0/2227114683 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd09803e070 con 0x7fd0a4072f40 2024-03-07T06:52:55.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.560+0000 7fd0aaa00640 1 -- 172.21.15.133:0/2227114683 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0a40734a0 con 0x7fd0a4072f40 2024-03-07T06:52:55.561 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.561+0000 7fd0a1600640 1 -- 172.21.15.133:0/2227114683 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4983 (secure 0 0 0) 0x7fd09807b0c0 con 0x7fd0a4072f40 2024-03-07T06:52:55.561 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:55.561 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3269394125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3269394125},{"type":"v1","addr":"172.21.15.161:6835","nonce":3269394125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/72243172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":72243172},{"type":"v1","addr":"172.21.15.161:6837","nonce":72243172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3714291615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3714291615},{"type":"v1","addr":"172.21.15.133:6834","nonce":3714291615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:52:35.182567+0000","modified":"2024-03-07T06:52:38.268724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6835/4067860709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":4067860709},{"type":"v1","addr":"172.21.15.133:6835","nonce":4067860709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5100,"qdb_cluster":[5100]},"id":2}]} 2024-03-07T06:52:55.561 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 18 2024-03-07T06:52:55.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.564+0000 7fd0aaa00640 1 -- 172.21.15.133:0/2227114683 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd08406b4d0 msgr2=0x7fd08406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:55.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.564+0000 7fd0aaa00640 1 --2- 172.21.15.133:0/2227114683 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd08406b4d0 0x7fd08406d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd0a406ea80 tx=0x7fd090009040 comp rx=0 tx=0).stop 2024-03-07T06:52:55.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.564+0000 7fd0aaa00640 1 -- 172.21.15.133:0/2227114683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a4072f40 msgr2=0x7fd0a41461f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:55.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.564+0000 7fd0aaa00640 1 --2- 172.21.15.133:0/2227114683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a4072f40 0x7fd0a41461f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd098000c00 tx=0x7fd0980046c0 comp rx=0 tx=0).stop 2024-03-07T06:52:55.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.565+0000 7fd0aaa00640 1 -- 172.21.15.133:0/2227114683 shutdown_connections 2024-03-07T06:52:55.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.565+0000 7fd0aaa00640 1 --2- 172.21.15.133:0/2227114683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd0a415bd30 0x7fd0a40aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:55.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.565+0000 7fd0aaa00640 1 --2- 172.21.15.133:0/2227114683 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd08406b4d0 0x7fd08406d990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:55.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.565+0000 7fd0aaa00640 1 --2- 172.21.15.133:0/2227114683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0a40cf5d0 0x7fd0a40aa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:55.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.565+0000 7fd0aaa00640 1 --2- 172.21.15.133:0/2227114683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0a4072f40 0x7fd0a41461f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:55.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.565+0000 7fd0aaa00640 1 -- 172.21.15.133:0/2227114683 >> 172.21.15.133:0/2227114683 conn(0x7fd0a40b2e90 msgr2=0x7fd0a40b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:55.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.565+0000 7fd0aaa00640 1 -- 172.21.15.133:0/2227114683 shutdown_connections 2024-03-07T06:52:55.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:55.565+0000 7fd0aaa00640 1 -- 172.21.15.133:0/2227114683 wait complete. 2024-03-07T06:52:55.583 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump mds_cache strays_created 2024-03-07T06:52:55.792 INFO:teuthology.orchestra.run.smithi133.stdout:{"mds_cache":{"strays_created":1001}} 2024-03-07T06:52:55.792 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_cache": { "strays_created": 1001 } } 2024-03-07T06:52:55.793 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:52:56.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.035+0000 7fd06a800640 1 Processor -- start 2024-03-07T06:52:56.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.036+0000 7fd06a800640 1 -- start start 2024-03-07T06:52:56.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.037+0000 7fd06a800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd064072950 0x7fd0640cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:56.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.037+0000 7fd06a800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd0640cdee0 0x7fd06415c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:56.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.037+0000 7fd06a800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0640731b0 con 0x7fd06415fbb0 2024-03-07T06:52:56.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.037+0000 7fd06a800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd064072f60 con 0x7fd064072950 2024-03-07T06:52:56.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.037+0000 7fd06a800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd064073490 con 0x7fd0640cdee0 2024-03-07T06:52:56.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.037+0000 7fd05be00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd064072950 0x7fd0640cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:56.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.037+0000 7fd063400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd06415fbb0 0x7fd06415ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47304/0 (socket says 172.21.15.133:47304) 2024-03-07T06:52:56.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.037+0000 7fd063400640 1 -- 172.21.15.133:0/3833547733 learned_addr learned my addr 172.21.15.133:0/3833547733 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:56.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.038+0000 7fd063e00640 1 --2- 172.21.15.133:0/3833547733 >> v2:172.21.15.161:3300/0 conn(0x7fd0640cdee0 0x7fd06415c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:56.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.038+0000 7fd05be00640 1 -- 172.21.15.133:0/3833547733 >> v2:172.21.15.161:3300/0 conn(0x7fd0640cdee0 msgr2=0x7fd06415c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:56.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.038+0000 7fd05be00640 1 --2- 172.21.15.133:0/3833547733 >> v2:172.21.15.161:3300/0 conn(0x7fd0640cdee0 0x7fd06415c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.038+0000 7fd05be00640 1 -- 172.21.15.133:0/3833547733 >> v1:172.21.15.133:6789/0 conn(0x7fd06415fbb0 legacy=0x7fd06415ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:56.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.038+0000 7fd062a00640 1 -- 172.21.15.133:0/3833547733 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911895745 0 0) 0x7fd0640731b0 con 0x7fd06415fbb0 2024-03-07T06:52:56.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.038+0000 7fd05be00640 1 -- 172.21.15.133:0/3833547733 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd064072150 con 0x7fd064072950 2024-03-07T06:52:56.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.038+0000 7fd063e00640 1 --2- 172.21.15.133:0/3833547733 >> v2:172.21.15.161:3300/0 conn(0x7fd0640cdee0 0x7fd06415c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:56.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.039+0000 7fd05be00640 1 --2- 172.21.15.133:0/3833547733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd064072950 0x7fd0640cf7d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd0640ccab0 tx=0x7fd04c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dd4e7ca9262fb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.039+0000 7fd062a00640 1 -- 172.21.15.133:0/3833547733 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c0315a0 con 0x7fd064072950 2024-03-07T06:52:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.039+0000 7fd062a00640 1 -- 172.21.15.133:0/3833547733 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c037070 con 0x7fd064072950 2024-03-07T06:52:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.039+0000 7fd062a00640 1 -- 172.21.15.133:0/3833547733 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c031960 con 0x7fd064072950 2024-03-07T06:52:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.040+0000 7fd06a800640 1 -- 172.21.15.133:0/3833547733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd064072950 msgr2=0x7fd0640cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.040+0000 7fd06a800640 1 --2- 172.21.15.133:0/3833547733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd064072950 0x7fd0640cf7d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd0640ccab0 tx=0x7fd04c02b840 comp rx=0 tx=0).stop 2024-03-07T06:52:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.040+0000 7fd06a800640 1 -- 172.21.15.133:0/3833547733 shutdown_connections 2024-03-07T06:52:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.040+0000 7fd06a800640 1 --2- 172.21.15.133:0/3833547733 >> v2:172.21.15.161:3300/0 conn(0x7fd0640cdee0 0x7fd06415c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.040+0000 7fd06a800640 1 --2- 172.21.15.133:0/3833547733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd064072950 0x7fd0640cf7d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.040+0000 7fd06a800640 1 -- 172.21.15.133:0/3833547733 >> 172.21.15.133:0/3833547733 conn(0x7fd0640b3120 msgr2=0x7fd0640b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.040+0000 7fd06a800640 1 -- 172.21.15.133:0/3833547733 shutdown_connections 2024-03-07T06:52:56.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.040+0000 7fd06a800640 1 -- 172.21.15.133:0/3833547733 wait complete. 2024-03-07T06:52:56.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.042+0000 7fd06a800640 1 Processor -- start 2024-03-07T06:52:56.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.042+0000 7fd06a800640 1 -- start start 2024-03-07T06:52:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd06a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0640cdee0 0x7fd064151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd06a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd06415fbb0 0x7fd06409c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd06a800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd06409c680 0x7fd0640c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd06a800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd064160980 con 0x7fd06415fbb0 2024-03-07T06:52:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd06a800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640ce310 con 0x7fd0640cdee0 2024-03-07T06:52:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd06a800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640b1a00 con 0x7fd06409c680 2024-03-07T06:52:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd05be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd06415fbb0 0x7fd06409c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd05be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd06415fbb0 0x7fd06409c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36882/0 (socket says 172.21.15.133:36882) 2024-03-07T06:52:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd05be00640 1 -- 172.21.15.133:0/3981046113 learned_addr learned my addr 172.21.15.133:0/3981046113 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd063400640 1 --2- 172.21.15.133:0/3981046113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0640cdee0 0x7fd064151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:56.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd05be00640 1 -- 172.21.15.133:0/3981046113 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd06409c680 msgr2=0x7fd0640c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:56.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd05be00640 1 --2- 172.21.15.133:0/3981046113 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd06409c680 0x7fd0640c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd05be00640 1 -- 172.21.15.133:0/3981046113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0640cdee0 msgr2=0x7fd064151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:56.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd05be00640 1 --2- 172.21.15.133:0/3981046113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0640cdee0 0x7fd064151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd05be00640 1 -- 172.21.15.133:0/3981046113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd064159d90 con 0x7fd06415fbb0 2024-03-07T06:52:56.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.043+0000 7fd063400640 1 --2- 172.21.15.133:0/3981046113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0640cdee0 0x7fd064151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:56.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.044+0000 7fd05be00640 1 --2- 172.21.15.133:0/3981046113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd06415fbb0 0x7fd06409c140 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009890 tx=0x7fd04c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:56.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.044+0000 7fd05b400640 1 -- 172.21.15.133:0/3981046113 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c032840 con 0x7fd06415fbb0 2024-03-07T06:52:56.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.044+0000 7fd05b400640 1 -- 172.21.15.133:0/3981046113 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c032a00 con 0x7fd06415fbb0 2024-03-07T06:52:56.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.044+0000 7fd06a800640 1 -- 172.21.15.133:0/3981046113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06409ce20 con 0x7fd06415fbb0 2024-03-07T06:52:56.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.044+0000 7fd05b400640 1 -- 172.21.15.133:0/3981046113 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c032df0 con 0x7fd06415fbb0 2024-03-07T06:52:56.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.044+0000 7fd06a800640 1 -- 172.21.15.133:0/3981046113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06409d3e0 con 0x7fd06415fbb0 2024-03-07T06:52:56.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.045+0000 7fd06a800640 1 -- 172.21.15.133:0/3981046113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0640ae1c0 con 0x7fd06415fbb0 2024-03-07T06:52:56.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.047+0000 7fd05b400640 1 -- 172.21.15.133:0/3981046113 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd04c04c020 con 0x7fd06415fbb0 2024-03-07T06:52:56.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.048+0000 7fd05b400640 1 --2- 172.21.15.133:0/3981046113 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd04006b4d0 0x7fd04006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:56.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.048+0000 7fd05b400640 1 -- 172.21.15.133:0/3981046113 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd04c02f020 con 0x7fd06415fbb0 2024-03-07T06:52:56.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.049+0000 7fd063400640 1 --2- 172.21.15.133:0/3981046113 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd04006b4d0 0x7fd04006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:56.049 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.050+0000 7fd063400640 1 --2- 172.21.15.133:0/3981046113 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd04006b4d0 0x7fd04006d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd054000f60 tx=0x7fd054009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:56.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.053+0000 7fd05b400640 1 -- 172.21.15.133:0/3981046113 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd0640ae1c0 con 0x7fd06415fbb0 2024-03-07T06:52:56.463 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.463+0000 7fd06a800640 1 -- 172.21.15.133:0/3981046113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0640cf970 con 0x7fd06415fbb0 2024-03-07T06:52:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.464+0000 7fd05b400640 1 -- 172.21.15.133:0/3981046113 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4983 (secure 0 0 0) 0x7fd04c040070 con 0x7fd06415fbb0 2024-03-07T06:52:56.464 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:56.464 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3269394125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3269394125},{"type":"v1","addr":"172.21.15.161:6835","nonce":3269394125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/72243172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":72243172},{"type":"v1","addr":"172.21.15.161:6837","nonce":72243172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3714291615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3714291615},{"type":"v1","addr":"172.21.15.133:6834","nonce":3714291615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:52:35.182567+0000","modified":"2024-03-07T06:52:38.268724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6835/4067860709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":4067860709},{"type":"v1","addr":"172.21.15.133:6835","nonce":4067860709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5100,"qdb_cluster":[5100]},"id":2}]} 2024-03-07T06:52:56.464 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 18 2024-03-07T06:52:56.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.466+0000 7fd06a800640 1 -- 172.21.15.133:0/3981046113 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd04006b4d0 msgr2=0x7fd04006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:56.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.466+0000 7fd06a800640 1 --2- 172.21.15.133:0/3981046113 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd04006b4d0 0x7fd04006d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd054000f60 tx=0x7fd054009040 comp rx=0 tx=0).stop 2024-03-07T06:52:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.466+0000 7fd06a800640 1 -- 172.21.15.133:0/3981046113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd06415fbb0 msgr2=0x7fd06409c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.466+0000 7fd06a800640 1 --2- 172.21.15.133:0/3981046113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd06415fbb0 0x7fd06409c140 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009890 tx=0x7fd04c0046c0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.467+0000 7fd06a800640 1 -- 172.21.15.133:0/3981046113 shutdown_connections 2024-03-07T06:52:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.467+0000 7fd06a800640 1 --2- 172.21.15.133:0/3981046113 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd06409c680 0x7fd0640c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.467+0000 7fd06a800640 1 --2- 172.21.15.133:0/3981046113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd06415fbb0 0x7fd06409c140 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.467+0000 7fd06a800640 1 --2- 172.21.15.133:0/3981046113 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd04006b4d0 0x7fd04006d990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.467+0000 7fd06a800640 1 --2- 172.21.15.133:0/3981046113 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd0640cdee0 0x7fd064151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.467+0000 7fd06a800640 1 -- 172.21.15.133:0/3981046113 >> 172.21.15.133:0/3981046113 conn(0x7fd0640b3120 msgr2=0x7fd06415ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.467+0000 7fd06a800640 1 -- 172.21.15.133:0/3981046113 shutdown_connections 2024-03-07T06:52:56.467 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.467+0000 7fd06a800640 1 -- 172.21.15.133:0/3981046113 wait complete. 2024-03-07T06:52:56.484 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump mds_cache strays_enqueued 2024-03-07T06:52:56.691 INFO:teuthology.orchestra.run.smithi133.stdout:{"mds_cache":{"strays_enqueued":1001}} 2024-03-07T06:52:56.692 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_cache": { "strays_enqueued": 1001 } } 2024-03-07T06:52:56.692 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:52:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.889+0000 7f797f600640 1 Processor -- start 2024-03-07T06:52:56.889 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.889+0000 7f797f600640 1 -- start start 2024-03-07T06:52:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.891+0000 7f797f600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f797806dab0 0x7f7978071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:56.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.891+0000 7f797f600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7978070400 0x7f797806de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:56.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.891+0000 7f797f600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797815e650 con 0x7f79780a9430 2024-03-07T06:52:56.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.891+0000 7f797f600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f797806e3d0 con 0x7f797806dab0 2024-03-07T06:52:56.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.891+0000 7f797f600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7978070970 con 0x7f7978070400 2024-03-07T06:52:56.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.891+0000 7f797cc00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f797806dab0 0x7f7978071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:56.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.891+0000 7f797d600640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f79780a9430 0x7f7978161d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47314/0 (socket says 172.21.15.133:47314) 2024-03-07T06:52:56.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.891+0000 7f797d600640 1 -- 172.21.15.133:0/3361734428 learned_addr learned my addr 172.21.15.133:0/3361734428 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:56.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.892+0000 7f796fe00640 1 --2- 172.21.15.133:0/3361734428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7978070400 0x7f797806de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:56.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.892+0000 7f797cc00640 1 -- 172.21.15.133:0/3361734428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7978070400 msgr2=0x7f797806de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:56.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.892+0000 7f797cc00640 1 --2- 172.21.15.133:0/3361734428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7978070400 0x7f797806de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.892+0000 7f797cc00640 1 -- 172.21.15.133:0/3361734428 >> v1:172.21.15.133:6789/0 conn(0x7f79780a9430 legacy=0x7f7978161d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:56.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.892+0000 7f797cc00640 1 -- 172.21.15.133:0/3361734428 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7978070c50 con 0x7f797806dab0 2024-03-07T06:52:56.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.892+0000 7f7977e00640 1 -- 172.21.15.133:0/3361734428 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747386386 0 0) 0x7f797815e650 con 0x7f79780a9430 2024-03-07T06:52:56.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.893+0000 7f797cc00640 1 --2- 172.21.15.133:0/3361734428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f797806dab0 0x7f7978071a10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f79680097a0 tx=0x7f796802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b70714ef8822ff server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:56.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.893+0000 7f7977e00640 1 -- 172.21.15.133:0/3361734428 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968035020 con 0x7f797806dab0 2024-03-07T06:52:56.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.894+0000 7f7977e00640 1 -- 172.21.15.133:0/3361734428 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7968031830 con 0x7f797806dab0 2024-03-07T06:52:56.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.894+0000 7f7977e00640 1 -- 172.21.15.133:0/3361734428 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968031c00 con 0x7f797806dab0 2024-03-07T06:52:56.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.894+0000 7f797f600640 1 -- 172.21.15.133:0/3361734428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f797806dab0 msgr2=0x7f7978071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:56.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.894+0000 7f797f600640 1 --2- 172.21.15.133:0/3361734428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f797806dab0 0x7f7978071a10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f79680097a0 tx=0x7f796802b8d0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.894+0000 7f797f600640 1 -- 172.21.15.133:0/3361734428 shutdown_connections 2024-03-07T06:52:56.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.894+0000 7f797f600640 1 --2- 172.21.15.133:0/3361734428 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7978070400 0x7f797806de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.894+0000 7f797f600640 1 --2- 172.21.15.133:0/3361734428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f797806dab0 0x7f7978071a10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.894 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.894+0000 7f797f600640 1 -- 172.21.15.133:0/3361734428 >> 172.21.15.133:0/3361734428 conn(0x7f79780cd160 msgr2=0x7f79780cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:56.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.894+0000 7f797f600640 1 -- 172.21.15.133:0/3361734428 shutdown_connections 2024-03-07T06:52:56.895 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.894+0000 7f797f600640 1 -- 172.21.15.133:0/3361734428 wait complete. 2024-03-07T06:52:56.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.897+0000 7f797f600640 1 Processor -- start 2024-03-07T06:52:56.896 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.897+0000 7f797f600640 1 -- start start 2024-03-07T06:52:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.899+0000 7f797f600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f797806dab0 0x7f797814d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.899+0000 7f797f600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7978070400 0x7f79781385e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.899+0000 7f797f600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79780a9430 0x7f7978138b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.899+0000 7f797f600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79780714f0 con 0x7f79780a9430 2024-03-07T06:52:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.899+0000 7f797f600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7978071210 con 0x7f7978070400 2024-03-07T06:52:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.899+0000 7f797f600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79780673b0 con 0x7f797806dab0 2024-03-07T06:52:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.899+0000 7f797d600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79780a9430 0x7f7978138b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.899+0000 7f797d600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79780a9430 0x7f7978138b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36914/0 (socket says 172.21.15.133:36914) 2024-03-07T06:52:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.899+0000 7f797d600640 1 -- 172.21.15.133:0/625813647 learned_addr learned my addr 172.21.15.133:0/625813647 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.899+0000 7f797cc00640 1 --2- 172.21.15.133:0/625813647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f797806dab0 0x7f797814d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.899+0000 7f796fe00640 1 --2- 172.21.15.133:0/625813647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7978070400 0x7f79781385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.900+0000 7f797d600640 1 -- 172.21.15.133:0/625813647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f797806dab0 msgr2=0x7f797814d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.900+0000 7f797d600640 1 --2- 172.21.15.133:0/625813647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f797806dab0 0x7f797814d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.900+0000 7f797d600640 1 -- 172.21.15.133:0/625813647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7978070400 msgr2=0x7f79781385e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.900+0000 7f797d600640 1 --2- 172.21.15.133:0/625813647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7978070400 0x7f79781385e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.900+0000 7f797d600640 1 -- 172.21.15.133:0/625813647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7978139420 con 0x7f79780a9430 2024-03-07T06:52:56.900 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.900+0000 7f797d600640 1 --2- 172.21.15.133:0/625813647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79780a9430 0x7f7978138b20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f797000c800 tx=0x7f797000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:56.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.900+0000 7f7975600640 1 -- 172.21.15.133:0/625813647 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7970018070 con 0x7f79780a9430 2024-03-07T06:52:56.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.900+0000 7f7975600640 1 -- 172.21.15.133:0/625813647 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7970004d80 con 0x7f79780a9430 2024-03-07T06:52:56.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.900+0000 7f797f600640 1 -- 172.21.15.133:0/625813647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797813c3a0 con 0x7f79780a9430 2024-03-07T06:52:56.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.900+0000 7f7975600640 1 -- 172.21.15.133:0/625813647 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797000a880 con 0x7f79780a9430 2024-03-07T06:52:56.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.900+0000 7f797f600640 1 -- 172.21.15.133:0/625813647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797815f9a0 con 0x7f79780a9430 2024-03-07T06:52:56.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.902+0000 7f797f600640 1 -- 172.21.15.133:0/625813647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7978062c90 con 0x7f79780a9430 2024-03-07T06:52:56.902 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.903+0000 7f7975600640 1 -- 172.21.15.133:0/625813647 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f797001b070 con 0x7f79780a9430 2024-03-07T06:52:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.903+0000 7f7975600640 1 --2- 172.21.15.133:0/625813647 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f795406b4d0 0x7f795406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.904+0000 7f7975600640 1 -- 172.21.15.133:0/625813647 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7970089700 con 0x7f79780a9430 2024-03-07T06:52:56.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.904+0000 7f797cc00640 1 --2- 172.21.15.133:0/625813647 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f795406b4d0 0x7f795406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:56.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.905+0000 7f797cc00640 1 --2- 172.21.15.133:0/625813647 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f795406b4d0 0x7f795406d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7968036000 tx=0x7f7968008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:56.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:56.908+0000 7f7975600640 1 -- 172.21.15.133:0/625813647 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7970058630 con 0x7f79780a9430 2024-03-07T06:52:57.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.145+0000 7f797f600640 1 -- 172.21.15.133:0/625813647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f797806f340 con 0x7f79780a9430 2024-03-07T06:52:57.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.146+0000 7f7975600640 1 -- 172.21.15.133:0/625813647 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4983 (secure 0 0 0) 0x7f7970057f30 con 0x7f79780a9430 2024-03-07T06:52:57.146 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:57.146 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3269394125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3269394125},{"type":"v1","addr":"172.21.15.161:6835","nonce":3269394125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/72243172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":72243172},{"type":"v1","addr":"172.21.15.161:6837","nonce":72243172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3714291615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3714291615},{"type":"v1","addr":"172.21.15.133:6834","nonce":3714291615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:52:35.182567+0000","modified":"2024-03-07T06:52:38.268724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6835/4067860709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":4067860709},{"type":"v1","addr":"172.21.15.133:6835","nonce":4067860709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5100,"qdb_cluster":[5100]},"id":2}]} 2024-03-07T06:52:57.146 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 18 2024-03-07T06:52:57.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.149+0000 7f797f600640 1 -- 172.21.15.133:0/625813647 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f795406b4d0 msgr2=0x7f795406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:57.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.149+0000 7f797f600640 1 --2- 172.21.15.133:0/625813647 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f795406b4d0 0x7f795406d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7968036000 tx=0x7f7968008030 comp rx=0 tx=0).stop 2024-03-07T06:52:57.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.149+0000 7f797f600640 1 -- 172.21.15.133:0/625813647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79780a9430 msgr2=0x7f7978138b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:57.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.149+0000 7f797f600640 1 --2- 172.21.15.133:0/625813647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79780a9430 0x7f7978138b20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f797000c800 tx=0x7f797000ccc0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.149+0000 7f797f600640 1 -- 172.21.15.133:0/625813647 shutdown_connections 2024-03-07T06:52:57.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.149+0000 7f797f600640 1 --2- 172.21.15.133:0/625813647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79780a9430 0x7f7978138b20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.149+0000 7f797f600640 1 --2- 172.21.15.133:0/625813647 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f795406b4d0 0x7f795406d990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.149+0000 7f797f600640 1 --2- 172.21.15.133:0/625813647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7978070400 0x7f79781385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.149+0000 7f797f600640 1 --2- 172.21.15.133:0/625813647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f797806dab0 0x7f797814d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.149+0000 7f797f600640 1 -- 172.21.15.133:0/625813647 >> 172.21.15.133:0/625813647 conn(0x7f79780cd160 msgr2=0x7f7978154480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:57.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.149+0000 7f797f600640 1 -- 172.21.15.133:0/625813647 shutdown_connections 2024-03-07T06:52:57.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.149+0000 7f797f600640 1 -- 172.21.15.133:0/625813647 wait complete. 2024-03-07T06:52:57.168 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json perf dump purge_queue pq_executed 2024-03-07T06:52:57.394 INFO:teuthology.orchestra.run.smithi133.stdout:{"purge_queue":{"pq_executed":1001}} 2024-03-07T06:52:57.395 DEBUG:tasks.cephfs.filesystem:_json_asok output { "purge_queue": { "pq_executed": 1001 } } 2024-03-07T06:52:57.395 DEBUG:teuthology.orchestra.run.smithi133:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata stat 10000000000.00000000 2024-03-07T06:52:57.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.487+0000 7fe74f037f40 1 Processor -- start 2024-03-07T06:52:57.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.488+0000 7fe74f037f40 1 -- start start 2024-03-07T06:52:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.489+0000 7fe74f037f40 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7d67cef60 0x55d7d679af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.489+0000 7fe74f037f40 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d7d67e7440 0x55d7d67ca530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.489+0000 7fe74f037f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7d6874420 0x55d7d679d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.489+0000 7fe74f037f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d6743ac0 con 0x55d7d6874420 2024-03-07T06:52:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.489+0000 7fe74f037f40 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d679ce70 con 0x55d7d67cef60 2024-03-07T06:52:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.489+0000 7fe74f037f40 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7d686e930 con 0x55d7d67e7440 2024-03-07T06:52:57.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.489+0000 7fe74c200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7d6874420 0x55d7d679d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.489+0000 7fe74c200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7d6874420 0x55d7d679d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36932/0 (socket says 172.21.15.133:36932) 2024-03-07T06:52:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.489+0000 7fe74c200640 1 -- 172.21.15.133:0/4292777755 learned_addr learned my addr 172.21.15.133:0/4292777755 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.490+0000 7fe74b800640 1 --2- 172.21.15.133:0/4292777755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7d67cef60 0x55d7d679af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.490+0000 7fe743e00640 1 --2- 172.21.15.133:0/4292777755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d7d67e7440 0x55d7d67ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.490+0000 7fe74c200640 1 -- 172.21.15.133:0/4292777755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d7d67e7440 msgr2=0x55d7d67ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.490+0000 7fe74c200640 1 --2- 172.21.15.133:0/4292777755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d7d67e7440 0x55d7d67ca530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.490+0000 7fe74c200640 1 -- 172.21.15.133:0/4292777755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7d67cef60 msgr2=0x55d7d679af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.490+0000 7fe74c200640 1 --2- 172.21.15.133:0/4292777755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7d67cef60 0x55d7d679af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.490+0000 7fe74c200640 1 -- 172.21.15.133:0/4292777755 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7d6872ff0 con 0x55d7d6874420 2024-03-07T06:52:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.490+0000 7fe74b800640 1 --2- 172.21.15.133:0/4292777755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7d67cef60 0x55d7d679af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.490+0000 7fe74c200640 1 --2- 172.21.15.133:0/4292777755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7d6874420 0x55d7d679d300 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe74400a4b0 tx=0x7fe74400a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c3c225d6f1a417 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.490+0000 7fe74a400640 1 -- 172.21.15.133:0/4292777755 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744015070 con 0x55d7d6874420 2024-03-07T06:52:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.490+0000 7fe74a400640 1 -- 172.21.15.133:0/4292777755 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74400e730 con 0x55d7d6874420 2024-03-07T06:52:57.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.490+0000 7fe74a400640 1 -- 172.21.15.133:0/4292777755 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74400eb20 con 0x55d7d6874420 2024-03-07T06:52:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.491+0000 7fe74f037f40 1 -- 172.21.15.133:0/4292777755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7d6874420 msgr2=0x55d7d679d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.491+0000 7fe74f037f40 1 --2- 172.21.15.133:0/4292777755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7d6874420 0x55d7d679d300 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe74400a4b0 tx=0x7fe74400a970 comp rx=0 tx=0).stop 2024-03-07T06:52:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.491+0000 7fe74f037f40 1 -- 172.21.15.133:0/4292777755 shutdown_connections 2024-03-07T06:52:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.491+0000 7fe74f037f40 1 --2- 172.21.15.133:0/4292777755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7d6874420 0x55d7d679d300 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.491+0000 7fe74f037f40 1 --2- 172.21.15.133:0/4292777755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d7d67e7440 0x55d7d67ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.491+0000 7fe74f037f40 1 --2- 172.21.15.133:0/4292777755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7d67cef60 0x55d7d679af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.491+0000 7fe74f037f40 1 -- 172.21.15.133:0/4292777755 >> 172.21.15.133:0/4292777755 conn(0x55d7d67ea8e0 msgr2=0x55d7d6877dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.491+0000 7fe74f037f40 1 -- 172.21.15.133:0/4292777755 shutdown_connections 2024-03-07T06:52:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.491+0000 7fe74f037f40 1 -- 172.21.15.133:0/4292777755 wait complete. 2024-03-07T06:52:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.492+0000 7fe74f037f40 1 Processor -- start 2024-03-07T06:52:57.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.493+0000 7fe74f037f40 1 -- start start 2024-03-07T06:52:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.494+0000 7fe74f037f40 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d7d67cef60 0x55d7d6856140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.494+0000 7fe74f037f40 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7d67e7440 0x55d7d679d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.494+0000 7fe74f037f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7d6874420 0x55d7d690e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.494+0000 7fe74f037f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d6873570 con 0x55d7d6874420 2024-03-07T06:52:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.494+0000 7fe74f037f40 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7d6873290 con 0x55d7d67e7440 2024-03-07T06:52:57.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.494+0000 7fe74f037f40 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7d68785b0 con 0x55d7d67cef60 2024-03-07T06:52:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.494+0000 7fe74c200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7d6874420 0x55d7d690e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.494+0000 7fe74c200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7d6874420 0x55d7d690e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36940/0 (socket says 172.21.15.133:36940) 2024-03-07T06:52:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.494+0000 7fe74c200640 1 -- 172.21.15.133:0/1360653881 learned_addr learned my addr 172.21.15.133:0/1360653881 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.495+0000 7fe74b800640 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d7d67cef60 0x55d7d6856140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.495+0000 7fe743e00640 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7d67e7440 0x55d7d679d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.495+0000 7fe74c200640 1 -- 172.21.15.133:0/1360653881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d7d67cef60 msgr2=0x55d7d6856140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.495+0000 7fe74c200640 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d7d67cef60 0x55d7d6856140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.495+0000 7fe74c200640 1 -- 172.21.15.133:0/1360653881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7d67e7440 msgr2=0x55d7d679d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.495+0000 7fe74c200640 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7d67e7440 0x55d7d679d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.495+0000 7fe74c200640 1 -- 172.21.15.133:0/1360653881 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7d69030a0 con 0x55d7d6874420 2024-03-07T06:52:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.495+0000 7fe743e00640 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d7d67e7440 0x55d7d679d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.495+0000 7fe74c200640 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7d6874420 0x55d7d690e8e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe744010000 tx=0x7fe7440032f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:57.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.495+0000 7fe749000640 1 -- 172.21.15.133:0/1360653881 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744020070 con 0x55d7d6874420 2024-03-07T06:52:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.495+0000 7fe749000640 1 -- 172.21.15.133:0/1360653881 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe744016040 con 0x55d7d6874420 2024-03-07T06:52:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.496+0000 7fe749000640 1 -- 172.21.15.133:0/1360653881 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744015070 con 0x55d7d6874420 2024-03-07T06:52:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.496+0000 7fe74f037f40 1 -- 172.21.15.133:0/1360653881 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d7d68fed80 con 0x55d7d6874420 2024-03-07T06:52:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.496+0000 7fe74f037f40 1 -- 172.21.15.133:0/1360653881 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7d68feaa0 con 0x55d7d6874420 2024-03-07T06:52:57.497 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.498+0000 7fe749000640 1 -- 172.21.15.133:0/1360653881 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe74401cb30 con 0x55d7d6874420 2024-03-07T06:52:57.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.498+0000 7fe749000640 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe728069eb0 0x7fe72806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.499+0000 7fe749000640 1 -- 172.21.15.133:0/1360653881 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe74408a310 con 0x55d7d6874420 2024-03-07T06:52:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.499+0000 7fe74f037f40 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x55d7d679da30 0x55d7d690f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.499+0000 7fe74f037f40 1 -- 172.21.15.133:0/1360653881 --> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v9 -- 0x55d7d6905340 con 0x55d7d679da30 2024-03-07T06:52:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.499+0000 7fe74b800640 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe728069eb0 0x7fe72806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.499+0000 7fe743e00640 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x55d7d679da30 0x55d7d690f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.500+0000 7fe743e00640 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x55d7d679da30 0x55d7d690f330 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.500+0000 7fe743e00640 1 -- 172.21.15.133:0/1360653881 <== osd.3 v2:172.21.15.133:6816/2661528260 1 ==== osd_op_reply(1 10000000000.00000000 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe7340097b0 con 0x55d7d679da30 2024-03-07T06:52:57.501 INFO:teuthology.orchestra.run.smithi133.stderr: error stat-ing cephfs_metadata/10000000000.00000000: (2) No such file or directory 2024-03-07T06:52:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.500+0000 7fe74b800640 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe728069eb0 0x7fe72806c370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe73c057000 tx=0x7fe73c052000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.500+0000 7fe74f037f40 1 -- 172.21.15.133:0/1360653881 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x55d7d679da30 msgr2=0x55d7d690f330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.500+0000 7fe74f037f40 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x55d7d679da30 0x55d7d690f330 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.500+0000 7fe74f037f40 1 -- 172.21.15.133:0/1360653881 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe728069eb0 msgr2=0x7fe72806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.500+0000 7fe74f037f40 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe728069eb0 0x7fe72806c370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe73c057000 tx=0x7fe73c052000 comp rx=0 tx=0).stop 2024-03-07T06:52:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.500+0000 7fe74f037f40 1 -- 172.21.15.133:0/1360653881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7d6874420 msgr2=0x55d7d690e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.500+0000 7fe74f037f40 1 --2- 172.21.15.133:0/1360653881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7d6874420 0x55d7d690e8e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe744010000 tx=0x7fe7440032f0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.500+0000 7fe74c200640 1 -- 172.21.15.133:0/1360653881 reap_dead start 2024-03-07T06:52:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.500+0000 7fe74f037f40 1 -- 172.21.15.133:0/1360653881 shutdown_connections 2024-03-07T06:52:57.501 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.500+0000 7fe74f037f40 1 -- 172.21.15.133:0/1360653881 >> 172.21.15.133:0/1360653881 conn(0x55d7d67ea8e0 msgr2=0x55d7d6876ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.501+0000 7fe74f037f40 1 -- 172.21.15.133:0/1360653881 shutdown_connections 2024-03-07T06:52:57.502 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.501+0000 7fe74f037f40 1 -- 172.21.15.133:0/1360653881 wait complete. 2024-03-07T06:52:57.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:52:57.505 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2024-03-07T06:52:57.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.690+0000 7f7b85600640 1 Processor -- start 2024-03-07T06:52:57.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.690+0000 7f7b85600640 1 -- start start 2024-03-07T06:52:57.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.692+0000 7f7b85600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b800b7560 0x7f7b800d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.692+0000 7f7b85600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b800b79d0 con 0x7f7b800b70d0 2024-03-07T06:52:57.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.692+0000 7f7b85600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b80143810 con 0x7f7b80140cb0 2024-03-07T06:52:57.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.692+0000 7f7b85600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b801435c0 con 0x7f7b800b7560 2024-03-07T06:52:57.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.692+0000 7f7b7f400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7b800b70d0 0x7f7b800d1af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47322/0 (socket says 172.21.15.133:47322) 2024-03-07T06:52:57.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.692+0000 7f7b7f400640 1 -- 172.21.15.133:0/2857150311 learned_addr learned my addr 172.21.15.133:0/2857150311 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:52:57.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b84c00640 1 --2- 172.21.15.133:0/2857150311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b800b7560 0x7f7b800d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 -- 172.21.15.133:0/2857150311 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775737403 0 0) 0x7f7b800b79d0 con 0x7f7b800b70d0 2024-03-07T06:52:57.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 -- 172.21.15.133:0/2857150311 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b64003520 con 0x7f7b800b70d0 2024-03-07T06:52:57.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 -- 172.21.15.133:0/2857150311 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4066720227 0 0) 0x7f7b64003520 con 0x7f7b800b70d0 2024-03-07T06:52:57.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 -- 172.21.15.133:0/2857150311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b800b7560 msgr2=0x7f7b800d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:57.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 --2- 172.21.15.133:0/2857150311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b800b7560 0x7f7b800d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 -- 172.21.15.133:0/2857150311 >> v1:172.21.15.161:6789/0 conn(0x7f7b80140cb0 legacy=0x7f7b80162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:57.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 -- 172.21.15.133:0/2857150311 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b80143af0 con 0x7f7b800b70d0 2024-03-07T06:52:57.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 -- 172.21.15.133:0/2857150311 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f7b6c001670 con 0x7f7b800b70d0 2024-03-07T06:52:57.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 -- 172.21.15.133:0/2857150311 >> v1:172.21.15.133:6789/0 conn(0x7f7b800b70d0 legacy=0x7f7b800d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:57.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 --2- 172.21.15.133:0/2857150311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b640039e0 0x7f7b64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 --2- 172.21.15.133:0/2857150311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b800b7560 0x7f7b64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 --2- 172.21.15.133:0/2857150311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b64004d40 0x7f7b640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 -- 172.21.15.133:0/2857150311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b801435c0 con 0x7f7b64004d40 2024-03-07T06:52:57.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 -- 172.21.15.133:0/2857150311 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6400baa0 con 0x7f7b640039e0 2024-03-07T06:52:57.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b7ea00640 1 -- 172.21.15.133:0/2857150311 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6400bca0 con 0x7f7b800b7560 2024-03-07T06:52:57.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.693+0000 7f7b84c00640 1 --2- 172.21.15.133:0/2857150311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b64004d40 0x7f7b640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.694+0000 7f7b7fe00640 1 --2- 172.21.15.133:0/2857150311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b640039e0 0x7f7b64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.694+0000 7f7b7f400640 1 --2- 172.21.15.133:0/2857150311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b800b7560 0x7f7b64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.694+0000 7f7b84c00640 1 -- 172.21.15.133:0/2857150311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b800b7560 msgr2=0x7f7b64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:57.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.694+0000 7f7b84c00640 1 --2- 172.21.15.133:0/2857150311 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b800b7560 0x7f7b64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.694+0000 7f7b84c00640 1 -- 172.21.15.133:0/2857150311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b640039e0 msgr2=0x7f7b64003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:52:57.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.694+0000 7f7b84c00640 1 --2- 172.21.15.133:0/2857150311 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b640039e0 0x7f7b64003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.694+0000 7f7b84c00640 1 -- 172.21.15.133:0/2857150311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64003520 con 0x7f7b64004d40 2024-03-07T06:52:57.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.696+0000 7f7b84c00640 1 --2- 172.21.15.133:0/2857150311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b64004d40 0x7f7b640051c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009f90 tx=0x7f7b64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4ad0872056b1e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:57.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.696+0000 7f7b7ea00640 1 -- 172.21.15.133:0/2857150311 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78039070 con 0x7f7b64004d40 2024-03-07T06:52:57.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.696+0000 7f7b7ea00640 1 -- 172.21.15.133:0/2857150311 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7802cbd0 con 0x7f7b64004d40 2024-03-07T06:52:57.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.696+0000 7f7b7ea00640 1 -- 172.21.15.133:0/2857150311 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78009c10 con 0x7f7b64004d40 2024-03-07T06:52:57.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.696+0000 7f7b85600640 1 -- 172.21.15.133:0/2857150311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b64004d40 msgr2=0x7f7b640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:57.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.696+0000 7f7b85600640 1 --2- 172.21.15.133:0/2857150311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b64004d40 0x7f7b640051c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009f90 tx=0x7f7b64005830 comp rx=0 tx=0).stop 2024-03-07T06:52:57.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.697+0000 7f7b84c00640 1 -- 172.21.15.133:0/2857150311 reap_dead start 2024-03-07T06:52:57.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.697+0000 7f7b85600640 1 -- 172.21.15.133:0/2857150311 shutdown_connections 2024-03-07T06:52:57.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.697+0000 7f7b85600640 1 -- 172.21.15.133:0/2857150311 >> 172.21.15.133:0/2857150311 conn(0x7f7b800674e0 msgr2=0x7f7b800d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:57.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.697+0000 7f7b85600640 1 -- 172.21.15.133:0/2857150311 shutdown_connections 2024-03-07T06:52:57.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.697+0000 7f7b85600640 1 -- 172.21.15.133:0/2857150311 wait complete. 2024-03-07T06:52:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.698+0000 7f7b85600640 1 Processor -- start 2024-03-07T06:52:57.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.699+0000 7f7b85600640 1 -- start start 2024-03-07T06:52:57.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b85600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b8014add0 0x7f7b80155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b85600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b8014b1b0 0x7f7b80156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b85600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80154130 0x7f7b80071460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b85600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b801440b0 con 0x7f7b80154130 2024-03-07T06:52:57.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b85600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80144420 con 0x7f7b8014b1b0 2024-03-07T06:52:57.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b85600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80161bb0 con 0x7f7b8014add0 2024-03-07T06:52:57.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b84c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80154130 0x7f7b80071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b84c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80154130 0x7f7b80071460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36964/0 (socket says 172.21.15.133:36964) 2024-03-07T06:52:57.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b84c00640 1 -- 172.21.15.133:0/505797953 learned_addr learned my addr 172.21.15.133:0/505797953 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:52:57.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b7f400640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b8014b1b0 0x7f7b80156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b7fe00640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b8014add0 0x7f7b80155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b84c00640 1 -- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b8014add0 msgr2=0x7f7b80155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:57.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b84c00640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b8014add0 0x7f7b80155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b84c00640 1 -- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b8014b1b0 msgr2=0x7f7b80156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b84c00640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b8014b1b0 0x7f7b80156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b84c00640 1 -- 172.21.15.133:0/505797953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b80155a90 con 0x7f7b80154130 2024-03-07T06:52:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.700+0000 7f7b7f400640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b8014b1b0 0x7f7b80156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:52:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.701+0000 7f7b84c00640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80154130 0x7f7b80071460 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800e230 tx=0x7f7b78009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.701+0000 7f7b77e00640 1 -- 172.21.15.133:0/505797953 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78039070 con 0x7f7b80154130 2024-03-07T06:52:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.701+0000 7f7b77e00640 1 -- 172.21.15.133:0/505797953 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7803a040 con 0x7f7b80154130 2024-03-07T06:52:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.701+0000 7f7b77e00640 1 -- 172.21.15.133:0/505797953 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78009c10 con 0x7f7b80154130 2024-03-07T06:52:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.701+0000 7f7b85600640 1 -- 172.21.15.133:0/505797953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b80155d70 con 0x7f7b80154130 2024-03-07T06:52:57.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.701+0000 7f7b85600640 1 -- 172.21.15.133:0/505797953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b80156330 con 0x7f7b80154130 2024-03-07T06:52:57.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.702+0000 7f7b85600640 1 -- 172.21.15.133:0/505797953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b80062c90 con 0x7f7b80154130 2024-03-07T06:52:57.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.703+0000 7f7b77e00640 1 -- 172.21.15.133:0/505797953 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7b78046030 con 0x7f7b80154130 2024-03-07T06:52:57.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.705+0000 7f7b77e00640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7b6006b500 0x7f7b6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:52:57.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.705+0000 7f7b77e00640 1 -- 172.21.15.133:0/505797953 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b780abc90 con 0x7f7b80154130 2024-03-07T06:52:57.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.706+0000 7f7b7fe00640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7b6006b500 0x7f7b6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:52:57.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.707+0000 7f7b7fe00640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7b6006b500 0x7f7b6006d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b70007fd0 tx=0x7f7b70009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:52:57.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:57.708+0000 7f7b77e00640 1 -- 172.21.15.133:0/505797953 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7b7804d060 con 0x7f7b80154130 2024-03-07T06:52:58.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:58.039+0000 7f7b85600640 1 -- 172.21.15.133:0/505797953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f7b800b9f20 con 0x7f7b80154130 2024-03-07T06:52:58.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:58.040+0000 7f7b77e00640 1 -- 172.21.15.133:0/505797953 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1695 (secure 0 0 0) 0x7f7b7807aba0 con 0x7f7b80154130 2024-03-07T06:52:58.040 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:52:58.040 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:52:58.040 INFO:teuthology.orchestra.run.smithi133.stdout: "stats": { 2024-03-07T06:52:58.040 INFO:teuthology.orchestra.run.smithi133.stdout: "total_bytes": 773094113280, 2024-03-07T06:52:58.040 INFO:teuthology.orchestra.run.smithi133.stdout: "total_avail_bytes": 772836065280, 2024-03-07T06:52:58.040 INFO:teuthology.orchestra.run.smithi133.stdout: "total_used_bytes": 258048000, 2024-03-07T06:52:58.040 INFO:teuthology.orchestra.run.smithi133.stdout: "total_used_raw_bytes": 258048000, 2024-03-07T06:52:58.040 INFO:teuthology.orchestra.run.smithi133.stdout: "total_used_raw_ratio": 0.0003337860107421875, 2024-03-07T06:52:58.041 INFO:teuthology.orchestra.run.smithi133.stdout: "num_osds": 8, 2024-03-07T06:52:58.041 INFO:teuthology.orchestra.run.smithi133.stdout: "num_per_pool_osds": 8, 2024-03-07T06:52:58.041 INFO:teuthology.orchestra.run.smithi133.stdout: "num_per_pool_omap_osds": 8 2024-03-07T06:52:58.041 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:58.041 INFO:teuthology.orchestra.run.smithi133.stdout: "stats_by_class": { 2024-03-07T06:52:58.041 INFO:teuthology.orchestra.run.smithi133.stdout: "ssd": { 2024-03-07T06:52:58.041 INFO:teuthology.orchestra.run.smithi133.stdout: "total_bytes": 773094113280, 2024-03-07T06:52:58.041 INFO:teuthology.orchestra.run.smithi133.stdout: "total_avail_bytes": 772836065280, 2024-03-07T06:52:58.041 INFO:teuthology.orchestra.run.smithi133.stdout: "total_used_bytes": 258048000, 2024-03-07T06:52:58.041 INFO:teuthology.orchestra.run.smithi133.stdout: "total_used_raw_bytes": 258048000, 2024-03-07T06:52:58.041 INFO:teuthology.orchestra.run.smithi133.stdout: "total_used_raw_ratio": 0.0003337860107421875 2024-03-07T06:52:58.041 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:58.041 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:58.041 INFO:teuthology.orchestra.run.smithi133.stdout: "pools": [ 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: "name": ".mgr", 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 1, 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: "stats": { 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: "stored": 590368, 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": 2, 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: "kb_used": 1160, 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: "bytes_used": 1187840, 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: "percent_used": 1.6179950534933596e-06, 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: "max_avail": 367071002624 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephfs_metadata", 2024-03-07T06:52:58.042 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4, 2024-03-07T06:52:58.043 INFO:teuthology.orchestra.run.smithi133.stdout: "stats": { 2024-03-07T06:52:58.043 INFO:teuthology.orchestra.run.smithi133.stdout: "stored": 8560688, 2024-03-07T06:52:58.043 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": 25, 2024-03-07T06:52:58.043 INFO:teuthology.orchestra.run.smithi133.stdout: "kb_used": 16784, 2024-03-07T06:52:58.043 INFO:teuthology.orchestra.run.smithi133.stdout: "bytes_used": 17186816, 2024-03-07T06:52:58.043 INFO:teuthology.orchestra.run.smithi133.stdout: "percent_used": 2.3410204448737204e-05, 2024-03-07T06:52:58.043 INFO:teuthology.orchestra.run.smithi133.stdout: "max_avail": 367071002624 2024-03-07T06:52:58.043 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:58.043 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:52:58.043 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:52:58.043 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephfs_data", 2024-03-07T06:52:58.043 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 5, 2024-03-07T06:52:58.043 INFO:teuthology.orchestra.run.smithi133.stdout: "stats": { 2024-03-07T06:52:58.043 INFO:teuthology.orchestra.run.smithi133.stdout: "stored": 300032, 2024-03-07T06:52:58.044 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": 293, 2024-03-07T06:52:58.044 INFO:teuthology.orchestra.run.smithi133.stdout: "kb_used": 2344, 2024-03-07T06:52:58.044 INFO:teuthology.orchestra.run.smithi133.stdout: "bytes_used": 2400256, 2024-03-07T06:52:58.044 INFO:teuthology.orchestra.run.smithi133.stdout: "percent_used": 3.2694601941329893e-06, 2024-03-07T06:52:58.044 INFO:teuthology.orchestra.run.smithi133.stdout: "max_avail": 367071002624 2024-03-07T06:52:58.044 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:58.044 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:52:58.044 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-03-07T06:52:58.044 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:52:58.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:58.042+0000 7f7b85600640 1 -- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7b6006b500 msgr2=0x7f7b6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:58.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:58.042+0000 7f7b85600640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7b6006b500 0x7f7b6006d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b70007fd0 tx=0x7f7b70009040 comp rx=0 tx=0).stop 2024-03-07T06:52:58.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:58.042+0000 7f7b85600640 1 -- 172.21.15.133:0/505797953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80154130 msgr2=0x7f7b80071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:52:58.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:58.042+0000 7f7b85600640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80154130 0x7f7b80071460 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800e230 tx=0x7f7b78009750 comp rx=0 tx=0).stop 2024-03-07T06:52:58.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:58.043+0000 7f7b85600640 1 -- 172.21.15.133:0/505797953 shutdown_connections 2024-03-07T06:52:58.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:58.043+0000 7f7b85600640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80154130 0x7f7b80071460 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:58.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:58.043+0000 7f7b85600640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7b6006b500 0x7f7b6006d9c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:58.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:58.043+0000 7f7b85600640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b8014b1b0 0x7f7b80156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:58.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:58.043+0000 7f7b85600640 1 --2- 172.21.15.133:0/505797953 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b8014add0 0x7f7b80155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:52:58.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:58.043+0000 7f7b85600640 1 -- 172.21.15.133:0/505797953 >> 172.21.15.133:0/505797953 conn(0x7f7b800674e0 msgr2=0x7f7b800b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:52:58.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:58.043+0000 7f7b85600640 1 -- 172.21.15.133:0/505797953 shutdown_connections 2024-03-07T06:52:58.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:52:58.043+0000 7f7b85600640 1 -- 172.21.15.133:0/505797953 wait complete. 2024-03-07T06:52:58.061 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=60 retry_count=0)... 2024-03-07T06:52:59.554 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:52:59.563 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:53:03.089 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2024-03-07T06:53:03.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.297+0000 7f71b7c00640 1 Processor -- start 2024-03-07T06:53:03.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.297+0000 7f71b7c00640 1 -- start start 2024-03-07T06:53:03.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.299+0000 7f71b7c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f71b0072950 0x7f71b00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:03.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.299+0000 7f71b7c00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b00731b0 con 0x7f71b00cdee0 2024-03-07T06:53:03.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.299+0000 7f71b7c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b0072150 con 0x7f71b015fbb0 2024-03-07T06:53:03.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.299+0000 7f71b7c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b0072f60 con 0x7f71b0072950 2024-03-07T06:53:03.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.299+0000 7f71b5c00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f71b00cdee0 0x7f71b015c550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50140/0 (socket says 172.21.15.133:50140) 2024-03-07T06:53:03.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.299+0000 7f71b5c00640 1 -- 172.21.15.133:0/2947981123 learned_addr learned my addr 172.21.15.133:0/2947981123 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:03.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71afe00640 1 --2- 172.21.15.133:0/2947981123 >> v2:172.21.15.161:3300/0 conn(0x7f71b0072950 0x7f71b00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:03.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 -- 172.21.15.133:0/2947981123 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291626090 0 0) 0x7f71b00731b0 con 0x7f71b00cdee0 2024-03-07T06:53:03.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 -- 172.21.15.133:0/2947981123 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7198003520 con 0x7f71b00cdee0 2024-03-07T06:53:03.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 -- 172.21.15.133:0/2947981123 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 737440870 0 0) 0x7f7198003520 con 0x7f71b00cdee0 2024-03-07T06:53:03.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 -- 172.21.15.133:0/2947981123 >> v2:172.21.15.161:3300/0 conn(0x7f71b0072950 msgr2=0x7f71b00cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:03.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 --2- 172.21.15.133:0/2947981123 >> v2:172.21.15.161:3300/0 conn(0x7f71b0072950 0x7f71b00cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:03.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 -- 172.21.15.133:0/2947981123 >> v1:172.21.15.161:6789/0 conn(0x7f71b015fbb0 legacy=0x7f71b015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:03.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 -- 172.21.15.133:0/2947981123 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b0072430 con 0x7f71b00cdee0 2024-03-07T06:53:03.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 -- 172.21.15.133:0/2947981123 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f71a80019f0 con 0x7f71b00cdee0 2024-03-07T06:53:03.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 -- 172.21.15.133:0/2947981123 >> v1:172.21.15.133:6789/0 conn(0x7f71b00cdee0 legacy=0x7f71b015c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:03.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 --2- 172.21.15.133:0/2947981123 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71980039b0 0x7f7198003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:03.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 --2- 172.21.15.133:0/2947981123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7198004800 0x7f7198004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:03.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 --2- 172.21.15.133:0/2947981123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7198005210 0x7f719800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:03.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 -- 172.21.15.133:0/2947981123 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0072f60 con 0x7f7198004800 2024-03-07T06:53:03.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 -- 172.21.15.133:0/2947981123 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719800c140 con 0x7f7198005210 2024-03-07T06:53:03.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.300+0000 7f71af400640 1 -- 172.21.15.133:0/2947981123 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719800c340 con 0x7f71980039b0 2024-03-07T06:53:03.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.301+0000 7f71afe00640 1 --2- 172.21.15.133:0/2947981123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7198004800 0x7f7198004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.301+0000 7f71b5200640 1 --2- 172.21.15.133:0/2947981123 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71980039b0 0x7f7198003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.301+0000 7f71b5c00640 1 --2- 172.21.15.133:0/2947981123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7198005210 0x7f719800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.301+0000 7f71afe00640 1 -- 172.21.15.133:0/2947981123 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71980039b0 msgr2=0x7f7198003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.301+0000 7f71afe00640 1 --2- 172.21.15.133:0/2947981123 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71980039b0 0x7f7198003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.301+0000 7f71afe00640 1 -- 172.21.15.133:0/2947981123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7198005210 msgr2=0x7f719800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.301+0000 7f71afe00640 1 --2- 172.21.15.133:0/2947981123 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7198005210 0x7f719800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.301+0000 7f71afe00640 1 -- 172.21.15.133:0/2947981123 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7198003520 con 0x7f7198004800 2024-03-07T06:53:03.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.301+0000 7f71b5c00640 1 -- 172.21.15.133:0/2947981123 reap_dead start 2024-03-07T06:53:03.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.302+0000 7f71afe00640 1 --2- 172.21.15.133:0/2947981123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7198004800 0x7f7198004c80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f719c009a90 tx=0x7f7198005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a8f2270ec2553a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:03.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.302+0000 7f71af400640 1 -- 172.21.15.133:0/2947981123 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c039070 con 0x7f7198004800 2024-03-07T06:53:03.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.302+0000 7f71af400640 1 -- 172.21.15.133:0/2947981123 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c02c680 con 0x7f7198004800 2024-03-07T06:53:03.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.302+0000 7f71af400640 1 -- 172.21.15.133:0/2947981123 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c009690 con 0x7f7198004800 2024-03-07T06:53:03.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.303+0000 7f71b7c00640 1 -- 172.21.15.133:0/2947981123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7198004800 msgr2=0x7f7198004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:03.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.303+0000 7f71b7c00640 1 --2- 172.21.15.133:0/2947981123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7198004800 0x7f7198004c80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f719c009a90 tx=0x7f7198005830 comp rx=0 tx=0).stop 2024-03-07T06:53:03.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.303+0000 7f71b7c00640 1 -- 172.21.15.133:0/2947981123 shutdown_connections 2024-03-07T06:53:03.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.303+0000 7f71b7c00640 1 --2- 172.21.15.133:0/2947981123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7198004800 0x7f7198004c80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:03.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.303+0000 7f71b7c00640 1 -- 172.21.15.133:0/2947981123 >> 172.21.15.133:0/2947981123 conn(0x7f71b00b3120 msgr2=0x7f71b00b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:03.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.303+0000 7f71b7c00640 1 -- 172.21.15.133:0/2947981123 shutdown_connections 2024-03-07T06:53:03.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.303+0000 7f71b7c00640 1 -- 172.21.15.133:0/2947981123 wait complete. 2024-03-07T06:53:03.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.305+0000 7f71b7c00640 1 Processor -- start 2024-03-07T06:53:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.305+0000 7f71b7c00640 1 -- start start 2024-03-07T06:53:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.306+0000 7f71b7c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7198004800 0x7f71b00b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.306+0000 7f71b7c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71b0149320 0x7f71b0149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.306+0000 7f71b7c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71b015dab0 0x7f71b015df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.306+0000 7f71b7c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0160980 con 0x7f71b015dab0 2024-03-07T06:53:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.306+0000 7f71b7c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0073490 con 0x7f7198004800 2024-03-07T06:53:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.306+0000 7f71b7c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00b1a00 con 0x7f71b0149320 2024-03-07T06:53:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.306+0000 7f71b5c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71b015dab0 0x7f71b015df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.306+0000 7f71b5c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71b015dab0 0x7f71b015df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43880/0 (socket says 172.21.15.133:43880) 2024-03-07T06:53:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.306+0000 7f71b5c00640 1 -- 172.21.15.133:0/955620747 learned_addr learned my addr 172.21.15.133:0/955620747 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.307+0000 7f71b5200640 1 --2- 172.21.15.133:0/955620747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7198004800 0x7f71b00b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.307+0000 7f71afe00640 1 --2- 172.21.15.133:0/955620747 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71b0149320 0x7f71b0149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.307+0000 7f71b5c00640 1 -- 172.21.15.133:0/955620747 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71b0149320 msgr2=0x7f71b0149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:03.309 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.307+0000 7f71b5c00640 1 --2- 172.21.15.133:0/955620747 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71b0149320 0x7f71b0149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:03.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.307+0000 7f71b5c00640 1 -- 172.21.15.133:0/955620747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7198004800 msgr2=0x7f71b00b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:03.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.307+0000 7f71b5c00640 1 --2- 172.21.15.133:0/955620747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7198004800 0x7f71b00b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:03.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.307+0000 7f71b5c00640 1 -- 172.21.15.133:0/955620747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b015c3c0 con 0x7f71b015dab0 2024-03-07T06:53:03.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.307+0000 7f71b5c00640 1 --2- 172.21.15.133:0/955620747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71b015dab0 0x7f71b015df30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f71b0071560 tx=0x7f71a8007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:03.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.307+0000 7f71acc00640 1 -- 172.21.15.133:0/955620747 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a800e9e0 con 0x7f71b015dab0 2024-03-07T06:53:03.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.307+0000 7f71acc00640 1 -- 172.21.15.133:0/955620747 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a8018070 con 0x7f71b015dab0 2024-03-07T06:53:03.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.307+0000 7f71acc00640 1 -- 172.21.15.133:0/955620747 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a800edc0 con 0x7f71b015dab0 2024-03-07T06:53:03.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.307+0000 7f71b7c00640 1 -- 172.21.15.133:0/955620747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b015c6a0 con 0x7f71b015dab0 2024-03-07T06:53:03.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.308+0000 7f71b7c00640 1 -- 172.21.15.133:0/955620747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b0149cc0 con 0x7f71b015dab0 2024-03-07T06:53:03.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.309+0000 7f71b7c00640 1 -- 172.21.15.133:0/955620747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b0062c90 con 0x7f71b015dab0 2024-03-07T06:53:03.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.310+0000 7f71acc00640 1 -- 172.21.15.133:0/955620747 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f71a801d460 con 0x7f71b015dab0 2024-03-07T06:53:03.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.310+0000 7f71acc00640 1 --2- 172.21.15.133:0/955620747 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f719006b500 0x7f719006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:03.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.311+0000 7f71acc00640 1 -- 172.21.15.133:0/955620747 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71a808a4f0 con 0x7f71b015dab0 2024-03-07T06:53:03.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.311+0000 7f71b5200640 1 --2- 172.21.15.133:0/955620747 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f719006b500 0x7f719006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:03.312 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.312+0000 7f71b5200640 1 --2- 172.21.15.133:0/955620747 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f719006b500 0x7f719006d9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71a4007fd0 tx=0x7f71a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:03.314 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.315+0000 7f71acc00640 1 -- 172.21.15.133:0/955620747 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f71a8059240 con 0x7f71b015dab0 2024-03-07T06:53:03.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.587+0000 7f71b7c00640 1 -- 172.21.15.133:0/955620747 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f71b015bf00 con 0x7f71b015dab0 2024-03-07T06:53:03.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.587+0000 7f71acc00640 1 -- 172.21.15.133:0/955620747 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1658 (secure 0 0 0) 0x7f71a8058b40 con 0x7f71b015dab0 2024-03-07T06:53:03.586 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:03.586 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:53:03.586 INFO:teuthology.orchestra.run.smithi133.stdout: "stats": { 2024-03-07T06:53:03.586 INFO:teuthology.orchestra.run.smithi133.stdout: "total_bytes": 773094113280, 2024-03-07T06:53:03.586 INFO:teuthology.orchestra.run.smithi133.stdout: "total_avail_bytes": 772836253696, 2024-03-07T06:53:03.586 INFO:teuthology.orchestra.run.smithi133.stdout: "total_used_bytes": 257859584, 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: "total_used_raw_bytes": 257859584, 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: "total_used_raw_ratio": 0.00033354229526594281, 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: "num_osds": 8, 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: "num_per_pool_osds": 8, 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: "num_per_pool_omap_osds": 8 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: "stats_by_class": { 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: "ssd": { 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: "total_bytes": 773094113280, 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: "total_avail_bytes": 772836253696, 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: "total_used_bytes": 257859584, 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: "total_used_raw_bytes": 257859584, 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: "total_used_raw_ratio": 0.00033354229526594281 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:03.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:03.588 INFO:teuthology.orchestra.run.smithi133.stdout: "pools": [ 2024-03-07T06:53:03.588 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:03.588 INFO:teuthology.orchestra.run.smithi133.stdout: "name": ".mgr", 2024-03-07T06:53:03.588 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 1, 2024-03-07T06:53:03.588 INFO:teuthology.orchestra.run.smithi133.stdout: "stats": { 2024-03-07T06:53:03.588 INFO:teuthology.orchestra.run.smithi133.stdout: "stored": 590368, 2024-03-07T06:53:03.588 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": 2, 2024-03-07T06:53:03.588 INFO:teuthology.orchestra.run.smithi133.stdout: "kb_used": 1160, 2024-03-07T06:53:03.588 INFO:teuthology.orchestra.run.smithi133.stdout: "bytes_used": 1187840, 2024-03-07T06:53:03.588 INFO:teuthology.orchestra.run.smithi133.stdout: "percent_used": 1.6180032389456755e-06, 2024-03-07T06:53:03.588 INFO:teuthology.orchestra.run.smithi133.stdout: "max_avail": 367069134848 2024-03-07T06:53:03.588 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:03.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:03.588 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:03.589 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephfs_metadata", 2024-03-07T06:53:03.589 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 4, 2024-03-07T06:53:03.589 INFO:teuthology.orchestra.run.smithi133.stdout: "stats": { 2024-03-07T06:53:03.589 INFO:teuthology.orchestra.run.smithi133.stdout: "stored": 1214476, 2024-03-07T06:53:03.589 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": 23, 2024-03-07T06:53:03.589 INFO:teuthology.orchestra.run.smithi133.stdout: "kb_used": 2432, 2024-03-07T06:53:03.589 INFO:teuthology.orchestra.run.smithi133.stdout: "bytes_used": 2490368, 2024-03-07T06:53:03.589 INFO:teuthology.orchestra.run.smithi133.stdout: "percent_used": 3.3922215152415447e-06, 2024-03-07T06:53:03.589 INFO:teuthology.orchestra.run.smithi133.stdout: "max_avail": 367069134848 2024-03-07T06:53:03.589 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:03.589 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:03.589 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:03.589 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephfs_data", 2024-03-07T06:53:03.589 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 5, 2024-03-07T06:53:03.590 INFO:teuthology.orchestra.run.smithi133.stdout: "stats": { 2024-03-07T06:53:03.590 INFO:teuthology.orchestra.run.smithi133.stdout: "stored": 0, 2024-03-07T06:53:03.590 INFO:teuthology.orchestra.run.smithi133.stdout: "objects": 0, 2024-03-07T06:53:03.590 INFO:teuthology.orchestra.run.smithi133.stdout: "kb_used": 0, 2024-03-07T06:53:03.590 INFO:teuthology.orchestra.run.smithi133.stdout: "bytes_used": 0, 2024-03-07T06:53:03.590 INFO:teuthology.orchestra.run.smithi133.stdout: "percent_used": 0, 2024-03-07T06:53:03.590 INFO:teuthology.orchestra.run.smithi133.stdout: "max_avail": 367069134848 2024-03-07T06:53:03.590 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:03.590 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:03.590 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-03-07T06:53:03.590 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:53:03.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.590+0000 7f71b7c00640 1 -- 172.21.15.133:0/955620747 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f719006b500 msgr2=0x7f719006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:03.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.590+0000 7f71b7c00640 1 --2- 172.21.15.133:0/955620747 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f719006b500 0x7f719006d9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71a4007fd0 tx=0x7f71a4009040 comp rx=0 tx=0).stop 2024-03-07T06:53:03.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.590+0000 7f71b7c00640 1 -- 172.21.15.133:0/955620747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71b015dab0 msgr2=0x7f71b015df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:03.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.590+0000 7f71b7c00640 1 --2- 172.21.15.133:0/955620747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71b015dab0 0x7f71b015df30 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f71b0071560 tx=0x7f71a8007a80 comp rx=0 tx=0).stop 2024-03-07T06:53:03.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.590+0000 7f71b7c00640 1 -- 172.21.15.133:0/955620747 shutdown_connections 2024-03-07T06:53:03.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.590+0000 7f71b7c00640 1 --2- 172.21.15.133:0/955620747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71b015dab0 0x7f71b015df30 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:03.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.590+0000 7f71b7c00640 1 --2- 172.21.15.133:0/955620747 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71b0149320 0x7f71b0149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:03.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.590+0000 7f71b7c00640 1 --2- 172.21.15.133:0/955620747 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f719006b500 0x7f719006d9c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:03.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.590+0000 7f71b7c00640 1 --2- 172.21.15.133:0/955620747 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7198004800 0x7f71b00b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:03.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.590+0000 7f71b7c00640 1 -- 172.21.15.133:0/955620747 >> 172.21.15.133:0/955620747 conn(0x7f71b00b3120 msgr2=0x7f71b00b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:03.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.590+0000 7f71b7c00640 1 -- 172.21.15.133:0/955620747 shutdown_connections 2024-03-07T06:53:03.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:03.590+0000 7f71b7c00640 1 -- 172.21.15.133:0/955620747 wait complete. 2024-03-07T06:53:03.608 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-03-07T06:53:03.608 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-07T06:53:03.609 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-07T06:53:03.723 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-07T06:53:03.842 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-07T06:53:03.842 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:53:03.842 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:03.895 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:53:03.896 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:03.945 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:53:04.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.185+0000 7fcfb1a00640 1 Processor -- start 2024-03-07T06:53:04.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.185+0000 7fcfb1a00640 1 -- start start 2024-03-07T06:53:04.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.187+0000 7fcfb1a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcfac071410 0x7fcfac071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:04.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.187+0000 7fcfb1a00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac162c80 con 0x7fcfac144540 2024-03-07T06:53:04.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.187+0000 7fcfb1a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac0b4bd0 con 0x7fcfac0b5dd0 2024-03-07T06:53:04.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.187+0000 7fcfb1a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfac141700 con 0x7fcfac071410 2024-03-07T06:53:04.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.188+0000 7fcfa3e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fcfac144540 0x7fcfac162570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50144/0 (socket says 172.21.15.133:50144) 2024-03-07T06:53:04.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.188+0000 7fcfa3e00640 1 -- 172.21.15.133:0/1703593871 learned_addr learned my addr 172.21.15.133:0/1703593871 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:04.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.188+0000 7fcfab400640 1 --2- 172.21.15.133:0/1703593871 >> v2:172.21.15.161:3300/0 conn(0x7fcfac071410 0x7fcfac071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:04.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.188+0000 7fcfaa000640 1 -- 172.21.15.133:0/1703593871 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2397863194 0 0) 0x7fcfac162c80 con 0x7fcfac144540 2024-03-07T06:53:04.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.188+0000 7fcfaa000640 1 -- 172.21.15.133:0/1703593871 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf90003520 con 0x7fcfac144540 2024-03-07T06:53:04.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfaa000640 1 -- 172.21.15.133:0/1703593871 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3131055010 0 0) 0x7fcf90003520 con 0x7fcfac144540 2024-03-07T06:53:04.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfaa000640 1 -- 172.21.15.133:0/1703593871 >> v2:172.21.15.161:3300/0 conn(0x7fcfac071410 msgr2=0x7fcfac071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:04.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfaa000640 1 --2- 172.21.15.133:0/1703593871 >> v2:172.21.15.161:3300/0 conn(0x7fcfac071410 0x7fcfac071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfaa000640 1 -- 172.21.15.133:0/1703593871 >> v1:172.21.15.161:6789/0 conn(0x7fcfac0b5dd0 legacy=0x7fcfac0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:04.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfaa000640 1 -- 172.21.15.133:0/1703593871 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac0b4eb0 con 0x7fcfac144540 2024-03-07T06:53:04.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfaa000640 1 -- 172.21.15.133:0/1703593871 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fcf94001670 con 0x7fcfac144540 2024-03-07T06:53:04.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfaa000640 1 -- 172.21.15.133:0/1703593871 >> v1:172.21.15.133:6789/0 conn(0x7fcfac144540 legacy=0x7fcfac162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:04.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfaa000640 1 --2- 172.21.15.133:0/1703593871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf900039b0 0x7fcf90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:04.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfaa000640 1 --2- 172.21.15.133:0/1703593871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf90004800 0x7fcf90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:04.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfaa000640 1 --2- 172.21.15.133:0/1703593871 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf900051c0 0x7fcf9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:04.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfaa000640 1 -- 172.21.15.133:0/1703593871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac141700 con 0x7fcf90004800 2024-03-07T06:53:04.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfaa000640 1 -- 172.21.15.133:0/1703593871 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9000c160 con 0x7fcf900039b0 2024-03-07T06:53:04.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfaa000640 1 -- 172.21.15.133:0/1703593871 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9000c360 con 0x7fcf900051c0 2024-03-07T06:53:04.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfa3e00640 1 --2- 172.21.15.133:0/1703593871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf90004800 0x7fcf90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:04.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfaaa00640 1 --2- 172.21.15.133:0/1703593871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf900039b0 0x7fcf90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:04.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfa3e00640 1 -- 172.21.15.133:0/1703593871 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf900051c0 msgr2=0x7fcf9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:04.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfa3e00640 1 --2- 172.21.15.133:0/1703593871 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf900051c0 0x7fcf9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfa3e00640 1 -- 172.21.15.133:0/1703593871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf900039b0 msgr2=0x7fcf90003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:04.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfa3e00640 1 --2- 172.21.15.133:0/1703593871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf900039b0 0x7fcf90003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfab400640 1 -- 172.21.15.133:0/1703593871 reap_dead start 2024-03-07T06:53:04.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.189+0000 7fcfa3e00640 1 -- 172.21.15.133:0/1703593871 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf90003520 con 0x7fcf90004800 2024-03-07T06:53:04.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.191+0000 7fcfa3e00640 1 --2- 172.21.15.133:0/1703593871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf90004800 0x7fcf90004c80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400c240 tx=0x7fcf90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc08a072049c2924 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:04.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.191+0000 7fcfaa000640 1 -- 172.21.15.133:0/1703593871 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf94007cf0 con 0x7fcf90004800 2024-03-07T06:53:04.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.191+0000 7fcfaa000640 1 -- 172.21.15.133:0/1703593871 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf94030070 con 0x7fcf90004800 2024-03-07T06:53:04.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.191+0000 7fcfaa000640 1 -- 172.21.15.133:0/1703593871 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf94005d70 con 0x7fcf90004800 2024-03-07T06:53:04.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.191+0000 7fcfb1a00640 1 -- 172.21.15.133:0/1703593871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf90004800 msgr2=0x7fcf90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:04.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.191+0000 7fcfb1a00640 1 --2- 172.21.15.133:0/1703593871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf90004800 0x7fcf90004c80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400c240 tx=0x7fcf90005830 comp rx=0 tx=0).stop 2024-03-07T06:53:04.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.191+0000 7fcfb1a00640 1 -- 172.21.15.133:0/1703593871 shutdown_connections 2024-03-07T06:53:04.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.191+0000 7fcfb1a00640 1 --2- 172.21.15.133:0/1703593871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf90004800 0x7fcf90004c80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.191+0000 7fcfb1a00640 1 -- 172.21.15.133:0/1703593871 >> 172.21.15.133:0/1703593871 conn(0x7fcfac0d0e70 msgr2=0x7fcfac0d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:04.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.192+0000 7fcfb1a00640 1 -- 172.21.15.133:0/1703593871 shutdown_connections 2024-03-07T06:53:04.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.192+0000 7fcfb1a00640 1 -- 172.21.15.133:0/1703593871 wait complete. 2024-03-07T06:53:04.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.193+0000 7fcfb1a00640 1 Processor -- start 2024-03-07T06:53:04.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.193+0000 7fcfb1a00640 1 -- start start 2024-03-07T06:53:04.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.194+0000 7fcfb1a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf90004800 0x7fcfac14e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:04.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.194+0000 7fcfb1a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfac0d2c20 0x7fcfac0d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:04.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.194+0000 7fcfb1a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfac1596c0 0x7fcfac159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:04.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.194+0000 7fcfb1a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac161e00 con 0x7fcf90004800 2024-03-07T06:53:04.194 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.194+0000 7fcfb1a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac071dc0 con 0x7fcfac0d2c20 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.194+0000 7fcfb1a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac162050 con 0x7fcfac1596c0 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.195+0000 7fcfaaa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf90004800 0x7fcfac14e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.195+0000 7fcfaaa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf90004800 0x7fcfac14e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43894/0 (socket says 172.21.15.133:43894) 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.195+0000 7fcfaaa00640 1 -- 172.21.15.133:0/446600068 learned_addr learned my addr 172.21.15.133:0/446600068 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.195+0000 7fcfaaa00640 1 -- 172.21.15.133:0/446600068 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfac1596c0 msgr2=0x7fcfac159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.195+0000 7fcfaaa00640 1 --2- 172.21.15.133:0/446600068 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfac1596c0 0x7fcfac159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.195+0000 7fcfa3e00640 1 --2- 172.21.15.133:0/446600068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfac0d2c20 0x7fcfac0d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.195+0000 7fcfaaa00640 1 -- 172.21.15.133:0/446600068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfac0d2c20 msgr2=0x7fcfac0d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.195+0000 7fcfaaa00640 1 --2- 172.21.15.133:0/446600068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfac0d2c20 0x7fcfac0d30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.195+0000 7fcfaaa00640 1 -- 172.21.15.133:0/446600068 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac0d36d0 con 0x7fcf90004800 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.195+0000 7fcfaaa00640 1 --2- 172.21.15.133:0/446600068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf90004800 0x7fcfac14e0a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00d400 tx=0x7fcf9c00d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.196+0000 7fcfa2a00640 1 -- 172.21.15.133:0/446600068 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c018070 con 0x7fcf90004800 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.196+0000 7fcfa2a00640 1 -- 172.21.15.133:0/446600068 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c003610 con 0x7fcf90004800 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.196+0000 7fcfa2a00640 1 -- 172.21.15.133:0/446600068 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c003a00 con 0x7fcf90004800 2024-03-07T06:53:04.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.196+0000 7fcfb1a00640 1 -- 172.21.15.133:0/446600068 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfac0d39b0 con 0x7fcf90004800 2024-03-07T06:53:04.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.196+0000 7fcfb1a00640 1 -- 172.21.15.133:0/446600068 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfac141e90 con 0x7fcf90004800 2024-03-07T06:53:04.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.197+0000 7fcfb1a00640 1 -- 172.21.15.133:0/446600068 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfac062c90 con 0x7fcf90004800 2024-03-07T06:53:04.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.198+0000 7fcfa2a00640 1 -- 172.21.15.133:0/446600068 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fcf9c01b070 con 0x7fcf90004800 2024-03-07T06:53:04.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.198+0000 7fcfa2a00640 1 --2- 172.21.15.133:0/446600068 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fcf8806b500 0x7fcf8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:04.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.199+0000 7fcfa2a00640 1 -- 172.21.15.133:0/446600068 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf9c089970 con 0x7fcf90004800 2024-03-07T06:53:04.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.199+0000 7fcfa3e00640 1 --2- 172.21.15.133:0/446600068 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fcf8806b500 0x7fcf8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:04.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.201+0000 7fcfa3e00640 1 --2- 172.21.15.133:0/446600068 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fcf8806b500 0x7fcf8806d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcf94034000 tx=0x7fcf94039040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:04.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.203+0000 7fcfa2a00640 1 -- 172.21.15.133:0/446600068 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fcf9c0581c0 con 0x7fcf90004800 2024-03-07T06:53:04.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.399+0000 7fcfb1a00640 1 -- 172.21.15.133:0/446600068 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfac161210 con 0x7fcf90004800 2024-03-07T06:53:04.399 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.401+0000 7fcfa2a00640 1 -- 172.21.15.133:0/446600068 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16717 (secure 0 0 0) 0x7fcf9c05c260 con 0x7fcf90004800 2024-03-07T06:53:04.400 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:04.400 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":35,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:52:36.759644+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:52:33.007428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-07T06:52:34.201590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:53:04.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.403+0000 7fcfb1a00640 1 -- 172.21.15.133:0/446600068 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fcf8806b500 msgr2=0x7fcf8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:04.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.403+0000 7fcfb1a00640 1 --2- 172.21.15.133:0/446600068 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fcf8806b500 0x7fcf8806d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcf94034000 tx=0x7fcf94039040 comp rx=0 tx=0).stop 2024-03-07T06:53:04.402 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.403+0000 7fcfb1a00640 1 -- 172.21.15.133:0/446600068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf90004800 msgr2=0x7fcfac14e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:04.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.403+0000 7fcfb1a00640 1 --2- 172.21.15.133:0/446600068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf90004800 0x7fcfac14e0a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00d400 tx=0x7fcf9c00d8c0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.403+0000 7fcfb1a00640 1 -- 172.21.15.133:0/446600068 shutdown_connections 2024-03-07T06:53:04.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.403+0000 7fcfb1a00640 1 --2- 172.21.15.133:0/446600068 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcfac1596c0 0x7fcfac159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.403+0000 7fcfb1a00640 1 --2- 172.21.15.133:0/446600068 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fcf8806b500 0x7fcf8806d9c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.403+0000 7fcfb1a00640 1 --2- 172.21.15.133:0/446600068 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcfac0d2c20 0x7fcfac0d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.403+0000 7fcfb1a00640 1 --2- 172.21.15.133:0/446600068 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf90004800 0x7fcfac14e0a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.403+0000 7fcfb1a00640 1 -- 172.21.15.133:0/446600068 >> 172.21.15.133:0/446600068 conn(0x7fcfac0d0e70 msgr2=0x7fcfac15da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:04.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.404+0000 7fcfb1a00640 1 -- 172.21.15.133:0/446600068 shutdown_connections 2024-03-07T06:53:04.403 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.404+0000 7fcfb1a00640 1 -- 172.21.15.133:0/446600068 wait complete. 2024-03-07T06:53:04.420 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:52:33.007428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:52:34.201590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:53:04.423 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:53:04.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.635+0000 7f0e4e400640 1 Processor -- start 2024-03-07T06:53:04.634 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.635+0000 7f0e4e400640 1 -- start start 2024-03-07T06:53:04.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.637+0000 7f0e4e400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0e4815e350 0x7f0e48161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:04.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.637+0000 7f0e4e400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e480b7200 0x7f0e481623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:04.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.637+0000 7f0e4e400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e48071ff0 con 0x7f0e48071280 2024-03-07T06:53:04.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.637+0000 7f0e4e400640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e480d2160 con 0x7f0e4815e350 2024-03-07T06:53:04.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.637+0000 7f0e4e400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480d23b0 con 0x7f0e480b7200 2024-03-07T06:53:04.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.637+0000 7f0e47400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0e4815e350 0x7f0e48161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:04.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.637+0000 7f0e47400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0e4815e350 0x7f0e48161eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43898/0 (socket says 172.21.15.133:43898) 2024-03-07T06:53:04.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.637+0000 7f0e47400640 1 -- 172.21.15.133:0/2706824771 learned_addr learned my addr 172.21.15.133:0/2706824771 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:04.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.637+0000 7f0e3fe00640 1 --2- 172.21.15.133:0/2706824771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e480b7200 0x7f0e481623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:04.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.638+0000 7f0e47400640 1 -- 172.21.15.133:0/2706824771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e480b7200 msgr2=0x7f0e481623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:04.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.638+0000 7f0e47400640 1 --2- 172.21.15.133:0/2706824771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e480b7200 0x7f0e481623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.638+0000 7f0e47400640 1 -- 172.21.15.133:0/2706824771 >> v1:172.21.15.161:6789/0 conn(0x7f0e48071280 legacy=0x7f0e48071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:04.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.638+0000 7f0e47400640 1 -- 172.21.15.133:0/2706824771 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e480d2690 con 0x7f0e4815e350 2024-03-07T06:53:04.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.639+0000 7f0e47400640 1 --2- 172.21.15.133:0/2706824771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4815e350 0x7f0e48161eb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009870 tx=0x7f0e3802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53ced4291096a69 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:04.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.639+0000 7f0e46a00640 1 -- 172.21.15.133:0/2706824771 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38035020 con 0x7f0e4815e350 2024-03-07T06:53:04.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.639+0000 7f0e46a00640 1 -- 172.21.15.133:0/2706824771 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e38031640 con 0x7f0e4815e350 2024-03-07T06:53:04.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.639+0000 7f0e46a00640 1 -- 172.21.15.133:0/2706824771 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3802d040 con 0x7f0e4815e350 2024-03-07T06:53:04.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.640+0000 7f0e4e400640 1 -- 172.21.15.133:0/2706824771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4815e350 msgr2=0x7f0e48161eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:04.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.640+0000 7f0e4e400640 1 --2- 172.21.15.133:0/2706824771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4815e350 0x7f0e48161eb0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0e38009870 tx=0x7f0e3802b8f0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.640+0000 7f0e4e400640 1 -- 172.21.15.133:0/2706824771 shutdown_connections 2024-03-07T06:53:04.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.640+0000 7f0e4e400640 1 --2- 172.21.15.133:0/2706824771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e480b7200 0x7f0e481623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.640+0000 7f0e4e400640 1 --2- 172.21.15.133:0/2706824771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e4815e350 0x7f0e48161eb0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.640+0000 7f0e4e400640 1 -- 172.21.15.133:0/2706824771 >> 172.21.15.133:0/2706824771 conn(0x7f0e480d0e70 msgr2=0x7f0e480d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:04.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.640+0000 7f0e4e400640 1 -- 172.21.15.133:0/2706824771 shutdown_connections 2024-03-07T06:53:04.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.640+0000 7f0e4e400640 1 -- 172.21.15.133:0/2706824771 wait complete. 2024-03-07T06:53:04.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.642+0000 7f0e4e400640 1 Processor -- start 2024-03-07T06:53:04.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.642+0000 7f0e4e400640 1 -- start start 2024-03-07T06:53:04.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.643+0000 7f0e4e400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e48071280 0x7f0e481486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:04.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.643+0000 7f0e4e400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e480b7200 0x7f0e4814aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:04.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.643+0000 7f0e4e400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e4815e350 0x7f0e4815c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:04.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.643+0000 7f0e4e400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480d2e90 con 0x7f0e480b7200 2024-03-07T06:53:04.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.643+0000 7f0e4e400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4815e730 con 0x7f0e48071280 2024-03-07T06:53:04.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.643+0000 7f0e4e400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480d3010 con 0x7f0e4815e350 2024-03-07T06:53:04.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.643+0000 7f0e3fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e480b7200 0x7f0e4814aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:04.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.643+0000 7f0e3fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e480b7200 0x7f0e4814aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43904/0 (socket says 172.21.15.133:43904) 2024-03-07T06:53:04.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.643+0000 7f0e3fe00640 1 -- 172.21.15.133:0/3566753038 learned_addr learned my addr 172.21.15.133:0/3566753038 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:04.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.643+0000 7f0e47e00640 1 --2- 172.21.15.133:0/3566753038 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e4815e350 0x7f0e4815c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:04.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.643+0000 7f0e47400640 1 --2- 172.21.15.133:0/3566753038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e48071280 0x7f0e481486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:04.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.644+0000 7f0e3fe00640 1 -- 172.21.15.133:0/3566753038 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e4815e350 msgr2=0x7f0e4815c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:04.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.644+0000 7f0e3fe00640 1 --2- 172.21.15.133:0/3566753038 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e4815e350 0x7f0e4815c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.644+0000 7f0e3fe00640 1 -- 172.21.15.133:0/3566753038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e48071280 msgr2=0x7f0e481486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:04.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.644+0000 7f0e3fe00640 1 --2- 172.21.15.133:0/3566753038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e48071280 0x7f0e481486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.644+0000 7f0e3fe00640 1 -- 172.21.15.133:0/3566753038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4814bab0 con 0x7f0e480b7200 2024-03-07T06:53:04.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.644+0000 7f0e3fe00640 1 --2- 172.21.15.133:0/3566753038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e480b7200 0x7f0e4814aff0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000d830 tx=0x7f0e3000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:04.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.644+0000 7f0e3f400640 1 -- 172.21.15.133:0/3566753038 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3000b660 con 0x7f0e480b7200 2024-03-07T06:53:04.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.644+0000 7f0e3f400640 1 -- 172.21.15.133:0/3566753038 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e30019070 con 0x7f0e480b7200 2024-03-07T06:53:04.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.644+0000 7f0e3f400640 1 -- 172.21.15.133:0/3566753038 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3000ba20 con 0x7f0e480b7200 2024-03-07T06:53:04.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.644+0000 7f0e4e400640 1 -- 172.21.15.133:0/3566753038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4814bd90 con 0x7f0e480b7200 2024-03-07T06:53:04.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.645+0000 7f0e4e400640 1 -- 172.21.15.133:0/3566753038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e48149450 con 0x7f0e480b7200 2024-03-07T06:53:04.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.646+0000 7f0e4e400640 1 -- 172.21.15.133:0/3566753038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e48062c90 con 0x7f0e480b7200 2024-03-07T06:53:04.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.647+0000 7f0e3f400640 1 -- 172.21.15.133:0/3566753038 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0e3001e460 con 0x7f0e480b7200 2024-03-07T06:53:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.648+0000 7f0e3f400640 1 --2- 172.21.15.133:0/3566753038 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0e2406b4d0 0x7f0e2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:04.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.648+0000 7f0e3f400640 1 -- 172.21.15.133:0/3566753038 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e3008a8f0 con 0x7f0e480b7200 2024-03-07T06:53:04.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.648+0000 7f0e47400640 1 --2- 172.21.15.133:0/3566753038 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0e2406b4d0 0x7f0e2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:04.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.649+0000 7f0e47400640 1 --2- 172.21.15.133:0/3566753038 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0e2406b4d0 0x7f0e2406d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0e38036000 tx=0x7f0e3803c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:04.651 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.652+0000 7f0e3f400640 1 -- 172.21.15.133:0/3566753038 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0e300596d0 con 0x7f0e480b7200 2024-03-07T06:53:04.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.858+0000 7f0e4e400640 1 -- 172.21.15.133:0/3566753038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e4814bf60 con 0x7f0e480b7200 2024-03-07T06:53:04.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.859+0000 7f0e3f400640 1 -- 172.21.15.133:0/3566753038 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e30058fd0 con 0x7f0e480b7200 2024-03-07T06:53:04.858 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:53:04.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.861+0000 7f0e4e400640 1 -- 172.21.15.133:0/3566753038 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0e2406b4d0 msgr2=0x7f0e2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:04.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.861+0000 7f0e4e400640 1 --2- 172.21.15.133:0/3566753038 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0e2406b4d0 0x7f0e2406d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0e38036000 tx=0x7f0e3803c040 comp rx=0 tx=0).stop 2024-03-07T06:53:04.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.861+0000 7f0e4e400640 1 -- 172.21.15.133:0/3566753038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e480b7200 msgr2=0x7f0e4814aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:04.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.861+0000 7f0e4e400640 1 --2- 172.21.15.133:0/3566753038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e480b7200 0x7f0e4814aff0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000d830 tx=0x7f0e3000dcf0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.861+0000 7f0e4e400640 1 -- 172.21.15.133:0/3566753038 shutdown_connections 2024-03-07T06:53:04.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.861+0000 7f0e4e400640 1 --2- 172.21.15.133:0/3566753038 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0e4815e350 0x7f0e4815c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.861+0000 7f0e4e400640 1 --2- 172.21.15.133:0/3566753038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e480b7200 0x7f0e4814aff0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.861+0000 7f0e4e400640 1 --2- 172.21.15.133:0/3566753038 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0e2406b4d0 0x7f0e2406d990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.861+0000 7f0e4e400640 1 --2- 172.21.15.133:0/3566753038 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0e48071280 0x7f0e481486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:04.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.861+0000 7f0e4e400640 1 -- 172.21.15.133:0/3566753038 >> 172.21.15.133:0/3566753038 conn(0x7f0e480d0e70 msgr2=0x7f0e480b8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:04.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.862+0000 7f0e4e400640 1 -- 172.21.15.133:0/3566753038 shutdown_connections 2024-03-07T06:53:04.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:04.862+0000 7f0e4e400640 1 -- 172.21.15.133:0/3566753038 wait complete. 2024-03-07T06:53:04.884 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:53:05.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.091+0000 7f6fe8e00640 1 Processor -- start 2024-03-07T06:53:05.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.091+0000 7f6fe8e00640 1 -- start start 2024-03-07T06:53:05.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.093+0000 7f6fe8e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6fe415e480 0x7f6fe40b99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.093+0000 7f6fe8e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6fe4071740 0x7f6fe40cf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.093+0000 7f6fe8e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe415e860 con 0x7f6fe40ced50 2024-03-07T06:53:05.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.093+0000 7f6fe8e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe40ce860 con 0x7f6fe4071740 2024-03-07T06:53:05.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.093+0000 7f6fe8e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe40d3220 con 0x7f6fe415e480 2024-03-07T06:53:05.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.093+0000 7f6fe1400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6fe40ced50 0x7f6fe4161fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50154/0 (socket says 172.21.15.133:50154) 2024-03-07T06:53:05.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.093+0000 7f6fe2800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6fe4071740 0x7f6fe40cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.093+0000 7f6fe1400640 1 -- 172.21.15.133:0/3071802578 learned_addr learned my addr 172.21.15.133:0/3071802578 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:05.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.094+0000 7f6fe1e00640 1 --2- 172.21.15.133:0/3071802578 >> v2:172.21.15.161:3300/0 conn(0x7f6fe415e480 0x7f6fe40b99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.094+0000 7f6fe2800640 1 -- 172.21.15.133:0/3071802578 >> v2:172.21.15.161:3300/0 conn(0x7f6fe415e480 msgr2=0x7f6fe40b99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:05.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.094+0000 7f6fe2800640 1 --2- 172.21.15.133:0/3071802578 >> v2:172.21.15.161:3300/0 conn(0x7f6fe415e480 0x7f6fe40b99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.094+0000 7f6fe2800640 1 -- 172.21.15.133:0/3071802578 >> v1:172.21.15.133:6789/0 conn(0x7f6fe40ced50 legacy=0x7f6fe4161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.094+0000 7f6fe2800640 1 -- 172.21.15.133:0/3071802578 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe40d3500 con 0x7f6fe4071740 2024-03-07T06:53:05.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.095+0000 7f6fe2800640 1 --2- 172.21.15.133:0/3071802578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fe4071740 0x7f6fe40cf200 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6fe408d1a0 tx=0x7f6fdc02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcddba6b8e7ff624 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:05.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.095+0000 7f6fe0a00640 1 -- 172.21.15.133:0/3071802578 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc039070 con 0x7f6fe4071740 2024-03-07T06:53:05.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.095+0000 7f6fe0a00640 1 -- 172.21.15.133:0/3071802578 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc02cbe0 con 0x7f6fe4071740 2024-03-07T06:53:05.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.095+0000 7f6fe0a00640 1 -- 172.21.15.133:0/3071802578 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc034440 con 0x7f6fe4071740 2024-03-07T06:53:05.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.096+0000 7f6fe8e00640 1 -- 172.21.15.133:0/3071802578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fe4071740 msgr2=0x7f6fe40cf200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.094 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.096+0000 7f6fe8e00640 1 --2- 172.21.15.133:0/3071802578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fe4071740 0x7f6fe40cf200 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6fe408d1a0 tx=0x7f6fdc02be30 comp rx=0 tx=0).stop 2024-03-07T06:53:05.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.096+0000 7f6fe8e00640 1 -- 172.21.15.133:0/3071802578 shutdown_connections 2024-03-07T06:53:05.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.096+0000 7f6fe8e00640 1 --2- 172.21.15.133:0/3071802578 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fe4071740 0x7f6fe40cf200 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.096+0000 7f6fe8e00640 1 --2- 172.21.15.133:0/3071802578 >> v2:172.21.15.161:3300/0 conn(0x7f6fe415e480 0x7f6fe40b99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.096+0000 7f6fe8e00640 1 -- 172.21.15.133:0/3071802578 >> 172.21.15.133:0/3071802578 conn(0x7f6fe40d1130 msgr2=0x7f6fe40d4080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:05.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.096+0000 7f6fe8e00640 1 -- 172.21.15.133:0/3071802578 shutdown_connections 2024-03-07T06:53:05.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.096+0000 7f6fe8e00640 1 -- 172.21.15.133:0/3071802578 wait complete. 2024-03-07T06:53:05.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.099+0000 7f6fe8e00640 1 Processor -- start 2024-03-07T06:53:05.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.099+0000 7f6fe8e00640 1 -- start start 2024-03-07T06:53:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.101+0000 7f6fe8e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fe4071740 0x7f6fe41f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.101+0000 7f6fe8e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6fe40ced50 0x7f6fe4155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.101+0000 7f6fe8e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe415e480 0x7f6fe4155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.101+0000 7f6fe8e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40d2e60 con 0x7f6fe4071740 2024-03-07T06:53:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.101+0000 7f6fe8e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40d3ac0 con 0x7f6fe415e480 2024-03-07T06:53:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.101+0000 7f6fe8e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40d2fe0 con 0x7f6fe40ced50 2024-03-07T06:53:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.101+0000 7f6fe1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fe4071740 0x7f6fe41f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.101+0000 7f6fe1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fe4071740 0x7f6fe41f4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43922/0 (socket says 172.21.15.133:43922) 2024-03-07T06:53:05.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.101+0000 7f6fe1e00640 1 -- 172.21.15.133:0/719715947 learned_addr learned my addr 172.21.15.133:0/719715947 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:05.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.101+0000 7f6fe1e00640 1 -- 172.21.15.133:0/719715947 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6fe40ced50 msgr2=0x7f6fe4155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.101+0000 7f6fe1e00640 1 --2- 172.21.15.133:0/719715947 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6fe40ced50 0x7f6fe4155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.101+0000 7f6fe1e00640 1 -- 172.21.15.133:0/719715947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe415e480 msgr2=0x7f6fe4155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.101+0000 7f6fe1e00640 1 --2- 172.21.15.133:0/719715947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe415e480 0x7f6fe4155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.101+0000 7f6fe1e00640 1 -- 172.21.15.133:0/719715947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe4154060 con 0x7f6fe4071740 2024-03-07T06:53:05.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.102+0000 7f6fe1e00640 1 --2- 172.21.15.133:0/719715947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fe4071740 0x7f6fe41f4050 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400d800 tx=0x7f6fd400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:05.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.102+0000 7f6fda000640 1 -- 172.21.15.133:0/719715947 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd400b6c0 con 0x7f6fe4071740 2024-03-07T06:53:05.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.102+0000 7f6fda000640 1 -- 172.21.15.133:0/719715947 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd4019070 con 0x7f6fe4071740 2024-03-07T06:53:05.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.102+0000 7f6fda000640 1 -- 172.21.15.133:0/719715947 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd400ba80 con 0x7f6fe4071740 2024-03-07T06:53:05.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.102+0000 7f6fe8e00640 1 -- 172.21.15.133:0/719715947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe4154340 con 0x7f6fe4071740 2024-03-07T06:53:05.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.102+0000 7f6fe8e00640 1 -- 172.21.15.133:0/719715947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe4154900 con 0x7f6fe4071740 2024-03-07T06:53:05.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.103+0000 7f6fe8e00640 1 -- 172.21.15.133:0/719715947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe40ce1b0 con 0x7f6fe4071740 2024-03-07T06:53:05.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.104+0000 7f6fda000640 1 -- 172.21.15.133:0/719715947 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6fd400bc20 con 0x7f6fe4071740 2024-03-07T06:53:05.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.105+0000 7f6fda000640 1 --2- 172.21.15.133:0/719715947 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6fc006b4d0 0x7f6fc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.105+0000 7f6fda000640 1 -- 172.21.15.133:0/719715947 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fd408a760 con 0x7f6fe4071740 2024-03-07T06:53:05.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.106+0000 7f6fe1400640 1 --2- 172.21.15.133:0/719715947 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6fc006b4d0 0x7f6fc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.107+0000 7f6fe1400640 1 --2- 172.21.15.133:0/719715947 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6fc006b4d0 0x7f6fc006d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc005880 tx=0x7f6fcc007a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:05.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.110+0000 7f6fda000640 1 -- 172.21.15.133:0/719715947 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6fe40ce1b0 con 0x7f6fe4071740 2024-03-07T06:53:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.304+0000 7f6fe8e00640 1 -- 172.21.15.133:0/719715947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6fe4062c90 con 0x7f6fe4071740 2024-03-07T06:53:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.304+0000 7f6fda000640 1 -- 172.21.15.133:0/719715947 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6fd4059540 con 0x7f6fe4071740 2024-03-07T06:53:05.303 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:53:05.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.307+0000 7f6fe8e00640 1 -- 172.21.15.133:0/719715947 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6fc006b4d0 msgr2=0x7f6fc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.307+0000 7f6fe8e00640 1 --2- 172.21.15.133:0/719715947 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6fc006b4d0 0x7f6fc006d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc005880 tx=0x7f6fcc007a20 comp rx=0 tx=0).stop 2024-03-07T06:53:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.307+0000 7f6fe8e00640 1 -- 172.21.15.133:0/719715947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fe4071740 msgr2=0x7f6fe41f4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.307+0000 7f6fe8e00640 1 --2- 172.21.15.133:0/719715947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fe4071740 0x7f6fe41f4050 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400d800 tx=0x7f6fd400dcc0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.307+0000 7f6fe8e00640 1 -- 172.21.15.133:0/719715947 shutdown_connections 2024-03-07T06:53:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.307+0000 7f6fe8e00640 1 --2- 172.21.15.133:0/719715947 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6fc006b4d0 0x7f6fc006d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.307+0000 7f6fe8e00640 1 --2- 172.21.15.133:0/719715947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6fe415e480 0x7f6fe4155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.307+0000 7f6fe8e00640 1 --2- 172.21.15.133:0/719715947 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6fe40ced50 0x7f6fe4155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.307+0000 7f6fe8e00640 1 --2- 172.21.15.133:0/719715947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fe4071740 0x7f6fe41f4050 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.307+0000 7f6fe8e00640 1 -- 172.21.15.133:0/719715947 >> 172.21.15.133:0/719715947 conn(0x7f6fe40d1130 msgr2=0x7f6fe40cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.307+0000 7f6fe8e00640 1 -- 172.21.15.133:0/719715947 shutdown_connections 2024-03-07T06:53:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.308+0000 7f6fe8e00640 1 -- 172.21.15.133:0/719715947 wait complete. 2024-03-07T06:53:05.333 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:53:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.534+0000 7f8c75c00640 1 Processor -- start 2024-03-07T06:53:05.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.534+0000 7f8c75c00640 1 -- start start 2024-03-07T06:53:05.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.535+0000 7f8c75c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c70162110 0x7f8c701624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.535+0000 7f8c75c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8c70070bc0 0x7f8c70071020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.535+0000 7f8c75c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8c7015e8d0 0x7f8c7015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.535+0000 7f8c75c00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c70162a30 con 0x7f8c7015e8d0 2024-03-07T06:53:05.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.535+0000 7f8c75c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c70071610 con 0x7f8c70070bc0 2024-03-07T06:53:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.535+0000 7f8c75c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c700718f0 con 0x7f8c70162110 2024-03-07T06:53:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.535+0000 7f8c6f400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8c7015e8d0 0x7f8c7015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.535+0000 7f8c6f400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8c7015e8d0 0x7f8c7015ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43936/0 (socket says 172.21.15.133:43936) 2024-03-07T06:53:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.535+0000 7f8c6f400640 1 -- 172.21.15.133:0/3497319424 learned_addr learned my addr 172.21.15.133:0/3497319424 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.536+0000 7f8c6e000640 1 --2- 172.21.15.133:0/3497319424 >> v2:172.21.15.161:3300/0 conn(0x7f8c70070bc0 0x7f8c70071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.536+0000 7f8c6f400640 1 -- 172.21.15.133:0/3497319424 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c70162110 msgr2=0x7f8c701624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.536+0000 7f8c6f400640 1 --2- 172.21.15.133:0/3497319424 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c70162110 0x7f8c701624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.536+0000 7f8c6f400640 1 -- 172.21.15.133:0/3497319424 >> v2:172.21.15.161:3300/0 conn(0x7f8c70070bc0 msgr2=0x7f8c70071020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.536+0000 7f8c6f400640 1 --2- 172.21.15.133:0/3497319424 >> v2:172.21.15.161:3300/0 conn(0x7f8c70070bc0 0x7f8c70071020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.536+0000 7f8c6f400640 1 -- 172.21.15.133:0/3497319424 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c70071bd0 con 0x7f8c7015e8d0 2024-03-07T06:53:05.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.536+0000 7f8c6e000640 1 --2- 172.21.15.133:0/3497319424 >> v2:172.21.15.161:3300/0 conn(0x7f8c70070bc0 0x7f8c70071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.537+0000 7f8c6f400640 1 --2- 172.21.15.133:0/3497319424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c7015e8d0 0x7f8c7015ed60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8c700d3e60 tx=0x7f8c6802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197853f46fbebdf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.537+0000 7f8c6d600640 1 -- 172.21.15.133:0/3497319424 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6802ca70 con 0x7f8c7015e8d0 2024-03-07T06:53:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.537+0000 7f8c6d600640 1 -- 172.21.15.133:0/3497319424 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6802cc10 con 0x7f8c7015e8d0 2024-03-07T06:53:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.537+0000 7f8c6d600640 1 -- 172.21.15.133:0/3497319424 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c680343f0 con 0x7f8c7015e8d0 2024-03-07T06:53:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.538+0000 7f8c75c00640 1 -- 172.21.15.133:0/3497319424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c7015e8d0 msgr2=0x7f8c7015ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.538+0000 7f8c75c00640 1 --2- 172.21.15.133:0/3497319424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c7015e8d0 0x7f8c7015ed60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8c700d3e60 tx=0x7f8c6802bed0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.538+0000 7f8c75c00640 1 -- 172.21.15.133:0/3497319424 shutdown_connections 2024-03-07T06:53:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.538+0000 7f8c75c00640 1 --2- 172.21.15.133:0/3497319424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c7015e8d0 0x7f8c7015ed60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.538+0000 7f8c75c00640 1 --2- 172.21.15.133:0/3497319424 >> v2:172.21.15.161:3300/0 conn(0x7f8c70070bc0 0x7f8c70071020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.538+0000 7f8c75c00640 1 --2- 172.21.15.133:0/3497319424 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c70162110 0x7f8c701624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.538+0000 7f8c75c00640 1 -- 172.21.15.133:0/3497319424 >> 172.21.15.133:0/3497319424 conn(0x7f8c700d0e10 msgr2=0x7f8c700d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.538+0000 7f8c75c00640 1 -- 172.21.15.133:0/3497319424 shutdown_connections 2024-03-07T06:53:05.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.538+0000 7f8c75c00640 1 -- 172.21.15.133:0/3497319424 wait complete. 2024-03-07T06:53:05.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.540+0000 7f8c75c00640 1 Processor -- start 2024-03-07T06:53:05.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.540+0000 7f8c75c00640 1 -- start start 2024-03-07T06:53:05.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.542+0000 7f8c75c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c70070bc0 0x7f8c70154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.542+0000 7f8c75c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c7015e8d0 0x7f8c701f8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.542+0000 7f8c75c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c70162110 0x7f8c700b86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.542+0000 7f8c75c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70162e00 con 0x7f8c70162110 2024-03-07T06:53:05.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.542+0000 7f8c75c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c700d21d0 con 0x7f8c7015e8d0 2024-03-07T06:53:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.542+0000 7f8c75c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70162f80 con 0x7f8c70070bc0 2024-03-07T06:53:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.542+0000 7f8c6f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c70162110 0x7f8c700b86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.542+0000 7f8c6f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c70162110 0x7f8c700b86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43938/0 (socket says 172.21.15.133:43938) 2024-03-07T06:53:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.542+0000 7f8c6f400640 1 -- 172.21.15.133:0/4082861996 learned_addr learned my addr 172.21.15.133:0/4082861996 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.543+0000 7f8c6ea00640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c70070bc0 0x7f8c70154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.543+0000 7f8c6e000640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c7015e8d0 0x7f8c701f8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.543+0000 7f8c6f400640 1 -- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c70070bc0 msgr2=0x7f8c70154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.543+0000 7f8c6f400640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c70070bc0 0x7f8c70154830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.543+0000 7f8c6f400640 1 -- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c7015e8d0 msgr2=0x7f8c701f8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.543+0000 7f8c6f400640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c7015e8d0 0x7f8c701f8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.543+0000 7f8c6f400640 1 -- 172.21.15.133:0/4082861996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c700cc690 con 0x7f8c70162110 2024-03-07T06:53:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.543+0000 7f8c6ea00640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c70070bc0 0x7f8c70154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.543+0000 7f8c6e000640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c7015e8d0 0x7f8c701f8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.543+0000 7f8c6f400640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c70162110 0x7f8c700b86b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8c68036040 tx=0x7f8c68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:05.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.543+0000 7f8c66a00640 1 -- 172.21.15.133:0/4082861996 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c68043020 con 0x7f8c70162110 2024-03-07T06:53:05.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.543+0000 7f8c66a00640 1 -- 172.21.15.133:0/4082861996 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6803e450 con 0x7f8c70162110 2024-03-07T06:53:05.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.544+0000 7f8c75c00640 1 -- 172.21.15.133:0/4082861996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c701f9590 con 0x7f8c70162110 2024-03-07T06:53:05.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.544+0000 7f8c66a00640 1 -- 172.21.15.133:0/4082861996 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6803e860 con 0x7f8c70162110 2024-03-07T06:53:05.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.544+0000 7f8c75c00640 1 -- 172.21.15.133:0/4082861996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c700cc950 con 0x7f8c70162110 2024-03-07T06:53:05.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.545+0000 7f8c75c00640 1 -- 172.21.15.133:0/4082861996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c70062c90 con 0x7f8c70162110 2024-03-07T06:53:05.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.546+0000 7f8c66a00640 1 -- 172.21.15.133:0/4082861996 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8c68046020 con 0x7f8c70162110 2024-03-07T06:53:05.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.547+0000 7f8c66a00640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8c5006b4d0 0x7f8c5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.548+0000 7f8c66a00640 1 -- 172.21.15.133:0/4082861996 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c680acbc0 con 0x7f8c70162110 2024-03-07T06:53:05.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.548+0000 7f8c6ea00640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8c5006b4d0 0x7f8c5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.549+0000 7f8c6ea00640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8c5006b4d0 0x7f8c5006d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8c60006fd0 tx=0x7f8c60008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:05.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.551+0000 7f8c66a00640 1 -- 172.21.15.133:0/4082861996 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8c6804d030 con 0x7f8c70162110 2024-03-07T06:53:05.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.745+0000 7f8c75c00640 1 -- 172.21.15.133:0/4082861996 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c700ccb20 con 0x7f8c70162110 2024-03-07T06:53:05.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.746+0000 7f8c66a00640 1 -- 172.21.15.133:0/4082861996 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c6807b9a0 con 0x7f8c70162110 2024-03-07T06:53:05.744 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:53:05.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.748+0000 7f8c75c00640 1 -- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8c5006b4d0 msgr2=0x7f8c5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.748+0000 7f8c75c00640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8c5006b4d0 0x7f8c5006d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8c60006fd0 tx=0x7f8c60008040 comp rx=0 tx=0).stop 2024-03-07T06:53:05.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.748+0000 7f8c75c00640 1 -- 172.21.15.133:0/4082861996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c70162110 msgr2=0x7f8c700b86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.748+0000 7f8c75c00640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c70162110 0x7f8c700b86b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8c68036040 tx=0x7f8c68004a40 comp rx=0 tx=0).stop 2024-03-07T06:53:05.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.748+0000 7f8c75c00640 1 -- 172.21.15.133:0/4082861996 shutdown_connections 2024-03-07T06:53:05.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.748+0000 7f8c75c00640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c70162110 0x7f8c700b86b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.748+0000 7f8c75c00640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8c5006b4d0 0x7f8c5006d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.748+0000 7f8c75c00640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8c7015e8d0 0x7f8c701f8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.748+0000 7f8c75c00640 1 --2- 172.21.15.133:0/4082861996 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8c70070bc0 0x7f8c70154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.748+0000 7f8c75c00640 1 -- 172.21.15.133:0/4082861996 >> 172.21.15.133:0/4082861996 conn(0x7f8c700d0e10 msgr2=0x7f8c700b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:05.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.749+0000 7f8c75c00640 1 -- 172.21.15.133:0/4082861996 shutdown_connections 2024-03-07T06:53:05.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.749+0000 7f8c75c00640 1 -- 172.21.15.133:0/4082861996 wait complete. 2024-03-07T06:53:05.765 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-07T06:53:05.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.974+0000 7f83e1800640 1 Processor -- start 2024-03-07T06:53:05.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.974+0000 7f83e1800640 1 -- start start 2024-03-07T06:53:05.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.976+0000 7f83e1800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f83dc0d12c0 0x7f83dc143af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.976+0000 7f83e1800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83dc15ea00 con 0x7f83dc141250 2024-03-07T06:53:05.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.976+0000 7f83e1800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83dc071550 con 0x7f83dc140e70 2024-03-07T06:53:05.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.976+0000 7f83e1800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83dc073020 con 0x7f83dc0d12c0 2024-03-07T06:53:05.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.976+0000 7f83d3400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f83dc141250 0x7f83dc0d0bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50164/0 (socket says 172.21.15.133:50164) 2024-03-07T06:53:05.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.976+0000 7f83d3400640 1 -- 172.21.15.133:0/3876655950 learned_addr learned my addr 172.21.15.133:0/3876655950 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:05.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 -- 172.21.15.133:0/3876655950 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2637403042 0 0) 0x7f83dc15ea00 con 0x7f83dc141250 2024-03-07T06:53:05.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 -- 172.21.15.133:0/3876655950 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83c0003520 con 0x7f83dc141250 2024-03-07T06:53:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83e0e00640 1 --2- 172.21.15.133:0/3876655950 >> v2:172.21.15.161:3300/0 conn(0x7f83dc0d12c0 0x7f83dc143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 -- 172.21.15.133:0/3876655950 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1675501411 0 0) 0x7f83c0003520 con 0x7f83dc141250 2024-03-07T06:53:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 -- 172.21.15.133:0/3876655950 >> v2:172.21.15.161:3300/0 conn(0x7f83dc0d12c0 msgr2=0x7f83dc143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 --2- 172.21.15.133:0/3876655950 >> v2:172.21.15.161:3300/0 conn(0x7f83dc0d12c0 0x7f83dc143af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 -- 172.21.15.133:0/3876655950 >> v1:172.21.15.161:6789/0 conn(0x7f83dc140e70 legacy=0x7f83dc161c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 -- 172.21.15.133:0/3876655950 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc1627f0 con 0x7f83dc141250 2024-03-07T06:53:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 -- 172.21.15.133:0/3876655950 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f83c4001670 con 0x7f83dc141250 2024-03-07T06:53:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 -- 172.21.15.133:0/3876655950 >> v1:172.21.15.133:6789/0 conn(0x7f83dc141250 legacy=0x7f83dc0d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 --2- 172.21.15.133:0/3876655950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83c00039b0 0x7f83c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 --2- 172.21.15.133:0/3876655950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83c0004800 0x7f83c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 --2- 172.21.15.133:0/3876655950 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83c00051c0 0x7f83c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 -- 172.21.15.133:0/3876655950 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c000c030 con 0x7f83c0004800 2024-03-07T06:53:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 -- 172.21.15.133:0/3876655950 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c000c1e0 con 0x7f83c00039b0 2024-03-07T06:53:05.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.977+0000 7f83db400640 1 -- 172.21.15.133:0/3876655950 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c000c3e0 con 0x7f83c00051c0 2024-03-07T06:53:05.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.978+0000 7f83d3400640 1 --2- 172.21.15.133:0/3876655950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83c0004800 0x7f83c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.978+0000 7f83dbe00640 1 --2- 172.21.15.133:0/3876655950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83c00039b0 0x7f83c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.978+0000 7f83e0e00640 1 --2- 172.21.15.133:0/3876655950 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83c00051c0 0x7f83c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.978+0000 7f83d3400640 1 -- 172.21.15.133:0/3876655950 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83c00051c0 msgr2=0x7f83c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:05.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.978+0000 7f83d3400640 1 --2- 172.21.15.133:0/3876655950 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83c00051c0 0x7f83c000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.978+0000 7f83d3400640 1 -- 172.21.15.133:0/3876655950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83c00039b0 msgr2=0x7f83c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:05.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.978+0000 7f83d3400640 1 --2- 172.21.15.133:0/3876655950 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83c00039b0 0x7f83c0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.978+0000 7f83d3400640 1 -- 172.21.15.133:0/3876655950 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c0003520 con 0x7f83c0004800 2024-03-07T06:53:05.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.978+0000 7f83e0e00640 1 --2- 172.21.15.133:0/3876655950 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83c00051c0 0x7f83c000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:05.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.978+0000 7f83e0e00640 1 -- 172.21.15.133:0/3876655950 reap_dead start 2024-03-07T06:53:05.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.979+0000 7f83d3400640 1 --2- 172.21.15.133:0/3876655950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83c0004800 0x7f83c0004c80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f83c400c210 tx=0x7f83c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b00c8de951a469 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.979+0000 7f83db400640 1 -- 172.21.15.133:0/3876655950 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4007d10 con 0x7f83c0004800 2024-03-07T06:53:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.979+0000 7f83db400640 1 -- 172.21.15.133:0/3876655950 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c4030070 con 0x7f83c0004800 2024-03-07T06:53:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.979+0000 7f83db400640 1 -- 172.21.15.133:0/3876655950 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4005d90 con 0x7f83c0004800 2024-03-07T06:53:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.980+0000 7f83e1800640 1 -- 172.21.15.133:0/3876655950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83c0004800 msgr2=0x7f83c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.980+0000 7f83e1800640 1 --2- 172.21.15.133:0/3876655950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83c0004800 0x7f83c0004c80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f83c400c210 tx=0x7f83c0005830 comp rx=0 tx=0).stop 2024-03-07T06:53:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.980+0000 7f83e1800640 1 -- 172.21.15.133:0/3876655950 shutdown_connections 2024-03-07T06:53:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.980+0000 7f83e1800640 1 --2- 172.21.15.133:0/3876655950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83c0004800 0x7f83c0004c80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.980+0000 7f83e1800640 1 -- 172.21.15.133:0/3876655950 >> 172.21.15.133:0/3876655950 conn(0x7f83dc06b510 msgr2=0x7f83dc06b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.980+0000 7f83e1800640 1 -- 172.21.15.133:0/3876655950 shutdown_connections 2024-03-07T06:53:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.980+0000 7f83e1800640 1 -- 172.21.15.133:0/3876655950 wait complete. 2024-03-07T06:53:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.982+0000 7f83e1800640 1 Processor -- start 2024-03-07T06:53:05.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.982+0000 7f83e1800640 1 -- start start 2024-03-07T06:53:05.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.983+0000 7f83e1800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83c0004800 0x7f83dc0b2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.983+0000 7f83e1800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83dc0b43d0 0x7f83dc0b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.983+0000 7f83e1800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83dc0d01f0 0x7f83dc0d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.983+0000 7f83e1800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc071af0 con 0x7f83dc0b43d0 2024-03-07T06:53:05.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.983+0000 7f83e1800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc071810 con 0x7f83c0004800 2024-03-07T06:53:05.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.983+0000 7f83e1800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83dc1417a0 con 0x7f83dc0d01f0 2024-03-07T06:53:05.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.983+0000 7f83d3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83dc0b43d0 0x7f83dc0b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.983+0000 7f83d3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83dc0b43d0 0x7f83dc0b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43956/0 (socket says 172.21.15.133:43956) 2024-03-07T06:53:05.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.983+0000 7f83d3400640 1 -- 172.21.15.133:0/154143715 learned_addr learned my addr 172.21.15.133:0/154143715 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.984+0000 7f83e0e00640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83dc0d01f0 0x7f83dc0d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.984+0000 7f83dbe00640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83c0004800 0x7f83dc0b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.984+0000 7f83d3400640 1 -- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83dc0d01f0 msgr2=0x7f83dc0d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.984+0000 7f83d3400640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83dc0d01f0 0x7f83dc0d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.984+0000 7f83d3400640 1 -- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83c0004800 msgr2=0x7f83dc0b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.984+0000 7f83d3400640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83c0004800 0x7f83dc0b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.984+0000 7f83d3400640 1 -- 172.21.15.133:0/154143715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc0d0be0 con 0x7f83dc0b43d0 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.984+0000 7f83dbe00640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83c0004800 0x7f83dc0b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.984+0000 7f83e0e00640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83dc0d01f0 0x7f83dc0d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.984+0000 7f83d3400640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83dc0b43d0 0x7f83dc0b4830 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f83c400c210 tx=0x7f83dc0cb5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.984+0000 7f83d8c00640 1 -- 172.21.15.133:0/154143715 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4045070 con 0x7f83dc0b43d0 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.984+0000 7f83d8c00640 1 -- 172.21.15.133:0/154143715 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c402c9d0 con 0x7f83dc0b43d0 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.984+0000 7f83d8c00640 1 -- 172.21.15.133:0/154143715 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4007d10 con 0x7f83dc0b43d0 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.984+0000 7f83e1800640 1 -- 172.21.15.133:0/154143715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83dc0b4f90 con 0x7f83dc0b43d0 2024-03-07T06:53:05.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.985+0000 7f83e1800640 1 -- 172.21.15.133:0/154143715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83dc12e5d0 con 0x7f83dc0b43d0 2024-03-07T06:53:05.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.986+0000 7f83e1800640 1 -- 172.21.15.133:0/154143715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83dc062c90 con 0x7f83dc0b43d0 2024-03-07T06:53:05.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.987+0000 7f83d8c00640 1 -- 172.21.15.133:0/154143715 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f83c4007070 con 0x7f83dc0b43d0 2024-03-07T06:53:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.988+0000 7f83d8c00640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f83b806b4d0 0x7f83b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.988+0000 7f83d8c00640 1 -- 172.21.15.133:0/154143715 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83c40aadf0 con 0x7f83dc0b43d0 2024-03-07T06:53:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.989+0000 7f83dbe00640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f83b806b4d0 0x7f83b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:05.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.990+0000 7f83dbe00640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f83b806b4d0 0x7f83b806d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f83cc000c00 tx=0x7f83cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:05.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:05.992+0000 7f83d8c00640 1 -- 172.21.15.133:0/154143715 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f83dc0d0be0 con 0x7f83dc0b43d0 2024-03-07T06:53:06.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.265+0000 7f83e1800640 1 -- 172.21.15.133:0/154143715 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f83dc06c3e0 con 0x7f83dc0b43d0 2024-03-07T06:53:06.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.266+0000 7f83d8c00640 1 -- 172.21.15.133:0/154143715 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7f83c4079d00 con 0x7f83dc0b43d0 2024-03-07T06:53:06.265 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-03-07T06:53:06.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.270+0000 7f83e1800640 1 -- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f83b806b4d0 msgr2=0x7f83b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:06.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.270+0000 7f83e1800640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f83b806b4d0 0x7f83b806d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f83cc000c00 tx=0x7f83cc009040 comp rx=0 tx=0).stop 2024-03-07T06:53:06.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.270+0000 7f83e1800640 1 -- 172.21.15.133:0/154143715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83dc0b43d0 msgr2=0x7f83dc0b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:06.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.270+0000 7f83e1800640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83dc0b43d0 0x7f83dc0b4830 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f83c400c210 tx=0x7f83dc0cb5e0 comp rx=0 tx=0).stop 2024-03-07T06:53:06.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.270+0000 7f83e1800640 1 -- 172.21.15.133:0/154143715 shutdown_connections 2024-03-07T06:53:06.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.270+0000 7f83e1800640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83dc0d01f0 0x7f83dc0d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:06.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.270+0000 7f83e1800640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f83dc0b43d0 0x7f83dc0b4830 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:06.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.270+0000 7f83e1800640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f83b806b4d0 0x7f83b806d990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:06.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.270+0000 7f83e1800640 1 --2- 172.21.15.133:0/154143715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83c0004800 0x7f83dc0b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:06.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.270+0000 7f83e1800640 1 -- 172.21.15.133:0/154143715 >> 172.21.15.133:0/154143715 conn(0x7f83dc06b510 msgr2=0x7f83dc15ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:06.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.270+0000 7f83e1800640 1 -- 172.21.15.133:0/154143715 shutdown_connections 2024-03-07T06:53:06.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.270+0000 7f83e1800640 1 -- 172.21.15.133:0/154143715 wait complete. 2024-03-07T06:53:06.292 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-07T06:53:06.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:53:06.292 DEBUG:teuthology.orchestra.run.smithi133:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-07T06:53:06.410 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:53:06.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:53:06.410 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-07T06:53:06.457 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:53:06.457 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:06.497 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053429,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-07T06:53:06.498 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:53:06.498 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:06.535 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053417,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-07T06:53:06.535 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:53:06.535 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:06.586 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053429,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-07T06:53:06.586 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:53:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.801+0000 7f504d800640 1 Processor -- start 2024-03-07T06:53:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.801+0000 7f504d800640 1 -- start start 2024-03-07T06:53:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.803+0000 7f504d800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5048140cb0 0x7f5048162290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.803+0000 7f504d800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f50480d2b00 0x7f50480b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.803+0000 7f504d800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5048143810 con 0x7f50480b70d0 2024-03-07T06:53:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.803+0000 7f504d800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50480b7a70 con 0x7f50480d2b00 2024-03-07T06:53:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.803+0000 7f504d800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50481435c0 con 0x7f5048140cb0 2024-03-07T06:53:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.803+0000 7f5047400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f50480b70d0 0x7f50480b9300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50172/0 (socket says 172.21.15.133:50172) 2024-03-07T06:53:06.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.803+0000 7f5047400640 1 -- 172.21.15.133:0/3480358389 learned_addr learned my addr 172.21.15.133:0/3480358389 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.804+0000 7f504ce00640 1 --2- 172.21.15.133:0/3480358389 >> v2:172.21.15.161:3300/0 conn(0x7f50480d2b00 0x7f50480b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.804+0000 7f5047e00640 1 --2- 172.21.15.133:0/3480358389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5048140cb0 0x7f5048162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.804+0000 7f5046a00640 1 -- 172.21.15.133:0/3480358389 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091304830 0 0) 0x7f5048143810 con 0x7f50480b70d0 2024-03-07T06:53:06.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.804+0000 7f5046a00640 1 -- 172.21.15.133:0/3480358389 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f502c003520 con 0x7f50480b70d0 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.804+0000 7f5046a00640 1 -- 172.21.15.133:0/3480358389 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2802649711 0 0) 0x7f502c003520 con 0x7f50480b70d0 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.804+0000 7f5046a00640 1 -- 172.21.15.133:0/3480358389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5048140cb0 msgr2=0x7f5048162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.804+0000 7f5046a00640 1 --2- 172.21.15.133:0/3480358389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5048140cb0 0x7f5048162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.804+0000 7f5046a00640 1 -- 172.21.15.133:0/3480358389 >> v2:172.21.15.161:3300/0 conn(0x7f50480d2b00 msgr2=0x7f50480b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.804+0000 7f5046a00640 1 --2- 172.21.15.133:0/3480358389 >> v2:172.21.15.161:3300/0 conn(0x7f50480d2b00 0x7f50480b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.804+0000 7f5046a00640 1 -- 172.21.15.133:0/3480358389 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5048143af0 con 0x7f50480b70d0 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.804+0000 7f5046a00640 1 -- 172.21.15.133:0/3480358389 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f50380019f0 con 0x7f50480b70d0 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.804+0000 7f5046a00640 1 -- 172.21.15.133:0/3480358389 >> v1:172.21.15.133:6789/0 conn(0x7f50480b70d0 legacy=0x7f50480b9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.804+0000 7f5046a00640 1 --2- 172.21.15.133:0/3480358389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5048140cb0 0x7f502c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.804+0000 7f5046a00640 1 --2- 172.21.15.133:0/3480358389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c003f30 0x7f502c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.805+0000 7f5046a00640 1 --2- 172.21.15.133:0/3480358389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f502c004d40 0x7f502c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.805+0000 7f5046a00640 1 -- 172.21.15.133:0/3480358389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50481435c0 con 0x7f502c003f30 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.805+0000 7f5046a00640 1 -- 172.21.15.133:0/3480358389 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c00b9c0 con 0x7f502c004d40 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.805+0000 7f5046a00640 1 -- 172.21.15.133:0/3480358389 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c00bb40 con 0x7f5048140cb0 2024-03-07T06:53:06.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.805+0000 7f5047400640 1 --2- 172.21.15.133:0/3480358389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c003f30 0x7f502c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.805+0000 7f5047e00640 1 --2- 172.21.15.133:0/3480358389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5048140cb0 0x7f502c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.805+0000 7f504ce00640 1 --2- 172.21.15.133:0/3480358389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f502c004d40 0x7f502c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.805+0000 7f5047400640 1 -- 172.21.15.133:0/3480358389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5048140cb0 msgr2=0x7f502c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.805+0000 7f5047400640 1 --2- 172.21.15.133:0/3480358389 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5048140cb0 0x7f502c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.805+0000 7f5047400640 1 -- 172.21.15.133:0/3480358389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f502c004d40 msgr2=0x7f502c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.805+0000 7f5047400640 1 --2- 172.21.15.133:0/3480358389 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f502c004d40 0x7f502c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:06.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.805+0000 7f5047400640 1 -- 172.21.15.133:0/3480358389 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c003520 con 0x7f502c003f30 2024-03-07T06:53:06.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.806+0000 7f5047400640 1 --2- 172.21.15.133:0/3480358389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c003f30 0x7f502c004800 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f503800a870 tx=0x7f502c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de92cb9974b931b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:06.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.807+0000 7f5046a00640 1 -- 172.21.15.133:0/3480358389 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503800a4c0 con 0x7f502c003f30 2024-03-07T06:53:06.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.807+0000 7f5046a00640 1 -- 172.21.15.133:0/3480358389 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5038034630 con 0x7f502c003f30 2024-03-07T06:53:06.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.807+0000 7f5046a00640 1 -- 172.21.15.133:0/3480358389 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5038034a20 con 0x7f502c003f30 2024-03-07T06:53:06.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.807+0000 7f504d800640 1 -- 172.21.15.133:0/3480358389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c003f30 msgr2=0x7f502c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:06.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.807+0000 7f504d800640 1 --2- 172.21.15.133:0/3480358389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f502c003f30 0x7f502c004800 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f503800a870 tx=0x7f502c005830 comp rx=0 tx=0).stop 2024-03-07T06:53:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.807+0000 7f504ce00640 1 -- 172.21.15.133:0/3480358389 reap_dead start 2024-03-07T06:53:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.807+0000 7f504d800640 1 -- 172.21.15.133:0/3480358389 shutdown_connections 2024-03-07T06:53:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.807+0000 7f504d800640 1 -- 172.21.15.133:0/3480358389 >> 172.21.15.133:0/3480358389 conn(0x7f50480674e0 msgr2=0x7f50480d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.807+0000 7f504d800640 1 -- 172.21.15.133:0/3480358389 shutdown_connections 2024-03-07T06:53:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.807+0000 7f504d800640 1 -- 172.21.15.133:0/3480358389 wait complete. 2024-03-07T06:53:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.809+0000 7f504d800640 1 Processor -- start 2024-03-07T06:53:06.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.810+0000 7f504d800640 1 -- start start 2024-03-07T06:53:06.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.811+0000 7f504d800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f502c003f30 0x7f5048153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:06.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.811+0000 7f504d800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048154380 0x7f50481420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:06.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.811+0000 7f504d800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5048142620 0x7f5048142aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:06.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.811+0000 7f504d800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50481440b0 con 0x7f5048154380 2024-03-07T06:53:06.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.811+0000 7f504d800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5048144420 con 0x7f502c003f30 2024-03-07T06:53:06.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.811+0000 7f504d800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5048161bb0 con 0x7f5048142620 2024-03-07T06:53:06.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.811+0000 7f5047400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048154380 0x7f50481420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:06.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.811+0000 7f5047400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048154380 0x7f50481420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43968/0 (socket says 172.21.15.133:43968) 2024-03-07T06:53:06.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.811+0000 7f5047400640 1 -- 172.21.15.133:0/1198807758 learned_addr learned my addr 172.21.15.133:0/1198807758 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:06.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.812+0000 7f5047e00640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f502c003f30 0x7f5048153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:06.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.812+0000 7f504ce00640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5048142620 0x7f5048142aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:06.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.812+0000 7f5047400640 1 -- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5048142620 msgr2=0x7f5048142aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:06.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.812+0000 7f5047400640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5048142620 0x7f5048142aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:06.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.812+0000 7f5047400640 1 -- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f502c003f30 msgr2=0x7f5048153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:06.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.812+0000 7f5047400640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f502c003f30 0x7f5048153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:06.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.812+0000 7f5047400640 1 -- 172.21.15.133:0/1198807758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50481f62f0 con 0x7f5048154380 2024-03-07T06:53:06.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.812+0000 7f5047e00640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f502c003f30 0x7f5048153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:06.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.812+0000 7f504ce00640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5048142620 0x7f5048142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:06.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.812+0000 7f5047400640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048154380 0x7f50481420e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f503800a960 tx=0x7f5048144390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.812+0000 7f503fe00640 1 -- 172.21.15.133:0/1198807758 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5038044070 con 0x7f5048154380 2024-03-07T06:53:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.812+0000 7f503fe00640 1 -- 172.21.15.133:0/1198807758 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5038004b60 con 0x7f5048154380 2024-03-07T06:53:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.812+0000 7f503fe00640 1 -- 172.21.15.133:0/1198807758 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5038039040 con 0x7f5048154380 2024-03-07T06:53:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.812+0000 7f504d800640 1 -- 172.21.15.133:0/1198807758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50481f65d0 con 0x7f5048154380 2024-03-07T06:53:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.813+0000 7f504d800640 1 -- 172.21.15.133:0/1198807758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50481f6b90 con 0x7f5048154380 2024-03-07T06:53:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.814+0000 7f504d800640 1 -- 172.21.15.133:0/1198807758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5048062c90 con 0x7f5048154380 2024-03-07T06:53:06.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.814+0000 7f503fe00640 1 -- 172.21.15.133:0/1198807758 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5038047070 con 0x7f5048154380 2024-03-07T06:53:06.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.816+0000 7f503fe00640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f502406b4d0 0x7f502406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:06.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.816+0000 7f503fe00640 1 -- 172.21.15.133:0/1198807758 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50380abc90 con 0x7f5048154380 2024-03-07T06:53:06.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.816+0000 7f5047e00640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f502406b4d0 0x7f502406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:06.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.817+0000 7f5047e00640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f502406b4d0 0x7f502406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5030006ce0 tx=0x7f5030008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:06.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:06.821+0000 7f503fe00640 1 -- 172.21.15.133:0/1198807758 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f503807ab00 con 0x7f5048154380 2024-03-07T06:53:07.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.152+0000 7f504d800640 1 -- 172.21.15.133:0/1198807758 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5048153a00 con 0x7f5048154380 2024-03-07T06:53:07.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.153+0000 7f503fe00640 1 -- 172.21.15.133:0/1198807758 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4983 (secure 0 0 0) 0x7f503807a400 con 0x7f5048154380 2024-03-07T06:53:07.152 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:07.152 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3269394125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3269394125},{"type":"v1","addr":"172.21.15.161:6835","nonce":3269394125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/72243172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":72243172},{"type":"v1","addr":"172.21.15.161:6837","nonce":72243172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3714291615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3714291615},{"type":"v1","addr":"172.21.15.133:6834","nonce":3714291615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:52:35.182567+0000","modified":"2024-03-07T06:52:38.268724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6835/4067860709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":4067860709},{"type":"v1","addr":"172.21.15.133:6835","nonce":4067860709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5100,"qdb_cluster":[5100]},"id":2}]} 2024-03-07T06:53:07.152 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 18 2024-03-07T06:53:07.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.156+0000 7f504d800640 1 -- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f502406b4d0 msgr2=0x7f502406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:07.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.156+0000 7f504d800640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f502406b4d0 0x7f502406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5030006ce0 tx=0x7f5030008040 comp rx=0 tx=0).stop 2024-03-07T06:53:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.156+0000 7f504d800640 1 -- 172.21.15.133:0/1198807758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048154380 msgr2=0x7f50481420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.156+0000 7f504d800640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048154380 0x7f50481420e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f503800a960 tx=0x7f5048144390 comp rx=0 tx=0).stop 2024-03-07T06:53:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.157+0000 7f504d800640 1 -- 172.21.15.133:0/1198807758 shutdown_connections 2024-03-07T06:53:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.157+0000 7f504d800640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5048142620 0x7f5048142aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.157+0000 7f504d800640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5048154380 0x7f50481420e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.157+0000 7f504d800640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f502406b4d0 0x7f502406d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.157+0000 7f504d800640 1 --2- 172.21.15.133:0/1198807758 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f502c003f30 0x7f5048153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.157+0000 7f504d800640 1 -- 172.21.15.133:0/1198807758 >> 172.21.15.133:0/1198807758 conn(0x7f50480674e0 msgr2=0x7f5048160d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.157+0000 7f504d800640 1 -- 172.21.15.133:0/1198807758 shutdown_connections 2024-03-07T06:53:07.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.157+0000 7f504d800640 1 -- 172.21.15.133:0/1198807758 wait complete. 2024-03-07T06:53:07.174 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:53:07.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.368+0000 7fe0a9600640 1 Processor -- start 2024-03-07T06:53:07.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.369+0000 7fe0a9600640 1 -- start start 2024-03-07T06:53:07.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.370+0000 7fe0a9600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe0a4147bb0 0x7fe0a40d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:07.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.370+0000 7fe0a9600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe0a40b5090 0x7fe0a40d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:07.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.370+0000 7fe0a9600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a415fc30 con 0x7fe0a4072910 2024-03-07T06:53:07.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.370+0000 7fe0a9600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a4161ce0 con 0x7fe0a4147bb0 2024-03-07T06:53:07.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.370+0000 7fe0a9600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a415ff10 con 0x7fe0a40b5090 2024-03-07T06:53:07.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.370+0000 7fe0a2a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe0a4147bb0 0x7fe0a40d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:07.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.370+0000 7fe0a2a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe0a4147bb0 0x7fe0a40d0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43978/0 (socket says 172.21.15.133:43978) 2024-03-07T06:53:07.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.370+0000 7fe0a2a00640 1 -- 172.21.15.133:0/3324366162 learned_addr learned my addr 172.21.15.133:0/3324366162 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:07.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.371+0000 7fe0a2a00640 1 -- 172.21.15.133:0/3324366162 >> v2:172.21.15.161:3300/0 conn(0x7fe0a40b5090 msgr2=0x7fe0a40d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:07.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.371+0000 7fe0a2a00640 1 --2- 172.21.15.133:0/3324366162 >> v2:172.21.15.161:3300/0 conn(0x7fe0a40b5090 0x7fe0a40d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.371+0000 7fe0a2a00640 1 -- 172.21.15.133:0/3324366162 >> v1:172.21.15.161:6789/0 conn(0x7fe0a4072910 legacy=0x7fe0a4072d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:07.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.371+0000 7fe0a2a00640 1 -- 172.21.15.133:0/3324366162 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a41601f0 con 0x7fe0a4147bb0 2024-03-07T06:53:07.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.372+0000 7fe0a2a00640 1 --2- 172.21.15.133:0/3324366162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0a4147bb0 0x7fe0a40d0b50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40b3e00 tx=0x7fe09402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae08c6dd4c4f943c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:07.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.372+0000 7fe0a1600640 1 -- 172.21.15.133:0/3324366162 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09402e070 con 0x7fe0a4147bb0 2024-03-07T06:53:07.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.372+0000 7fe0a1600640 1 -- 172.21.15.133:0/3324366162 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe094037070 con 0x7fe0a4147bb0 2024-03-07T06:53:07.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.372+0000 7fe0a1600640 1 -- 172.21.15.133:0/3324366162 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe094031a00 con 0x7fe0a4147bb0 2024-03-07T06:53:07.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.373+0000 7fe0a9600640 1 -- 172.21.15.133:0/3324366162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0a4147bb0 msgr2=0x7fe0a40d0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:07.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.373+0000 7fe0a9600640 1 --2- 172.21.15.133:0/3324366162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0a4147bb0 0x7fe0a40d0b50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40b3e00 tx=0x7fe09402b880 comp rx=0 tx=0).stop 2024-03-07T06:53:07.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.373+0000 7fe0a9600640 1 -- 172.21.15.133:0/3324366162 shutdown_connections 2024-03-07T06:53:07.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.373+0000 7fe0a9600640 1 --2- 172.21.15.133:0/3324366162 >> v2:172.21.15.161:3300/0 conn(0x7fe0a40b5090 0x7fe0a40d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.373+0000 7fe0a9600640 1 --2- 172.21.15.133:0/3324366162 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0a4147bb0 0x7fe0a40d0b50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.373+0000 7fe0a9600640 1 -- 172.21.15.133:0/3324366162 >> 172.21.15.133:0/3324366162 conn(0x7fe0a406b480 msgr2=0x7fe0a406b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:07.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.373+0000 7fe0a9600640 1 -- 172.21.15.133:0/3324366162 shutdown_connections 2024-03-07T06:53:07.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.373+0000 7fe0a9600640 1 -- 172.21.15.133:0/3324366162 wait complete. 2024-03-07T06:53:07.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.376+0000 7fe0a9600640 1 Processor -- start 2024-03-07T06:53:07.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.376+0000 7fe0a9600640 1 -- start start 2024-03-07T06:53:07.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.378+0000 7fe0a9600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe0a4072910 0x7fe0a40b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:07.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.378+0000 7fe0a9600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0a40b5090 0x7fe0a415a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:07.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.378+0000 7fe0a9600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0a4147bb0 0x7fe0a415ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:07.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.378+0000 7fe0a9600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a4147f90 con 0x7fe0a4147bb0 2024-03-07T06:53:07.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.378+0000 7fe0a9600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a41607b0 con 0x7fe0a40b5090 2024-03-07T06:53:07.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.378+0000 7fe0a9600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a4160cd0 con 0x7fe0a4072910 2024-03-07T06:53:07.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.378+0000 7fe0a3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0a4147bb0 0x7fe0a415ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.379+0000 7fe0a3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0a4147bb0 0x7fe0a415ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43990/0 (socket says 172.21.15.133:43990) 2024-03-07T06:53:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.379+0000 7fe0a3400640 1 -- 172.21.15.133:0/3432100190 learned_addr learned my addr 172.21.15.133:0/3432100190 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.379+0000 7fe0a2000640 1 --2- 172.21.15.133:0/3432100190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0a40b5090 0x7fe0a415a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.379+0000 7fe0a2a00640 1 --2- 172.21.15.133:0/3432100190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe0a4072910 0x7fe0a40b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.379+0000 7fe0a3400640 1 -- 172.21.15.133:0/3432100190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe0a4072910 msgr2=0x7fe0a40b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.379+0000 7fe0a3400640 1 --2- 172.21.15.133:0/3432100190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe0a4072910 0x7fe0a40b6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.379+0000 7fe0a3400640 1 -- 172.21.15.133:0/3432100190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0a40b5090 msgr2=0x7fe0a415a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.379+0000 7fe0a3400640 1 --2- 172.21.15.133:0/3432100190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0a40b5090 0x7fe0a415a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.379+0000 7fe0a3400640 1 -- 172.21.15.133:0/3432100190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a40cb4c0 con 0x7fe0a4147bb0 2024-03-07T06:53:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.379+0000 7fe0a3400640 1 --2- 172.21.15.133:0/3432100190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0a4147bb0 0x7fe0a415ad60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00c780 tx=0x7fe09c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.379+0000 7fe09aa00640 1 -- 172.21.15.133:0/3432100190 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c017070 con 0x7fe0a4147bb0 2024-03-07T06:53:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.379+0000 7fe09aa00640 1 -- 172.21.15.133:0/3432100190 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09c003da0 con 0x7fe0a4147bb0 2024-03-07T06:53:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.380+0000 7fe09aa00640 1 -- 172.21.15.133:0/3432100190 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09c0052b0 con 0x7fe0a4147bb0 2024-03-07T06:53:07.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.380+0000 7fe0a9600640 1 -- 172.21.15.133:0/3432100190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a40cb7a0 con 0x7fe0a4147bb0 2024-03-07T06:53:07.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.380+0000 7fe0a9600640 1 -- 172.21.15.133:0/3432100190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a40cbd60 con 0x7fe0a4147bb0 2024-03-07T06:53:07.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.381+0000 7fe0a9600640 1 -- 172.21.15.133:0/3432100190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a40cc220 con 0x7fe0a4147bb0 2024-03-07T06:53:07.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.382+0000 7fe09aa00640 1 -- 172.21.15.133:0/3432100190 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe09c01a070 con 0x7fe0a4147bb0 2024-03-07T06:53:07.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.383+0000 7fe09aa00640 1 --2- 172.21.15.133:0/3432100190 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe08406b4d0 0x7fe08406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:07.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.383+0000 7fe09aa00640 1 -- 172.21.15.133:0/3432100190 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe09c0898e0 con 0x7fe0a4147bb0 2024-03-07T06:53:07.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.383+0000 7fe0a2a00640 1 --2- 172.21.15.133:0/3432100190 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe08406b4d0 0x7fe08406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:07.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.384+0000 7fe0a2a00640 1 --2- 172.21.15.133:0/3432100190 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe08406b4d0 0x7fe08406d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe094036000 tx=0x7fe094008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:07.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.387+0000 7fe09aa00640 1 -- 172.21.15.133:0/3432100190 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe09c058750 con 0x7fe0a4147bb0 2024-03-07T06:53:07.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.688+0000 7fe0a9600640 1 -- 172.21.15.133:0/3432100190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0a40726f0 con 0x7fe0a4147bb0 2024-03-07T06:53:07.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.690+0000 7fe09aa00640 1 -- 172.21.15.133:0/3432100190 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16717 (secure 0 0 0) 0x7fe09c058050 con 0x7fe0a4147bb0 2024-03-07T06:53:07.690 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:07.690 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":35,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:52:36.759644+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:52:33.007428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-07T06:52:34.201590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:53:07.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.693+0000 7fe0a9600640 1 -- 172.21.15.133:0/3432100190 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe08406b4d0 msgr2=0x7fe08406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:07.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.693+0000 7fe0a9600640 1 --2- 172.21.15.133:0/3432100190 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe08406b4d0 0x7fe08406d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe094036000 tx=0x7fe094008030 comp rx=0 tx=0).stop 2024-03-07T06:53:07.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.693+0000 7fe0a9600640 1 -- 172.21.15.133:0/3432100190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0a4147bb0 msgr2=0x7fe0a415ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:07.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.693+0000 7fe0a9600640 1 --2- 172.21.15.133:0/3432100190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0a4147bb0 0x7fe0a415ad60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe09c00c780 tx=0x7fe09c00cc40 comp rx=0 tx=0).stop 2024-03-07T06:53:07.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.693+0000 7fe0a9600640 1 -- 172.21.15.133:0/3432100190 shutdown_connections 2024-03-07T06:53:07.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.693+0000 7fe0a9600640 1 --2- 172.21.15.133:0/3432100190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0a4147bb0 0x7fe0a415ad60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.693+0000 7fe0a9600640 1 --2- 172.21.15.133:0/3432100190 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe08406b4d0 0x7fe08406d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.693+0000 7fe0a9600640 1 --2- 172.21.15.133:0/3432100190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe0a40b5090 0x7fe0a415a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.693+0000 7fe0a9600640 1 --2- 172.21.15.133:0/3432100190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe0a4072910 0x7fe0a40b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.693+0000 7fe0a9600640 1 -- 172.21.15.133:0/3432100190 >> 172.21.15.133:0/3432100190 conn(0x7fe0a406b480 msgr2=0x7fe0a40d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.694+0000 7fe0a9600640 1 -- 172.21.15.133:0/3432100190 shutdown_connections 2024-03-07T06:53:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.694+0000 7fe0a9600640 1 -- 172.21.15.133:0/3432100190 wait complete. 2024-03-07T06:53:07.710 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:52:33.007428+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:52:34.201590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:53:07.710 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:53:07.925 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.927+0000 7f3e1f400640 1 Processor -- start 2024-03-07T06:53:07.926 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.927+0000 7f3e1f400640 1 -- start start 2024-03-07T06:53:07.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.930+0000 7f3e1f400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e18162350 0x7f3e180719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:07.928 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.930+0000 7f3e1f400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3e1806e680 0x7f3e180cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:07.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.930+0000 7f3e1f400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e18072310 con 0x7f3e180b4270 2024-03-07T06:53:07.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.930+0000 7f3e1f400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e180725f0 con 0x7f3e1806e680 2024-03-07T06:53:07.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.930+0000 7f3e1f400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180728d0 con 0x7f3e18162350 2024-03-07T06:53:07.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.930+0000 7f3e1ca00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e18162350 0x7f3e180719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:07.929 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.930+0000 7f3e1ca00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e18162350 0x7f3e180719a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:60398/0 (socket says 172.21.15.133:60398) 2024-03-07T06:53:07.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.930+0000 7f3e1ca00640 1 -- 172.21.15.133:0/439358794 learned_addr learned my addr 172.21.15.133:0/439358794 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:07.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.931+0000 7f3e1d400640 1 --2- 172.21.15.133:0/439358794 >> v2:172.21.15.161:3300/0 conn(0x7f3e1806e680 0x7f3e180cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:07.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.931+0000 7f3e1ca00640 1 -- 172.21.15.133:0/439358794 >> v2:172.21.15.161:3300/0 conn(0x7f3e1806e680 msgr2=0x7f3e180cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:07.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.931+0000 7f3e1ca00640 1 --2- 172.21.15.133:0/439358794 >> v2:172.21.15.161:3300/0 conn(0x7f3e1806e680 0x7f3e180cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.931+0000 7f3e1ca00640 1 -- 172.21.15.133:0/439358794 >> v1:172.21.15.161:6789/0 conn(0x7f3e180b4270 legacy=0x7f3e1806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:07.930 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.931+0000 7f3e1ca00640 1 -- 172.21.15.133:0/439358794 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e18072bb0 con 0x7f3e18162350 2024-03-07T06:53:07.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.931+0000 7f3e1d400640 1 --2- 172.21.15.133:0/439358794 >> v2:172.21.15.161:3300/0 conn(0x7f3e1806e680 0x7f3e180cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-07T06:53:07.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.933+0000 7f3e1ca00640 1 --2- 172.21.15.133:0/439358794 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e18162350 0x7f3e180719a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009870 tx=0x7f3e0c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22220073c40fe232 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:07.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.933+0000 7f3e17400640 1 -- 172.21.15.133:0/439358794 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c034070 con 0x7f3e18162350 2024-03-07T06:53:07.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.933+0000 7f3e17400640 1 -- 172.21.15.133:0/439358794 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c030840 con 0x7f3e18162350 2024-03-07T06:53:07.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.933+0000 7f3e17400640 1 -- 172.21.15.133:0/439358794 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c030c10 con 0x7f3e18162350 2024-03-07T06:53:07.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.934+0000 7f3e1f400640 1 -- 172.21.15.133:0/439358794 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e18162350 msgr2=0x7f3e180719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:07.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.934+0000 7f3e1f400640 1 --2- 172.21.15.133:0/439358794 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e18162350 0x7f3e180719a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009870 tx=0x7f3e0c02b980 comp rx=0 tx=0).stop 2024-03-07T06:53:07.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.934+0000 7f3e1f400640 1 -- 172.21.15.133:0/439358794 shutdown_connections 2024-03-07T06:53:07.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.934+0000 7f3e1f400640 1 --2- 172.21.15.133:0/439358794 >> v2:172.21.15.161:3300/0 conn(0x7f3e1806e680 0x7f3e180cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.934+0000 7f3e1f400640 1 --2- 172.21.15.133:0/439358794 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e18162350 0x7f3e180719a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.934+0000 7f3e1f400640 1 -- 172.21.15.133:0/439358794 >> 172.21.15.133:0/439358794 conn(0x7f3e180b1d30 msgr2=0x7f3e180b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:07.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.934+0000 7f3e1f400640 1 -- 172.21.15.133:0/439358794 shutdown_connections 2024-03-07T06:53:07.933 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.934+0000 7f3e1f400640 1 -- 172.21.15.133:0/439358794 wait complete. 2024-03-07T06:53:07.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.936+0000 7f3e1f400640 1 Processor -- start 2024-03-07T06:53:07.934 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.936+0000 7f3e1f400640 1 -- start start 2024-03-07T06:53:07.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.937+0000 7f3e1f400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e1806e680 0x7f3e180aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:07.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.937+0000 7f3e1f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e180b4270 0x7f3e181f83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:07.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.937+0000 7f3e1f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e18162350 0x7f3e181edea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:07.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.937+0000 7f3e1f400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e18072060 con 0x7f3e180b4270 2024-03-07T06:53:07.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.937+0000 7f3e1f400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e18071ee0 con 0x7f3e18162350 2024-03-07T06:53:07.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.937+0000 7f3e1f400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e18162730 con 0x7f3e1806e680 2024-03-07T06:53:07.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.937+0000 7f3e17e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e180b4270 0x7f3e181f83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:07.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.937+0000 7f3e17e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e180b4270 0x7f3e181f83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44000/0 (socket says 172.21.15.133:44000) 2024-03-07T06:53:07.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.937+0000 7f3e17e00640 1 -- 172.21.15.133:0/1263537699 learned_addr learned my addr 172.21.15.133:0/1263537699 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:07.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.937+0000 7f3e1d400640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e18162350 0x7f3e181edea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:07.937 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.937+0000 7f3e1ca00640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e1806e680 0x7f3e180aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:07.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.938+0000 7f3e17e00640 1 -- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e1806e680 msgr2=0x7f3e180aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:07.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.938+0000 7f3e17e00640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e1806e680 0x7f3e180aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.938+0000 7f3e17e00640 1 -- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e18162350 msgr2=0x7f3e181edea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:07.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.938+0000 7f3e17e00640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e18162350 0x7f3e181edea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:07.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.938+0000 7f3e17e00640 1 -- 172.21.15.133:0/1263537699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e18160be0 con 0x7f3e180b4270 2024-03-07T06:53:07.938 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.938+0000 7f3e1ca00640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e1806e680 0x7f3e180aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:53:07.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.938+0000 7f3e17e00640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e180b4270 0x7f3e181f83f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400c400 tx=0x7f3e0400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:07.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.938+0000 7f3e14c00640 1 -- 172.21.15.133:0/1263537699 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e040052d0 con 0x7f3e180b4270 2024-03-07T06:53:07.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.938+0000 7f3e14c00640 1 -- 172.21.15.133:0/1263537699 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e04005490 con 0x7f3e180b4270 2024-03-07T06:53:07.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.938+0000 7f3e14c00640 1 -- 172.21.15.133:0/1263537699 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e04007260 con 0x7f3e180b4270 2024-03-07T06:53:07.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.938+0000 7f3e1f400640 1 -- 172.21.15.133:0/1263537699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e18160ec0 con 0x7f3e180b4270 2024-03-07T06:53:07.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.938+0000 7f3e1f400640 1 -- 172.21.15.133:0/1263537699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e18162cd0 con 0x7f3e180b4270 2024-03-07T06:53:07.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.940+0000 7f3e1f400640 1 -- 172.21.15.133:0/1263537699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e18062c90 con 0x7f3e180b4270 2024-03-07T06:53:07.939 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.941+0000 7f3e14c00640 1 -- 172.21.15.133:0/1263537699 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3e04007400 con 0x7f3e180b4270 2024-03-07T06:53:07.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.941+0000 7f3e14c00640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3df806b4d0 0x7f3df806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:07.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.942+0000 7f3e14c00640 1 -- 172.21.15.133:0/1263537699 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e040897d0 con 0x7f3e180b4270 2024-03-07T06:53:07.940 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.942+0000 7f3e1ca00640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3df806b4d0 0x7f3df806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:07.942 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.943+0000 7f3e1ca00640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3df806b4d0 0x7f3df806d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c036000 tx=0x7f3e0c03b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:07.949 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:07.951+0000 7f3e14c00640 1 -- 172.21.15.133:0/1263537699 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3e04057ee0 con 0x7f3e180b4270 2024-03-07T06:53:08.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.145+0000 7f3e1f400640 1 -- 172.21.15.133:0/1263537699 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e180b3a60 con 0x7f3e180b4270 2024-03-07T06:53:08.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.146+0000 7f3e14c00640 1 -- 172.21.15.133:0/1263537699 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e0405bf80 con 0x7f3e180b4270 2024-03-07T06:53:08.145 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:53:08.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.149+0000 7f3e1f400640 1 -- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3df806b4d0 msgr2=0x7f3df806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:08.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.149+0000 7f3e1f400640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3df806b4d0 0x7f3df806d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c036000 tx=0x7f3e0c03b040 comp rx=0 tx=0).stop 2024-03-07T06:53:08.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.149+0000 7f3e1f400640 1 -- 172.21.15.133:0/1263537699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e180b4270 msgr2=0x7f3e181f83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:08.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.149+0000 7f3e1f400640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e180b4270 0x7f3e181f83f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400c400 tx=0x7f3e0400c8c0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.149+0000 7f3e1f400640 1 -- 172.21.15.133:0/1263537699 shutdown_connections 2024-03-07T06:53:08.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.149+0000 7f3e1f400640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3df806b4d0 0x7f3df806d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.149+0000 7f3e1f400640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3e18162350 0x7f3e181edea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.149+0000 7f3e1f400640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e180b4270 0x7f3e181f83f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.149+0000 7f3e1f400640 1 --2- 172.21.15.133:0/1263537699 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3e1806e680 0x7f3e180aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.149+0000 7f3e1f400640 1 -- 172.21.15.133:0/1263537699 >> 172.21.15.133:0/1263537699 conn(0x7f3e180b1d30 msgr2=0x7f3e180cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:08.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.149+0000 7f3e1f400640 1 -- 172.21.15.133:0/1263537699 shutdown_connections 2024-03-07T06:53:08.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.149+0000 7f3e1f400640 1 -- 172.21.15.133:0/1263537699 wait complete. 2024-03-07T06:53:08.165 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:53:08.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.367+0000 7f3ff5200640 1 Processor -- start 2024-03-07T06:53:08.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.367+0000 7f3ff5200640 1 -- start start 2024-03-07T06:53:08.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.369+0000 7f3ff5200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ff00b7200 0x7f3ff0071990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:08.367 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.369+0000 7f3ff5200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff0071ff0 con 0x7f3ff01625c0 2024-03-07T06:53:08.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.369+0000 7f3ff5200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff00d23b0 con 0x7f3ff015e350 2024-03-07T06:53:08.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.369+0000 7f3ff5200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00d2160 con 0x7f3ff00b7200 2024-03-07T06:53:08.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.369+0000 7f3fed400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f3ff01625c0 0x7f3ff0071280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:50188/0 (socket says 172.21.15.133:50188) 2024-03-07T06:53:08.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.369+0000 7f3fed400640 1 -- 172.21.15.133:0/1744331729 learned_addr learned my addr 172.21.15.133:0/1744331729 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:08.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.369+0000 7f3fee800640 1 --2- 172.21.15.133:0/1744331729 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ff00b7200 0x7f3ff0071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:08.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.370+0000 7f3feca00640 1 -- 172.21.15.133:0/1744331729 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173040727 0 0) 0x7f3ff0071ff0 con 0x7f3ff01625c0 2024-03-07T06:53:08.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.370+0000 7f3feca00640 1 -- 172.21.15.133:0/1744331729 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd4003520 con 0x7f3ff01625c0 2024-03-07T06:53:08.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.370+0000 7f3feca00640 1 -- 172.21.15.133:0/1744331729 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504460022 0 0) 0x7f3ff00d23b0 con 0x7f3ff015e350 2024-03-07T06:53:08.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.370+0000 7f3feca00640 1 -- 172.21.15.133:0/1744331729 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ff0071ff0 con 0x7f3ff015e350 2024-03-07T06:53:08.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.370+0000 7f3feca00640 1 -- 172.21.15.133:0/1744331729 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3484954531 0 0) 0x7f3fd4003520 con 0x7f3ff01625c0 2024-03-07T06:53:08.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.370+0000 7f3fee800640 1 -- 172.21.15.133:0/1744331729 >> v1:172.21.15.161:6789/0 conn(0x7f3ff015e350 legacy=0x7f3ff0161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:08.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.370+0000 7f3fee800640 1 -- 172.21.15.133:0/1744331729 >> v1:172.21.15.133:6789/0 conn(0x7f3ff01625c0 legacy=0x7f3ff0071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:08.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.370+0000 7f3fee800640 1 -- 172.21.15.133:0/1744331729 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff00d2690 con 0x7f3ff00b7200 2024-03-07T06:53:08.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.372+0000 7f3fee800640 1 --2- 172.21.15.133:0/1744331729 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ff00b7200 0x7f3ff0071990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009dc0 tx=0x7f3fe802bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=564f8b0da727bb2e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:08.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.373+0000 7f3feca00640 1 -- 172.21.15.133:0/1744331729 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8039070 con 0x7f3ff00b7200 2024-03-07T06:53:08.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.373+0000 7f3feca00640 1 -- 172.21.15.133:0/1744331729 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe802c910 con 0x7f3ff00b7200 2024-03-07T06:53:08.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.373+0000 7f3feca00640 1 -- 172.21.15.133:0/1744331729 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc001670 con 0x7f3ff00b7200 2024-03-07T06:53:08.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.373+0000 7f3ff5200640 1 -- 172.21.15.133:0/1744331729 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ff00b7200 msgr2=0x7f3ff0071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:08.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.373+0000 7f3ff5200640 1 --2- 172.21.15.133:0/1744331729 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ff00b7200 0x7f3ff0071990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009dc0 tx=0x7f3fe802bde0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.374+0000 7f3ff5200640 1 -- 172.21.15.133:0/1744331729 shutdown_connections 2024-03-07T06:53:08.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.374+0000 7f3ff5200640 1 --2- 172.21.15.133:0/1744331729 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ff00b7200 0x7f3ff0071990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.374+0000 7f3ff5200640 1 -- 172.21.15.133:0/1744331729 >> 172.21.15.133:0/1744331729 conn(0x7f3ff00d0e70 msgr2=0x7f3ff00d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:08.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.374+0000 7f3ff5200640 1 -- 172.21.15.133:0/1744331729 shutdown_connections 2024-03-07T06:53:08.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.374+0000 7f3ff5200640 1 -- 172.21.15.133:0/1744331729 wait complete. 2024-03-07T06:53:08.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.376+0000 7f3ff5200640 1 Processor -- start 2024-03-07T06:53:08.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.376+0000 7f3ff5200640 1 -- start start 2024-03-07T06:53:08.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.378+0000 7f3ff5200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ff00b7200 0x7f3ff015af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:08.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.378+0000 7f3ff5200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff015e350 0x7f3ff00c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:08.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.378+0000 7f3ff5200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ff01625c0 0x7f3ff0152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:08.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.378+0000 7f3ff5200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff01691e0 con 0x7f3ff01625c0 2024-03-07T06:53:08.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.378+0000 7f3ff5200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff015e730 con 0x7f3ff015e350 2024-03-07T06:53:08.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.378+0000 7f3ff5200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff0169360 con 0x7f3ff00b7200 2024-03-07T06:53:08.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.378+0000 7f3fee800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ff01625c0 0x7f3ff0152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:08.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.379+0000 7f3fee800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ff01625c0 0x7f3ff0152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44012/0 (socket says 172.21.15.133:44012) 2024-03-07T06:53:08.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.379+0000 7f3fee800640 1 -- 172.21.15.133:0/1229289341 learned_addr learned my addr 172.21.15.133:0/1229289341 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:08.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.379+0000 7f3fed400640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff015e350 0x7f3ff00c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:08.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.379+0000 7f3fede00640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ff00b7200 0x7f3ff015af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:08.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.379+0000 7f3fee800640 1 -- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ff00b7200 msgr2=0x7f3ff015af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:08.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.379+0000 7f3fee800640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ff00b7200 0x7f3ff015af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.379+0000 7f3fee800640 1 -- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff015e350 msgr2=0x7f3ff00c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:08.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.379+0000 7f3fee800640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff015e350 0x7f3ff00c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.379+0000 7f3fee800640 1 -- 172.21.15.133:0/1229289341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff00c9f40 con 0x7f3ff01625c0 2024-03-07T06:53:08.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.379+0000 7f3fed400640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff015e350 0x7f3ff00c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:53:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.379+0000 7f3fee800640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ff01625c0 0x7f3ff0152770 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009ee0 tx=0x7f3fe8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.380+0000 7f3fe6000640 1 -- 172.21.15.133:0/1229289341 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8039050 con 0x7f3ff01625c0 2024-03-07T06:53:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.380+0000 7f3fe6000640 1 -- 172.21.15.133:0/1229289341 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe8045070 con 0x7f3ff01625c0 2024-03-07T06:53:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.380+0000 7f3fe6000640 1 -- 172.21.15.133:0/1229289341 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8009450 con 0x7f3ff01625c0 2024-03-07T06:53:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.380+0000 7f3ff5200640 1 -- 172.21.15.133:0/1229289341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff00c96a0 con 0x7f3ff01625c0 2024-03-07T06:53:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.380+0000 7f3ff5200640 1 -- 172.21.15.133:0/1229289341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff00c9c60 con 0x7f3ff01625c0 2024-03-07T06:53:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.381+0000 7f3ff5200640 1 -- 172.21.15.133:0/1229289341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff0062c90 con 0x7f3ff01625c0 2024-03-07T06:53:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.382+0000 7f3fe6000640 1 -- 172.21.15.133:0/1229289341 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3fe8040690 con 0x7f3ff01625c0 2024-03-07T06:53:08.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.383+0000 7f3fe6000640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3fd006b4d0 0x7f3fd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:08.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.383+0000 7f3fe6000640 1 -- 172.21.15.133:0/1229289341 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fe80aca70 con 0x7f3ff01625c0 2024-03-07T06:53:08.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.384+0000 7f3fede00640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3fd006b4d0 0x7f3fd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:08.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.385+0000 7f3fede00640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3fd006b4d0 0x7f3fd006d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00011e0 tx=0x7f3fe0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:08.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.387+0000 7f3fe6000640 1 -- 172.21.15.133:0/1229289341 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3fe807bad0 con 0x7f3ff01625c0 2024-03-07T06:53:08.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.615+0000 7f3ff5200640 1 -- 172.21.15.133:0/1229289341 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ff00b5830 con 0x7f3ff01625c0 2024-03-07T06:53:08.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.616+0000 7f3fe6000640 1 -- 172.21.15.133:0/1229289341 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3fe807b3d0 con 0x7f3ff01625c0 2024-03-07T06:53:08.615 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:53:08.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.619+0000 7f3ff5200640 1 -- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3fd006b4d0 msgr2=0x7f3fd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:08.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.619+0000 7f3ff5200640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3fd006b4d0 0x7f3fd006d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00011e0 tx=0x7f3fe0009040 comp rx=0 tx=0).stop 2024-03-07T06:53:08.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.619+0000 7f3ff5200640 1 -- 172.21.15.133:0/1229289341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ff01625c0 msgr2=0x7f3ff0152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:08.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.619+0000 7f3ff5200640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ff01625c0 0x7f3ff0152770 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009ee0 tx=0x7f3fe8004a40 comp rx=0 tx=0).stop 2024-03-07T06:53:08.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.619+0000 7f3ff5200640 1 -- 172.21.15.133:0/1229289341 shutdown_connections 2024-03-07T06:53:08.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.619+0000 7f3ff5200640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ff01625c0 0x7f3ff0152770 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.619+0000 7f3ff5200640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3fd006b4d0 0x7f3fd006d990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.619+0000 7f3ff5200640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ff015e350 0x7f3ff00c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.618 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.619+0000 7f3ff5200640 1 --2- 172.21.15.133:0/1229289341 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ff00b7200 0x7f3ff015af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.619+0000 7f3ff5200640 1 -- 172.21.15.133:0/1229289341 >> 172.21.15.133:0/1229289341 conn(0x7f3ff00d0e70 msgr2=0x7f3ff0071280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.619+0000 7f3ff5200640 1 -- 172.21.15.133:0/1229289341 shutdown_connections 2024-03-07T06:53:08.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.619+0000 7f3ff5200640 1 -- 172.21.15.133:0/1229289341 wait complete. 2024-03-07T06:53:08.634 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:53:08.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.837+0000 7f9bfa200640 1 Processor -- start 2024-03-07T06:53:08.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.837+0000 7f9bfa200640 1 -- start start 2024-03-07T06:53:08.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.838+0000 7f9bfa200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9bf4159880 0x7f9bf40cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:08.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.839+0000 7f9bfa200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9bf40b4f10 0x7f9bf40b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:08.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.839+0000 7f9bfa200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf4160720 con 0x7f9bf40ccff0 2024-03-07T06:53:08.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.839+0000 7f9bfa200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf40b5880 con 0x7f9bf4159880 2024-03-07T06:53:08.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.839+0000 7f9bfa200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf40cde00 con 0x7f9bf40b4f10 2024-03-07T06:53:08.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.839+0000 7f9bf8e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9bf4159880 0x7f9bf40cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:08.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.839+0000 7f9bf8e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9bf4159880 0x7f9bf40cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44020/0 (socket says 172.21.15.133:44020) 2024-03-07T06:53:08.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.839+0000 7f9bf8e00640 1 -- 172.21.15.133:0/3489272665 learned_addr learned my addr 172.21.15.133:0/3489272665 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:08.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.839+0000 7f9bf9800640 1 --2- 172.21.15.133:0/3489272665 >> v2:172.21.15.161:3300/0 conn(0x7f9bf40b4f10 0x7f9bf40b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:08.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.839+0000 7f9bf8e00640 1 -- 172.21.15.133:0/3489272665 >> v2:172.21.15.161:3300/0 conn(0x7f9bf40b4f10 msgr2=0x7f9bf40b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:08.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.839+0000 7f9bf8e00640 1 --2- 172.21.15.133:0/3489272665 >> v2:172.21.15.161:3300/0 conn(0x7f9bf40b4f10 0x7f9bf40b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.839+0000 7f9bf8e00640 1 -- 172.21.15.133:0/3489272665 >> v1:172.21.15.133:6789/0 conn(0x7f9bf40ccff0 legacy=0x7f9bf4160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:08.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.840+0000 7f9bf8e00640 1 -- 172.21.15.133:0/3489272665 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf40cd560 con 0x7f9bf4159880 2024-03-07T06:53:08.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.841+0000 7f9bf8e00640 1 --2- 172.21.15.133:0/3489272665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bf4159880 0x7f9bf40cf2b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40cfb00 tx=0x7f9be802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f746400c6d325dc server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:08.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.841+0000 7f9bf3400640 1 -- 172.21.15.133:0/3489272665 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8034070 con 0x7f9bf4159880 2024-03-07T06:53:08.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.841+0000 7f9bf3400640 1 -- 172.21.15.133:0/3489272665 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be8030860 con 0x7f9bf4159880 2024-03-07T06:53:08.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.841+0000 7f9bf3400640 1 -- 172.21.15.133:0/3489272665 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8030c30 con 0x7f9bf4159880 2024-03-07T06:53:08.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.841+0000 7f9bfa200640 1 -- 172.21.15.133:0/3489272665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bf4159880 msgr2=0x7f9bf40cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:08.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.841+0000 7f9bfa200640 1 --2- 172.21.15.133:0/3489272665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bf4159880 0x7f9bf40cf2b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40cfb00 tx=0x7f9be802b9a0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.842+0000 7f9bfa200640 1 -- 172.21.15.133:0/3489272665 shutdown_connections 2024-03-07T06:53:08.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.842+0000 7f9bfa200640 1 --2- 172.21.15.133:0/3489272665 >> v2:172.21.15.161:3300/0 conn(0x7f9bf40b4f10 0x7f9bf40b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.842+0000 7f9bfa200640 1 --2- 172.21.15.133:0/3489272665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bf4159880 0x7f9bf40cf2b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.842+0000 7f9bfa200640 1 -- 172.21.15.133:0/3489272665 >> 172.21.15.133:0/3489272665 conn(0x7f9bf4071860 msgr2=0x7f9bf4071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:08.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.842+0000 7f9bfa200640 1 -- 172.21.15.133:0/3489272665 shutdown_connections 2024-03-07T06:53:08.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.842+0000 7f9bfa200640 1 -- 172.21.15.133:0/3489272665 wait complete. 2024-03-07T06:53:08.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.844+0000 7f9bfa200640 1 Processor -- start 2024-03-07T06:53:08.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.845+0000 7f9bfa200640 1 -- start start 2024-03-07T06:53:08.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.846+0000 7f9bfa200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bf40b4f10 0x7f9bf4145490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:08.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.846+0000 7f9bfa200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bf40ccff0 0x7f9bf414df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:08.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.846+0000 7f9bfa200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9bf4159880 0x7f9bf414fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:08.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.846+0000 7f9bfa200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf415faf0 con 0x7f9bf40ccff0 2024-03-07T06:53:08.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.846+0000 7f9bfa200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf40cdb20 con 0x7f9bf40b4f10 2024-03-07T06:53:08.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.846+0000 7f9bfa200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf4159c60 con 0x7f9bf4159880 2024-03-07T06:53:08.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.846+0000 7f9bf3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bf40ccff0 0x7f9bf414df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:08.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.846+0000 7f9bf3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bf40ccff0 0x7f9bf414df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44036/0 (socket says 172.21.15.133:44036) 2024-03-07T06:53:08.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.846+0000 7f9bf3e00640 1 -- 172.21.15.133:0/2531407453 learned_addr learned my addr 172.21.15.133:0/2531407453 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:08.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.847+0000 7f9bf8e00640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bf40b4f10 0x7f9bf4145490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:08.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.847+0000 7f9bf9800640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9bf4159880 0x7f9bf414fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:08.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.847+0000 7f9bf3e00640 1 -- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9bf4159880 msgr2=0x7f9bf414fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:08.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.847+0000 7f9bf3e00640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9bf4159880 0x7f9bf414fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.847+0000 7f9bf3e00640 1 -- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bf40b4f10 msgr2=0x7f9bf4145490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:08.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.847+0000 7f9bf3e00640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bf40b4f10 0x7f9bf4145490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:08.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.847+0000 7f9bf3e00640 1 -- 172.21.15.133:0/2531407453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf414ea10 con 0x7f9bf40ccff0 2024-03-07T06:53:08.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.847+0000 7f9bf8e00640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bf40b4f10 0x7f9bf4145490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:08.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.847+0000 7f9bf3e00640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bf40ccff0 0x7f9bf414df50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c3f0 tx=0x7f9be000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:08.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.847+0000 7f9bf0c00640 1 -- 172.21.15.133:0/2531407453 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be00052d0 con 0x7f9bf40ccff0 2024-03-07T06:53:08.847 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.848+0000 7f9bf0c00640 1 -- 172.21.15.133:0/2531407453 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be0005490 con 0x7f9bf40ccff0 2024-03-07T06:53:08.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.848+0000 7f9bfa200640 1 -- 172.21.15.133:0/2531407453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf414ecf0 con 0x7f9bf40ccff0 2024-03-07T06:53:08.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.848+0000 7f9bf0c00640 1 -- 172.21.15.133:0/2531407453 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0007260 con 0x7f9bf40ccff0 2024-03-07T06:53:08.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.848+0000 7f9bfa200640 1 -- 172.21.15.133:0/2531407453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf41459d0 con 0x7f9bf40ccff0 2024-03-07T06:53:08.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.849+0000 7f9bfa200640 1 -- 172.21.15.133:0/2531407453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf414f210 con 0x7f9bf40ccff0 2024-03-07T06:53:08.848 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.850+0000 7f9bf0c00640 1 -- 172.21.15.133:0/2531407453 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9be0007400 con 0x7f9bf40ccff0 2024-03-07T06:53:08.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.851+0000 7f9bf0c00640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9bd406b4d0 0x7f9bd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:08.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.851+0000 7f9bf0c00640 1 -- 172.21.15.133:0/2531407453 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9be00896f0 con 0x7f9bf40ccff0 2024-03-07T06:53:08.850 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.851+0000 7f9bf8e00640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9bd406b4d0 0x7f9bd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:08.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.852+0000 7f9bf8e00640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9bd406b4d0 0x7f9bd406d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9be8036000 tx=0x7f9be803b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:08.854 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:08.855+0000 7f9bf0c00640 1 -- 172.21.15.133:0/2531407453 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9bf414f210 con 0x7f9bf40ccff0 2024-03-07T06:53:09.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.048+0000 7f9bfa200640 1 -- 172.21.15.133:0/2531407453 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9bf4062c90 con 0x7f9bf40ccff0 2024-03-07T06:53:09.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.048+0000 7f9bf0c00640 1 -- 172.21.15.133:0/2531407453 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9be0058560 con 0x7f9bf40ccff0 2024-03-07T06:53:09.047 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:53:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.051+0000 7f9bfa200640 1 -- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9bd406b4d0 msgr2=0x7f9bd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.051+0000 7f9bfa200640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9bd406b4d0 0x7f9bd406d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9be8036000 tx=0x7f9be803b040 comp rx=0 tx=0).stop 2024-03-07T06:53:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.051+0000 7f9bfa200640 1 -- 172.21.15.133:0/2531407453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bf40ccff0 msgr2=0x7f9bf414df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.051+0000 7f9bfa200640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bf40ccff0 0x7f9bf414df50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c3f0 tx=0x7f9be000c8b0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.051+0000 7f9bfa200640 1 -- 172.21.15.133:0/2531407453 shutdown_connections 2024-03-07T06:53:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.051+0000 7f9bfa200640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9bf4159880 0x7f9bf414fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.051+0000 7f9bfa200640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bf40ccff0 0x7f9bf414df50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.051+0000 7f9bfa200640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9bd406b4d0 0x7f9bd406d990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.051+0000 7f9bfa200640 1 --2- 172.21.15.133:0/2531407453 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9bf40b4f10 0x7f9bf4145490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.051+0000 7f9bfa200640 1 -- 172.21.15.133:0/2531407453 >> 172.21.15.133:0/2531407453 conn(0x7f9bf4071860 msgr2=0x7f9bf40af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.051+0000 7f9bfa200640 1 -- 172.21.15.133:0/2531407453 shutdown_connections 2024-03-07T06:53:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.051+0000 7f9bfa200640 1 -- 172.21.15.133:0/2531407453 wait complete. 2024-03-07T06:53:09.068 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:53:09.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.265+0000 7ff1f4800640 1 Processor -- start 2024-03-07T06:53:09.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.265+0000 7ff1f4800640 1 -- start start 2024-03-07T06:53:09.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.267+0000 7ff1f4800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff1ec15e050 0x7ff1ec0aef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:09.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.267+0000 7ff1f4800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1ec0ce4d0 0x7ff1ec0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:09.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.267+0000 7ff1f4800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec06f560 con 0x7ff1ec0b0a00 2024-03-07T06:53:09.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.267+0000 7ff1f4800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec072620 con 0x7ff1ec15e050 2024-03-07T06:53:09.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.267+0000 7ff1f4800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec06e7a0 con 0x7ff1ec0ce4d0 2024-03-07T06:53:09.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.268+0000 7ff1f1e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff1ec15e050 0x7ff1ec0aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:09.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.268+0000 7ff1f1e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff1ec15e050 0x7ff1ec0aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44048/0 (socket says 172.21.15.133:44048) 2024-03-07T06:53:09.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.268+0000 7ff1f1e00640 1 -- 172.21.15.133:0/648116862 learned_addr learned my addr 172.21.15.133:0/648116862 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:09.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.268+0000 7ff1f1e00640 1 -- 172.21.15.133:0/648116862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1ec0ce4d0 msgr2=0x7ff1ec0b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:09.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.268+0000 7ff1f1400640 1 --2- 172.21.15.133:0/648116862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1ec0ce4d0 0x7ff1ec0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:09.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.268+0000 7ff1f1e00640 1 --2- 172.21.15.133:0/648116862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1ec0ce4d0 0x7ff1ec0b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.268+0000 7ff1f1e00640 1 -- 172.21.15.133:0/648116862 >> v1:172.21.15.161:6789/0 conn(0x7ff1ec0b0a00 legacy=0x7ff1ec06d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:09.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.268+0000 7ff1f1e00640 1 -- 172.21.15.133:0/648116862 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec06ea80 con 0x7ff1ec15e050 2024-03-07T06:53:09.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.269+0000 7ff1f1e00640 1 --2- 172.21.15.133:0/648116862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ec15e050 0x7ff1ec0aef50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000aa10 tx=0x7ff1e002cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc67e3bd9f9f1f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:09.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.269+0000 7ff1f0a00640 1 -- 172.21.15.133:0/648116862 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e003a070 con 0x7ff1ec15e050 2024-03-07T06:53:09.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.270+0000 7ff1f0a00640 1 -- 172.21.15.133:0/648116862 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e002dae0 con 0x7ff1ec15e050 2024-03-07T06:53:09.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.270+0000 7ff1f0a00640 1 -- 172.21.15.133:0/648116862 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000b040 con 0x7ff1ec15e050 2024-03-07T06:53:09.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.270+0000 7ff1f4800640 1 -- 172.21.15.133:0/648116862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ec15e050 msgr2=0x7ff1ec0aef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:09.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.270+0000 7ff1f4800640 1 --2- 172.21.15.133:0/648116862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ec15e050 0x7ff1ec0aef50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000aa10 tx=0x7ff1e002cd30 comp rx=0 tx=0).stop 2024-03-07T06:53:09.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.270+0000 7ff1f4800640 1 -- 172.21.15.133:0/648116862 shutdown_connections 2024-03-07T06:53:09.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.270+0000 7ff1f4800640 1 --2- 172.21.15.133:0/648116862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1ec0ce4d0 0x7ff1ec0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.270+0000 7ff1f4800640 1 --2- 172.21.15.133:0/648116862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ec15e050 0x7ff1ec0aef50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.270+0000 7ff1f4800640 1 -- 172.21.15.133:0/648116862 >> 172.21.15.133:0/648116862 conn(0x7ff1ec0b3120 msgr2=0x7ff1ec0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:09.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.271+0000 7ff1f4800640 1 -- 172.21.15.133:0/648116862 shutdown_connections 2024-03-07T06:53:09.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.271+0000 7ff1f4800640 1 -- 172.21.15.133:0/648116862 wait complete. 2024-03-07T06:53:09.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.272+0000 7ff1f4800640 1 Processor -- start 2024-03-07T06:53:09.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.272+0000 7ff1f4800640 1 -- start start 2024-03-07T06:53:09.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.273+0000 7ff1f4800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1ec0b0a00 0x7ff1ec15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:09.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.273+0000 7ff1f4800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ec0ce4d0 0x7ff1ec1447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:09.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.273+0000 7ff1f4800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1ec15e050 0x7ff1ec0afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:09.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.273+0000 7ff1f4800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec0ce930 con 0x7ff1ec0ce4d0 2024-03-07T06:53:09.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.273+0000 7ff1f4800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec06f040 con 0x7ff1ec0b0a00 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.273+0000 7ff1f4800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec159d10 con 0x7ff1ec15e050 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.274+0000 7ff1f1400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ec0ce4d0 0x7ff1ec1447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.274+0000 7ff1f1400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ec0ce4d0 0x7ff1ec1447d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44052/0 (socket says 172.21.15.133:44052) 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.274+0000 7ff1f1400640 1 -- 172.21.15.133:0/3764687822 learned_addr learned my addr 172.21.15.133:0/3764687822 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.274+0000 7ff1f1e00640 1 --2- 172.21.15.133:0/3764687822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1ec0b0a00 0x7ff1ec15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.274+0000 7ff1f2800640 1 --2- 172.21.15.133:0/3764687822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1ec15e050 0x7ff1ec0afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.274+0000 7ff1f1400640 1 -- 172.21.15.133:0/3764687822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1ec15e050 msgr2=0x7ff1ec0afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.274+0000 7ff1f1400640 1 --2- 172.21.15.133:0/3764687822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1ec15e050 0x7ff1ec0afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.274+0000 7ff1f1400640 1 -- 172.21.15.133:0/3764687822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1ec0b0a00 msgr2=0x7ff1ec15d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.274+0000 7ff1f1400640 1 --2- 172.21.15.133:0/3764687822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1ec0b0a00 0x7ff1ec15d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.274+0000 7ff1f1400640 1 -- 172.21.15.133:0/3764687822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec159a30 con 0x7ff1ec0ce4d0 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.274+0000 7ff1f1400640 1 --2- 172.21.15.133:0/3764687822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ec0ce4d0 0x7ff1ec1447d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00c7f0 tx=0x7ff1dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.275+0000 7ff1e6000640 1 -- 172.21.15.133:0/3764687822 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc015070 con 0x7ff1ec0ce4d0 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.275+0000 7ff1e6000640 1 -- 172.21.15.133:0/3764687822 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc004240 con 0x7ff1ec0ce4d0 2024-03-07T06:53:09.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.275+0000 7ff1f4800640 1 -- 172.21.15.133:0/3764687822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec09cfe0 con 0x7ff1ec0ce4d0 2024-03-07T06:53:09.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.275+0000 7ff1e6000640 1 -- 172.21.15.133:0/3764687822 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc007620 con 0x7ff1ec0ce4d0 2024-03-07T06:53:09.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.275+0000 7ff1f4800640 1 -- 172.21.15.133:0/3764687822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec09d5a0 con 0x7ff1ec0ce4d0 2024-03-07T06:53:09.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.276+0000 7ff1f4800640 1 -- 172.21.15.133:0/3764687822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec09d7a0 con 0x7ff1ec0ce4d0 2024-03-07T06:53:09.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.277+0000 7ff1e6000640 1 -- 172.21.15.133:0/3764687822 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff1dc0079d0 con 0x7ff1ec0ce4d0 2024-03-07T06:53:09.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.277+0000 7ff1e6000640 1 --2- 172.21.15.133:0/3764687822 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff1d006b4d0 0x7ff1d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:09.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.278+0000 7ff1e6000640 1 -- 172.21.15.133:0/3764687822 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1dc08a640 con 0x7ff1ec0ce4d0 2024-03-07T06:53:09.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.278+0000 7ff1f1e00640 1 --2- 172.21.15.133:0/3764687822 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff1d006b4d0 0x7ff1d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:09.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.279+0000 7ff1f1e00640 1 --2- 172.21.15.133:0/3764687822 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff1d006b4d0 0x7ff1d006d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0037000 tx=0x7ff1e0031000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:09.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.282+0000 7ff1e6000640 1 -- 172.21.15.133:0/3764687822 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff1ec09d7a0 con 0x7ff1ec0ce4d0 2024-03-07T06:53:09.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.606+0000 7ff1f4800640 1 -- 172.21.15.133:0/3764687822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1ec062c90 con 0x7ff1ec0ce4d0 2024-03-07T06:53:09.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.607+0000 7ff1e6000640 1 -- 172.21.15.133:0/3764687822 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4983 (secure 0 0 0) 0x7ff1dc05de20 con 0x7ff1ec0ce4d0 2024-03-07T06:53:09.605 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:09.605 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3269394125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3269394125},{"type":"v1","addr":"172.21.15.161:6835","nonce":3269394125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/72243172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":72243172},{"type":"v1","addr":"172.21.15.161:6837","nonce":72243172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3714291615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3714291615},{"type":"v1","addr":"172.21.15.133:6834","nonce":3714291615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:52:35.182567+0000","modified":"2024-03-07T06:52:38.268724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6835/4067860709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":4067860709},{"type":"v1","addr":"172.21.15.133:6835","nonce":4067860709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5100,"qdb_cluster":[5100]},"id":2}]} 2024-03-07T06:53:09.606 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 18 2024-03-07T06:53:09.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.610+0000 7ff1f4800640 1 -- 172.21.15.133:0/3764687822 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff1d006b4d0 msgr2=0x7ff1d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:09.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.610+0000 7ff1f4800640 1 --2- 172.21.15.133:0/3764687822 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff1d006b4d0 0x7ff1d006d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0037000 tx=0x7ff1e0031000 comp rx=0 tx=0).stop 2024-03-07T06:53:09.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.610+0000 7ff1f4800640 1 -- 172.21.15.133:0/3764687822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ec0ce4d0 msgr2=0x7ff1ec1447d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:09.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.610+0000 7ff1f4800640 1 --2- 172.21.15.133:0/3764687822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ec0ce4d0 0x7ff1ec1447d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00c7f0 tx=0x7ff1dc00ccb0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.610+0000 7ff1f4800640 1 -- 172.21.15.133:0/3764687822 shutdown_connections 2024-03-07T06:53:09.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.610+0000 7ff1f4800640 1 --2- 172.21.15.133:0/3764687822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff1ec15e050 0x7ff1ec0afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.610+0000 7ff1f4800640 1 --2- 172.21.15.133:0/3764687822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1ec0ce4d0 0x7ff1ec1447d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.610+0000 7ff1f4800640 1 --2- 172.21.15.133:0/3764687822 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff1d006b4d0 0x7ff1d006d990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.610+0000 7ff1f4800640 1 --2- 172.21.15.133:0/3764687822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff1ec0b0a00 0x7ff1ec15d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.610+0000 7ff1f4800640 1 -- 172.21.15.133:0/3764687822 >> 172.21.15.133:0/3764687822 conn(0x7ff1ec0b3120 msgr2=0x7ff1ec06e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:09.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.610+0000 7ff1f4800640 1 -- 172.21.15.133:0/3764687822 shutdown_connections 2024-03-07T06:53:09.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.610+0000 7ff1f4800640 1 -- 172.21.15.133:0/3764687822 wait complete. 2024-03-07T06:53:09.634 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:53:09.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.830+0000 7f8b52e00640 1 Processor -- start 2024-03-07T06:53:09.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.831+0000 7f8b52e00640 1 -- start start 2024-03-07T06:53:09.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.832+0000 7f8b52e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8b4c0b5dd0 0x7f8b4c144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:09.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.832+0000 7f8b52e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8b4c0b43f0 0x7f8b4c162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:09.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.832+0000 7f8b52e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4c0b4bd0 con 0x7f8b4c162ab0 2024-03-07T06:53:09.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.832+0000 7f8b52e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b4c141700 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.832+0000 7f8b52e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b4c161e00 con 0x7f8b4c0b5dd0 2024-03-07T06:53:09.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.832+0000 7f8b43400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8b4c0b43f0 0x7f8b4c162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:09.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.833+0000 7f8b43400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8b4c0b43f0 0x7f8b4c162570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44064/0 (socket says 172.21.15.133:44064) 2024-03-07T06:53:09.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.833+0000 7f8b43400640 1 -- 172.21.15.133:0/4286836168 learned_addr learned my addr 172.21.15.133:0/4286836168 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:09.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.833+0000 7f8b43400640 1 -- 172.21.15.133:0/4286836168 >> v2:172.21.15.161:3300/0 conn(0x7f8b4c0b5dd0 msgr2=0x7f8b4c144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:09.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.833+0000 7f8b43400640 1 --2- 172.21.15.133:0/4286836168 >> v2:172.21.15.161:3300/0 conn(0x7f8b4c0b5dd0 0x7f8b4c144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.833+0000 7f8b43400640 1 -- 172.21.15.133:0/4286836168 >> v1:172.21.15.161:6789/0 conn(0x7f8b4c162ab0 legacy=0x7f8b4c071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:09.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.833+0000 7f8b43400640 1 -- 172.21.15.133:0/4286836168 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c0b4eb0 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.834+0000 7f8b43400640 1 --2- 172.21.15.133:0/4286836168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c0b43f0 0x7f8b4c162570 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0b8740 tx=0x7f8b3402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b31fc8550ad67c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:09.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.834+0000 7f8b4b400640 1 -- 172.21.15.133:0/4286836168 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3402e070 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.834+0000 7f8b4b400640 1 -- 172.21.15.133:0/4286836168 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b34037070 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.834+0000 7f8b4b400640 1 -- 172.21.15.133:0/4286836168 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b34031a00 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.835+0000 7f8b52e00640 1 -- 172.21.15.133:0/4286836168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c0b43f0 msgr2=0x7f8b4c162570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:09.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.835+0000 7f8b52e00640 1 --2- 172.21.15.133:0/4286836168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c0b43f0 0x7f8b4c162570 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c0b8740 tx=0x7f8b3402b880 comp rx=0 tx=0).stop 2024-03-07T06:53:09.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.835+0000 7f8b52e00640 1 -- 172.21.15.133:0/4286836168 shutdown_connections 2024-03-07T06:53:09.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.835+0000 7f8b52e00640 1 --2- 172.21.15.133:0/4286836168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c0b43f0 0x7f8b4c162570 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.835+0000 7f8b52e00640 1 --2- 172.21.15.133:0/4286836168 >> v2:172.21.15.161:3300/0 conn(0x7f8b4c0b5dd0 0x7f8b4c144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.835+0000 7f8b52e00640 1 -- 172.21.15.133:0/4286836168 >> 172.21.15.133:0/4286836168 conn(0x7f8b4c0d0e70 msgr2=0x7f8b4c0d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:09.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.835+0000 7f8b52e00640 1 -- 172.21.15.133:0/4286836168 shutdown_connections 2024-03-07T06:53:09.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.835+0000 7f8b52e00640 1 -- 172.21.15.133:0/4286836168 wait complete. 2024-03-07T06:53:09.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.837+0000 7f8b52e00640 1 Processor -- start 2024-03-07T06:53:09.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.837+0000 7f8b52e00640 1 -- start start 2024-03-07T06:53:09.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.838+0000 7f8b52e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c0b43f0 0x7f8b4c136990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:09.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.838+0000 7f8b52e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b4c0b5dd0 0x7f8b4c071610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:09.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.838+0000 7f8b52e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b4c158090 0x7f8b4c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:09.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.838+0000 7f8b52e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0b61b0 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.838+0000 7f8b52e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c162050 con 0x7f8b4c0b5dd0 2024-03-07T06:53:09.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.838+0000 7f8b52e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0b48a0 con 0x7f8b4c158090 2024-03-07T06:53:09.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.838+0000 7f8b4be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c0b43f0 0x7f8b4c136990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:09.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.838+0000 7f8b4be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c0b43f0 0x7f8b4c136990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44066/0 (socket says 172.21.15.133:44066) 2024-03-07T06:53:09.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.838+0000 7f8b4be00640 1 -- 172.21.15.133:0/1877156292 learned_addr learned my addr 172.21.15.133:0/1877156292 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:09.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.839+0000 7f8b43400640 1 --2- 172.21.15.133:0/1877156292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b4c0b5dd0 0x7f8b4c071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:09.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.839+0000 7f8b4be00640 1 -- 172.21.15.133:0/1877156292 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b4c158090 msgr2=0x7f8b4c071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:09.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.839+0000 7f8b4be00640 1 --2- 172.21.15.133:0/1877156292 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b4c158090 0x7f8b4c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.839+0000 7f8b4be00640 1 -- 172.21.15.133:0/1877156292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b4c0b5dd0 msgr2=0x7f8b4c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:09.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.839+0000 7f8b4be00640 1 --2- 172.21.15.133:0/1877156292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b4c0b5dd0 0x7f8b4c071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:09.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.839+0000 7f8b4be00640 1 -- 172.21.15.133:0/1877156292 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4c1371e0 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.839+0000 7f8b43400640 1 --2- 172.21.15.133:0/1877156292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b4c0b5dd0 0x7f8b4c071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:09.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.839+0000 7f8b4be00640 1 --2- 172.21.15.133:0/1877156292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c0b43f0 0x7f8b4c136990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00ba80 tx=0x7f8b3c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:09.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.839+0000 7f8b48c00640 1 -- 172.21.15.133:0/1877156292 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c019070 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.839+0000 7f8b48c00640 1 -- 172.21.15.133:0/1877156292 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3c00cc20 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.839+0000 7f8b52e00640 1 -- 172.21.15.133:0/1877156292 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4c138c20 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.839+0000 7f8b48c00640 1 -- 172.21.15.133:0/1877156292 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c014440 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.839+0000 7f8b52e00640 1 -- 172.21.15.133:0/1877156292 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c1391e0 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.840+0000 7f8b52e00640 1 -- 172.21.15.133:0/1877156292 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4c062c90 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.841+0000 7f8b48c00640 1 -- 172.21.15.133:0/1877156292 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8b3c01c070 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.842+0000 7f8b48c00640 1 --2- 172.21.15.133:0/1877156292 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8b2806b4d0 0x7f8b2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:09.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.842+0000 7f8b48c00640 1 -- 172.21.15.133:0/1877156292 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b3c089eb0 con 0x7f8b4c0b43f0 2024-03-07T06:53:09.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.843+0000 7f8b43400640 1 --2- 172.21.15.133:0/1877156292 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8b2806b4d0 0x7f8b2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:09.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.844+0000 7f8b43400640 1 --2- 172.21.15.133:0/1877156292 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8b2806b4d0 0x7f8b2806d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8b34036000 tx=0x7f8b34008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:09.851 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:09.852+0000 7f8b48c00640 1 -- 172.21.15.133:0/1877156292 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8b3c0586d0 con 0x7f8b4c0b43f0 2024-03-07T06:53:10.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.071+0000 7f8b52e00640 1 -- 172.21.15.133:0/1877156292 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b4c0b8a10 con 0x7f8b4c0b43f0 2024-03-07T06:53:10.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.074+0000 7f8b48c00640 1 -- 172.21.15.133:0/1877156292 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16717 (secure 0 0 0) 0x7f8b3c05c770 con 0x7f8b4c0b43f0 2024-03-07T06:53:10.073 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:10.073 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":35,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:52:36.759644+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:52:33.007428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-07T06:52:34.201590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:53:10.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.076+0000 7f8b52e00640 1 -- 172.21.15.133:0/1877156292 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8b2806b4d0 msgr2=0x7f8b2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:10.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.076+0000 7f8b52e00640 1 --2- 172.21.15.133:0/1877156292 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8b2806b4d0 0x7f8b2806d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8b34036000 tx=0x7f8b34008030 comp rx=0 tx=0).stop 2024-03-07T06:53:10.075 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.076+0000 7f8b52e00640 1 -- 172.21.15.133:0/1877156292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c0b43f0 msgr2=0x7f8b4c136990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:10.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.076+0000 7f8b52e00640 1 --2- 172.21.15.133:0/1877156292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c0b43f0 0x7f8b4c136990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00ba80 tx=0x7f8b3c00bf40 comp rx=0 tx=0).stop 2024-03-07T06:53:10.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.076+0000 7f8b52e00640 1 -- 172.21.15.133:0/1877156292 shutdown_connections 2024-03-07T06:53:10.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.076+0000 7f8b52e00640 1 --2- 172.21.15.133:0/1877156292 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b4c158090 0x7f8b4c071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.076+0000 7f8b52e00640 1 --2- 172.21.15.133:0/1877156292 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8b2806b4d0 0x7f8b2806d990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.076+0000 7f8b52e00640 1 --2- 172.21.15.133:0/1877156292 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b4c0b5dd0 0x7f8b4c071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.076+0000 7f8b52e00640 1 --2- 172.21.15.133:0/1877156292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b4c0b43f0 0x7f8b4c136990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.077+0000 7f8b52e00640 1 -- 172.21.15.133:0/1877156292 >> 172.21.15.133:0/1877156292 conn(0x7f8b4c0d0e70 msgr2=0x7f8b4c0b7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:10.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.077+0000 7f8b52e00640 1 -- 172.21.15.133:0/1877156292 shutdown_connections 2024-03-07T06:53:10.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.077+0000 7f8b52e00640 1 -- 172.21.15.133:0/1877156292 wait complete. 2024-03-07T06:53:10.096 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-07T06:53:10.096 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:53:10.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.292+0000 7f0775800640 1 Processor -- start 2024-03-07T06:53:10.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.292+0000 7f0775800640 1 -- start start 2024-03-07T06:53:10.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f0775800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f077015f170 0x7f077015e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:10.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f0775800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0770072cf0 0x7f0770071550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f0775800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07700d0990 0x7f07700d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f0775800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07700d1330 con 0x7f077015f170 2024-03-07T06:53:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f0775800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077006c3a0 con 0x7f0770072cf0 2024-03-07T06:53:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f0775800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0770071c50 con 0x7f07700d0990 2024-03-07T06:53:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f076ea00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f077015f170 0x7f077015e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f076ea00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f077015f170 0x7f077015e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44076/0 (socket says 172.21.15.133:44076) 2024-03-07T06:53:10.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f076ea00640 1 -- 172.21.15.133:0/3057164134 learned_addr learned my addr 172.21.15.133:0/3057164134 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:10.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f076f400640 1 --2- 172.21.15.133:0/3057164134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07700d0990 0x7f07700d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:10.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f076e000640 1 --2- 172.21.15.133:0/3057164134 >> v2:172.21.15.161:3300/0 conn(0x7f0770072cf0 0x7f0770071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:10.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f076ea00640 1 -- 172.21.15.133:0/3057164134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07700d0990 msgr2=0x7f07700d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:10.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f076ea00640 1 --2- 172.21.15.133:0/3057164134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07700d0990 0x7f07700d0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f076ea00640 1 -- 172.21.15.133:0/3057164134 >> v2:172.21.15.161:3300/0 conn(0x7f0770072cf0 msgr2=0x7f0770071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:10.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f076ea00640 1 --2- 172.21.15.133:0/3057164134 >> v2:172.21.15.161:3300/0 conn(0x7f0770072cf0 0x7f0770071550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.294+0000 7f076ea00640 1 -- 172.21.15.133:0/3057164134 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0770071f30 con 0x7f077015f170 2024-03-07T06:53:10.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.296+0000 7f076ea00640 1 --2- 172.21.15.133:0/3057164134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077015f170 0x7f077015e650 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0770161b70 tx=0x7f076002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c6eb86c0083e2b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:10.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.296+0000 7f076d600640 1 -- 172.21.15.133:0/3057164134 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760035020 con 0x7f077015f170 2024-03-07T06:53:10.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.296+0000 7f076d600640 1 -- 172.21.15.133:0/3057164134 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0760031640 con 0x7f077015f170 2024-03-07T06:53:10.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.296+0000 7f076d600640 1 -- 172.21.15.133:0/3057164134 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076002d040 con 0x7f077015f170 2024-03-07T06:53:10.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.296+0000 7f0775800640 1 -- 172.21.15.133:0/3057164134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077015f170 msgr2=0x7f077015e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:10.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.296+0000 7f0775800640 1 --2- 172.21.15.133:0/3057164134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077015f170 0x7f077015e650 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0770161b70 tx=0x7f076002b8f0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.297+0000 7f0775800640 1 -- 172.21.15.133:0/3057164134 shutdown_connections 2024-03-07T06:53:10.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.297+0000 7f0775800640 1 --2- 172.21.15.133:0/3057164134 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07700d0990 0x7f07700d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.297+0000 7f0775800640 1 --2- 172.21.15.133:0/3057164134 >> v2:172.21.15.161:3300/0 conn(0x7f0770072cf0 0x7f0770071550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.297+0000 7f0775800640 1 --2- 172.21.15.133:0/3057164134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f077015f170 0x7f077015e650 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.297+0000 7f0775800640 1 -- 172.21.15.133:0/3057164134 >> 172.21.15.133:0/3057164134 conn(0x7f077006b480 msgr2=0x7f077006b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:10.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.297+0000 7f0775800640 1 -- 172.21.15.133:0/3057164134 shutdown_connections 2024-03-07T06:53:10.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.297+0000 7f0775800640 1 -- 172.21.15.133:0/3057164134 wait complete. 2024-03-07T06:53:10.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.298+0000 7f0775800640 1 Processor -- start 2024-03-07T06:53:10.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.299+0000 7f0775800640 1 -- start start 2024-03-07T06:53:10.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.299+0000 7f0775800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0770072cf0 0x7f0770159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:10.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.299+0000 7f0775800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07700d0990 0x7f07700ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:10.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.299+0000 7f0775800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f077015f170 0x7f07701463e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:10.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.299+0000 7f0775800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07700727d0 con 0x7f0770072cf0 2024-03-07T06:53:10.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.299+0000 7f0775800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07700724f0 con 0x7f077015f170 2024-03-07T06:53:10.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.300+0000 7f0775800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07701470e0 con 0x7f07700d0990 2024-03-07T06:53:10.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.300+0000 7f076ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0770072cf0 0x7f0770159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:10.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.300+0000 7f076ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0770072cf0 0x7f0770159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44080/0 (socket says 172.21.15.133:44080) 2024-03-07T06:53:10.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.300+0000 7f076ea00640 1 -- 172.21.15.133:0/2019789555 learned_addr learned my addr 172.21.15.133:0/2019789555 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:10.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.300+0000 7f076e000640 1 --2- 172.21.15.133:0/2019789555 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07700d0990 0x7f07700ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:10.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.300+0000 7f076ea00640 1 -- 172.21.15.133:0/2019789555 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07700d0990 msgr2=0x7f07700ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:10.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.300+0000 7f076ea00640 1 --2- 172.21.15.133:0/2019789555 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07700d0990 0x7f07700ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.300+0000 7f076ea00640 1 -- 172.21.15.133:0/2019789555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f077015f170 msgr2=0x7f07701463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:10.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.300+0000 7f076ea00640 1 --2- 172.21.15.133:0/2019789555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f077015f170 0x7f07701463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.300+0000 7f076ea00640 1 -- 172.21.15.133:0/2019789555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07700cf080 con 0x7f0770072cf0 2024-03-07T06:53:10.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.300+0000 7f076ea00640 1 --2- 172.21.15.133:0/2019789555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0770072cf0 0x7f0770159690 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0760000c00 tx=0x7f07600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:10.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.301+0000 7f0766a00640 1 -- 172.21.15.133:0/2019789555 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760040070 con 0x7f0770072cf0 2024-03-07T06:53:10.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.301+0000 7f0766a00640 1 -- 172.21.15.133:0/2019789555 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0760039bf0 con 0x7f0770072cf0 2024-03-07T06:53:10.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.301+0000 7f0775800640 1 -- 172.21.15.133:0/2019789555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0770159d20 con 0x7f0770072cf0 2024-03-07T06:53:10.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.301+0000 7f0766a00640 1 -- 172.21.15.133:0/2019789555 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760035040 con 0x7f0770072cf0 2024-03-07T06:53:10.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.301+0000 7f0775800640 1 -- 172.21.15.133:0/2019789555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077015a2e0 con 0x7f0770072cf0 2024-03-07T06:53:10.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.302+0000 7f0775800640 1 -- 172.21.15.133:0/2019789555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0770062c90 con 0x7f0770072cf0 2024-03-07T06:53:10.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.303+0000 7f0766a00640 1 -- 172.21.15.133:0/2019789555 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f076004b060 con 0x7f0770072cf0 2024-03-07T06:53:10.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.304+0000 7f0766a00640 1 --2- 172.21.15.133:0/2019789555 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f075006b500 0x7f075006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:10.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.304+0000 7f0766a00640 1 -- 172.21.15.133:0/2019789555 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07600abcc0 con 0x7f0770072cf0 2024-03-07T06:53:10.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.305+0000 7f076e000640 1 --2- 172.21.15.133:0/2019789555 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f075006b500 0x7f075006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:10.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.306+0000 7f076e000640 1 --2- 172.21.15.133:0/2019789555 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f075006b500 0x7f075006d9c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f07700ce700 tx=0x7f075c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:10.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.309+0000 7f0766a00640 1 -- 172.21.15.133:0/2019789555 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f07600b2080 con 0x7f0770072cf0 2024-03-07T06:53:10.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.606+0000 7f0775800640 1 -- 172.21.15.133:0/2019789555 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0770071990 con 0x7f0770072cf0 2024-03-07T06:53:10.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.607+0000 7f0766a00640 1 -- 172.21.15.133:0/2019789555 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4983 (secure 0 0 0) 0x7f0760031610 con 0x7f0770072cf0 2024-03-07T06:53:10.606 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:10.606 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3269394125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3269394125},{"type":"v1","addr":"172.21.15.161:6835","nonce":3269394125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/72243172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":72243172},{"type":"v1","addr":"172.21.15.161:6837","nonce":72243172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3714291615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3714291615},{"type":"v1","addr":"172.21.15.133:6834","nonce":3714291615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:52:35.182567+0000","modified":"2024-03-07T06:52:38.268724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6835/4067860709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":4067860709},{"type":"v1","addr":"172.21.15.133:6835","nonce":4067860709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5100,"qdb_cluster":[5100]},"id":2}]} 2024-03-07T06:53:10.606 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 18 2024-03-07T06:53:10.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.609+0000 7f0775800640 1 -- 172.21.15.133:0/2019789555 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f075006b500 msgr2=0x7f075006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:10.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.609+0000 7f0775800640 1 --2- 172.21.15.133:0/2019789555 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f075006b500 0x7f075006d9c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f07700ce700 tx=0x7f075c008040 comp rx=0 tx=0).stop 2024-03-07T06:53:10.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.609+0000 7f0775800640 1 -- 172.21.15.133:0/2019789555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0770072cf0 msgr2=0x7f0770159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:10.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.609+0000 7f0775800640 1 --2- 172.21.15.133:0/2019789555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0770072cf0 0x7f0770159690 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0760000c00 tx=0x7f07600046c0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.608 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.609+0000 7f0775800640 1 -- 172.21.15.133:0/2019789555 shutdown_connections 2024-03-07T06:53:10.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.609+0000 7f0775800640 1 --2- 172.21.15.133:0/2019789555 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f075006b500 0x7f075006d9c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.609+0000 7f0775800640 1 --2- 172.21.15.133:0/2019789555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f077015f170 0x7f07701463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.609+0000 7f0775800640 1 --2- 172.21.15.133:0/2019789555 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07700d0990 0x7f07700ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.609+0000 7f0775800640 1 --2- 172.21.15.133:0/2019789555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0770072cf0 0x7f0770159690 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.609+0000 7f0775800640 1 -- 172.21.15.133:0/2019789555 >> 172.21.15.133:0/2019789555 conn(0x7f077006b480 msgr2=0x7f077015e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:10.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.609+0000 7f0775800640 1 -- 172.21.15.133:0/2019789555 shutdown_connections 2024-03-07T06:53:10.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.609+0000 7f0775800640 1 -- 172.21.15.133:0/2019789555 wait complete. 2024-03-07T06:53:10.626 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:53:10.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.859+0000 7f3102200640 1 Processor -- start 2024-03-07T06:53:10.858 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.860+0000 7f3102200640 1 -- start start 2024-03-07T06:53:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.861+0000 7f3102200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc0cd860 0x7f30fc06e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.861+0000 7f3102200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f30fc0b0160 0x7f30fc0a94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.861+0000 7f3102200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30fc161c40 con 0x7f30fc15f230 2024-03-07T06:53:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.861+0000 7f3102200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30fc0cdc40 con 0x7f30fc0b0160 2024-03-07T06:53:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.861+0000 7f3102200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc161f20 con 0x7f30fc0cd860 2024-03-07T06:53:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.862+0000 7f3101800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f30fc0b0160 0x7f30fc0a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.862+0000 7f30fbe00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f30fc15f230 0x7f30fc0afa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52578/0 (socket says 172.21.15.133:52578) 2024-03-07T06:53:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.862+0000 7f30fbe00640 1 -- 172.21.15.133:0/3090664504 learned_addr learned my addr 172.21.15.133:0/3090664504 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.862+0000 7f3100e00640 1 --2- 172.21.15.133:0/3090664504 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc0cd860 0x7f30fc06e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.862+0000 7f3101800640 1 -- 172.21.15.133:0/3090664504 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc0cd860 msgr2=0x7f30fc06e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.862+0000 7f3101800640 1 --2- 172.21.15.133:0/3090664504 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc0cd860 0x7f30fc06e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.862+0000 7f3101800640 1 -- 172.21.15.133:0/3090664504 >> v1:172.21.15.133:6789/0 conn(0x7f30fc15f230 legacy=0x7f30fc0afa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:10.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.862+0000 7f3101800640 1 -- 172.21.15.133:0/3090664504 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc162200 con 0x7f30fc0b0160 2024-03-07T06:53:10.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.862+0000 7f30fb400640 1 -- 172.21.15.133:0/3090664504 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562839862 0 0) 0x7f30fc161c40 con 0x7f30fc15f230 2024-03-07T06:53:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.863+0000 7f3101800640 1 --2- 172.21.15.133:0/3090664504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30fc0b0160 0x7f30fc0a94c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009d90 tx=0x7f30f402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39511c5112f06081 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.864+0000 7f30fb400640 1 -- 172.21.15.133:0/3090664504 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4039070 con 0x7f30fc0b0160 2024-03-07T06:53:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.864+0000 7f30fb400640 1 -- 172.21.15.133:0/3090664504 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f402cbc0 con 0x7f30fc0b0160 2024-03-07T06:53:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.864+0000 7f30fb400640 1 -- 172.21.15.133:0/3090664504 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4034440 con 0x7f30fc0b0160 2024-03-07T06:53:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.864+0000 7f3102200640 1 -- 172.21.15.133:0/3090664504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30fc0b0160 msgr2=0x7f30fc0a94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.864+0000 7f3102200640 1 --2- 172.21.15.133:0/3090664504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30fc0b0160 0x7f30fc0a94c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009d90 tx=0x7f30f402be10 comp rx=0 tx=0).stop 2024-03-07T06:53:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.864+0000 7f3102200640 1 -- 172.21.15.133:0/3090664504 shutdown_connections 2024-03-07T06:53:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.864+0000 7f3102200640 1 --2- 172.21.15.133:0/3090664504 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30fc0b0160 0x7f30fc0a94c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.864+0000 7f3102200640 1 --2- 172.21.15.133:0/3090664504 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc0cd860 0x7f30fc06e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.864+0000 7f3102200640 1 -- 172.21.15.133:0/3090664504 >> 172.21.15.133:0/3090664504 conn(0x7f30fc067090 msgr2=0x7f30fc06b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.864+0000 7f3102200640 1 -- 172.21.15.133:0/3090664504 shutdown_connections 2024-03-07T06:53:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.864+0000 7f3102200640 1 -- 172.21.15.133:0/3090664504 wait complete. 2024-03-07T06:53:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.866+0000 7f3102200640 1 Processor -- start 2024-03-07T06:53:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.866+0000 7f3102200640 1 -- start start 2024-03-07T06:53:10.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.867+0000 7f3102200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30fc0b0160 0x7f30fc158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:10.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.867+0000 7f3102200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0cd860 0x7f30fc1f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.867+0000 7f3102200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15f230 0x7f30fc0b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.867+0000 7f3102200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc06e730 con 0x7f30fc0b0160 2024-03-07T06:53:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.867+0000 7f3102200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc06e5b0 con 0x7f30fc0cd860 2024-03-07T06:53:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.867+0000 7f3102200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0a9a00 con 0x7f30fc15f230 2024-03-07T06:53:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.867+0000 7f3100e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30fc0b0160 0x7f30fc158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.867+0000 7f3100e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30fc0b0160 0x7f30fc158a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48948/0 (socket says 172.21.15.133:48948) 2024-03-07T06:53:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.867+0000 7f3100e00640 1 -- 172.21.15.133:0/685586887 learned_addr learned my addr 172.21.15.133:0/685586887 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.868+0000 7f3101800640 1 --2- 172.21.15.133:0/685586887 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15f230 0x7f30fc0b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.868+0000 7f3100e00640 1 -- 172.21.15.133:0/685586887 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15f230 msgr2=0x7f30fc0b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.868+0000 7f3100e00640 1 --2- 172.21.15.133:0/685586887 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15f230 0x7f30fc0b1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.868+0000 7f3100e00640 1 -- 172.21.15.133:0/685586887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0cd860 msgr2=0x7f30fc1f4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.868+0000 7f30fbe00640 1 --2- 172.21.15.133:0/685586887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0cd860 0x7f30fc1f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.868+0000 7f3100e00640 1 --2- 172.21.15.133:0/685586887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0cd860 0x7f30fc1f4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.868+0000 7f3100e00640 1 -- 172.21.15.133:0/685586887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc0d0f50 con 0x7f30fc0b0160 2024-03-07T06:53:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.868+0000 7f3100e00640 1 --2- 172.21.15.133:0/685586887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30fc0b0160 0x7f30fc158a80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f30f000d830 tx=0x7f30f000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.868+0000 7f30f8c00640 1 -- 172.21.15.133:0/685586887 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f000b6c0 con 0x7f30fc0b0160 2024-03-07T06:53:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.868+0000 7f30f8c00640 1 -- 172.21.15.133:0/685586887 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f0019070 con 0x7f30fc0b0160 2024-03-07T06:53:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.868+0000 7f30f8c00640 1 -- 172.21.15.133:0/685586887 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f000ba80 con 0x7f30fc0b0160 2024-03-07T06:53:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.869+0000 7f3102200640 1 -- 172.21.15.133:0/685586887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30fc0c2f00 con 0x7f30fc0b0160 2024-03-07T06:53:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.869+0000 7f3102200640 1 -- 172.21.15.133:0/685586887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc0c34c0 con 0x7f30fc0b0160 2024-03-07T06:53:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.870+0000 7f3102200640 1 -- 172.21.15.133:0/685586887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30fc062c90 con 0x7f30fc0b0160 2024-03-07T06:53:10.870 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.872+0000 7f30f8c00640 1 -- 172.21.15.133:0/685586887 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f30f001e460 con 0x7f30fc0b0160 2024-03-07T06:53:10.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.872+0000 7f30f8c00640 1 --2- 172.21.15.133:0/685586887 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f30dc06b4d0 0x7f30dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:10.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.873+0000 7f30f8c00640 1 -- 172.21.15.133:0/685586887 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30f008a860 con 0x7f30fc0b0160 2024-03-07T06:53:10.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.873+0000 7f30fbe00640 1 --2- 172.21.15.133:0/685586887 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f30dc06b4d0 0x7f30dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:10.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.874+0000 7f30fbe00640 1 --2- 172.21.15.133:0/685586887 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f30dc06b4d0 0x7f30dc06d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30fc1601d0 tx=0x7f30e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:10.876 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:10.877+0000 7f30f8c00640 1 -- 172.21.15.133:0/685586887 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f30f00596e0 con 0x7f30fc0b0160 2024-03-07T06:53:11.116 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.117+0000 7f3102200640 1 -- 172.21.15.133:0/685586887 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30fc0b3060 con 0x7f30fc0b0160 2024-03-07T06:53:11.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.118+0000 7f30f8c00640 1 -- 172.21.15.133:0/685586887 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4983 (secure 0 0 0) 0x7f30f008a070 con 0x7f30fc0b0160 2024-03-07T06:53:11.117 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:11.117 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3269394125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3269394125},{"type":"v1","addr":"172.21.15.161:6835","nonce":3269394125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/72243172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":72243172},{"type":"v1","addr":"172.21.15.161:6837","nonce":72243172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3714291615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3714291615},{"type":"v1","addr":"172.21.15.133:6834","nonce":3714291615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:52:35.182567+0000","modified":"2024-03-07T06:52:38.268724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5100},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5100":{"gid":5100,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.133:6835/4067860709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":4067860709},{"type":"v1","addr":"172.21.15.133:6835","nonce":4067860709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5100,"qdb_cluster":[5100]},"id":2}]} 2024-03-07T06:53:11.118 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 18 2024-03-07T06:53:11.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.122+0000 7f3102200640 1 -- 172.21.15.133:0/685586887 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f30dc06b4d0 msgr2=0x7f30dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:11.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.122+0000 7f3102200640 1 --2- 172.21.15.133:0/685586887 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f30dc06b4d0 0x7f30dc06d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30fc1601d0 tx=0x7f30e8009040 comp rx=0 tx=0).stop 2024-03-07T06:53:11.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.122+0000 7f3102200640 1 -- 172.21.15.133:0/685586887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30fc0b0160 msgr2=0x7f30fc158a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:11.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.122+0000 7f3102200640 1 --2- 172.21.15.133:0/685586887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30fc0b0160 0x7f30fc158a80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f30f000d830 tx=0x7f30f000dcf0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.122+0000 7f3102200640 1 -- 172.21.15.133:0/685586887 shutdown_connections 2024-03-07T06:53:11.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.122+0000 7f3102200640 1 --2- 172.21.15.133:0/685586887 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f30fc15f230 0x7f30fc0b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.122+0000 7f3102200640 1 --2- 172.21.15.133:0/685586887 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f30dc06b4d0 0x7f30dc06d990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.122+0000 7f3102200640 1 --2- 172.21.15.133:0/685586887 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f30fc0cd860 0x7f30fc1f4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.122+0000 7f3102200640 1 --2- 172.21.15.133:0/685586887 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30fc0b0160 0x7f30fc158a80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.122+0000 7f3102200640 1 -- 172.21.15.133:0/685586887 >> 172.21.15.133:0/685586887 conn(0x7f30fc067090 msgr2=0x7f30fc1608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:11.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.122+0000 7f3102200640 1 -- 172.21.15.133:0/685586887 shutdown_connections 2024-03-07T06:53:11.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.122+0000 7f3102200640 1 -- 172.21.15.133:0/685586887 wait complete. 2024-03-07T06:53:11.139 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:53:11.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.340+0000 7fa66be00640 1 Processor -- start 2024-03-07T06:53:11.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.340+0000 7fa66be00640 1 -- start start 2024-03-07T06:53:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.342+0000 7fa66be00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa6640ce5f0 0x7fa6640b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.342+0000 7fa66be00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6640af950 0x7fa664071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.342+0000 7fa66be00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa66406f140 con 0x7fa66415b1b0 2024-03-07T06:53:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.342+0000 7fa66be00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa664071610 con 0x7fa6640ce5f0 2024-03-07T06:53:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.342+0000 7fa66be00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66406f420 con 0x7fa6640af950 2024-03-07T06:53:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.342+0000 7fa668a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa6640ce5f0 0x7fa6640b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.342+0000 7fa669400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa66415b1b0 0x7fa6640cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52582/0 (socket says 172.21.15.133:52582) 2024-03-07T06:53:11.341 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.342+0000 7fa669400640 1 -- 172.21.15.133:0/3314262127 learned_addr learned my addr 172.21.15.133:0/3314262127 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:11.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.342+0000 7fa668a00640 1 -- 172.21.15.133:0/3314262127 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6640af950 msgr2=0x7fa664071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:11.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.343+0000 7fa668a00640 1 --2- 172.21.15.133:0/3314262127 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6640af950 0x7fa664071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.343+0000 7fa668a00640 1 -- 172.21.15.133:0/3314262127 >> v1:172.21.15.133:6789/0 conn(0x7fa66415b1b0 legacy=0x7fa6640cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:11.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.343+0000 7fa668a00640 1 -- 172.21.15.133:0/3314262127 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa664071860 con 0x7fa6640ce5f0 2024-03-07T06:53:11.342 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.343+0000 7fa663e00640 1 -- 172.21.15.133:0/3314262127 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146988320 0 0) 0x7fa66406f140 con 0x7fa66415b1b0 2024-03-07T06:53:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.344+0000 7fa668a00640 1 --2- 172.21.15.133:0/3314262127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6640ce5f0 0x7fa6640b3c10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa6500097a0 tx=0x7fa65002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425da35119a783a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.344+0000 7fa663e00640 1 -- 172.21.15.133:0/3314262127 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6500315a0 con 0x7fa6640ce5f0 2024-03-07T06:53:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.344+0000 7fa663e00640 1 -- 172.21.15.133:0/3314262127 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa650037070 con 0x7fa6640ce5f0 2024-03-07T06:53:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.344+0000 7fa663e00640 1 -- 172.21.15.133:0/3314262127 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650031960 con 0x7fa6640ce5f0 2024-03-07T06:53:11.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.345+0000 7fa66be00640 1 -- 172.21.15.133:0/3314262127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6640ce5f0 msgr2=0x7fa6640b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.345+0000 7fa66be00640 1 --2- 172.21.15.133:0/3314262127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6640ce5f0 0x7fa6640b3c10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa6500097a0 tx=0x7fa65002b840 comp rx=0 tx=0).stop 2024-03-07T06:53:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.345+0000 7fa66be00640 1 -- 172.21.15.133:0/3314262127 shutdown_connections 2024-03-07T06:53:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.345+0000 7fa66be00640 1 --2- 172.21.15.133:0/3314262127 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa6640af950 0x7fa664071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.345+0000 7fa66be00640 1 --2- 172.21.15.133:0/3314262127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6640ce5f0 0x7fa6640b3c10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.345+0000 7fa66be00640 1 -- 172.21.15.133:0/3314262127 >> 172.21.15.133:0/3314262127 conn(0x7fa6640b3120 msgr2=0x7fa6640b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.345+0000 7fa66be00640 1 -- 172.21.15.133:0/3314262127 shutdown_connections 2024-03-07T06:53:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.345+0000 7fa66be00640 1 -- 172.21.15.133:0/3314262127 wait complete. 2024-03-07T06:53:11.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.347+0000 7fa66be00640 1 Processor -- start 2024-03-07T06:53:11.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.347+0000 7fa66be00640 1 -- start start 2024-03-07T06:53:11.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.348+0000 7fa66be00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6640af950 0x7fa66414ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.348+0000 7fa66be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6640ce5f0 0x7fa66409b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.348+0000 7fa66be00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa66415b1b0 0x7fa66409b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.348+0000 7fa66be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66415b590 con 0x7fa6640ce5f0 2024-03-07T06:53:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.348+0000 7fa66be00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66406ee30 con 0x7fa6640af950 2024-03-07T06:53:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.348+0000 7fa66be00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66415b710 con 0x7fa66415b1b0 2024-03-07T06:53:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.349+0000 7fa668a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6640ce5f0 0x7fa66409b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.349+0000 7fa668a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6640ce5f0 0x7fa66409b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48964/0 (socket says 172.21.15.133:48964) 2024-03-07T06:53:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.349+0000 7fa668a00640 1 -- 172.21.15.133:0/4292760955 learned_addr learned my addr 172.21.15.133:0/4292760955 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.349+0000 7fa669400640 1 --2- 172.21.15.133:0/4292760955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6640af950 0x7fa66414ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.349+0000 7fa669e00640 1 --2- 172.21.15.133:0/4292760955 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa66415b1b0 0x7fa66409b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.349+0000 7fa668a00640 1 -- 172.21.15.133:0/4292760955 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa66415b1b0 msgr2=0x7fa66409b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.349+0000 7fa668a00640 1 --2- 172.21.15.133:0/4292760955 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa66415b1b0 0x7fa66409b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.349+0000 7fa668a00640 1 -- 172.21.15.133:0/4292760955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6640af950 msgr2=0x7fa66414ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:11.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.349+0000 7fa668a00640 1 --2- 172.21.15.133:0/4292760955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6640af950 0x7fa66414ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.349+0000 7fa668a00640 1 -- 172.21.15.133:0/4292760955 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66415e540 con 0x7fa6640ce5f0 2024-03-07T06:53:11.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.350+0000 7fa668a00640 1 --2- 172.21.15.133:0/4292760955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6640ce5f0 0x7fa66409b240 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa6500097a0 tx=0x7fa6500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:11.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.350+0000 7fa661600640 1 -- 172.21.15.133:0/4292760955 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650004030 con 0x7fa6640ce5f0 2024-03-07T06:53:11.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.350+0000 7fa661600640 1 -- 172.21.15.133:0/4292760955 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6500041f0 con 0x7fa6640ce5f0 2024-03-07T06:53:11.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.350+0000 7fa66be00640 1 -- 172.21.15.133:0/4292760955 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66409c4a0 con 0x7fa6640ce5f0 2024-03-07T06:53:11.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.350+0000 7fa661600640 1 -- 172.21.15.133:0/4292760955 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650032640 con 0x7fa6640ce5f0 2024-03-07T06:53:11.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.350+0000 7fa66be00640 1 -- 172.21.15.133:0/4292760955 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66409ac70 con 0x7fa6640ce5f0 2024-03-07T06:53:11.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.351+0000 7fa66be00640 1 -- 172.21.15.133:0/4292760955 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa664062c90 con 0x7fa6640ce5f0 2024-03-07T06:53:11.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.352+0000 7fa661600640 1 -- 172.21.15.133:0/4292760955 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa6500327e0 con 0x7fa6640ce5f0 2024-03-07T06:53:11.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.353+0000 7fa661600640 1 --2- 172.21.15.133:0/4292760955 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa64406b4d0 0x7fa64406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:11.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.353+0000 7fa661600640 1 -- 172.21.15.133:0/4292760955 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa65002f030 con 0x7fa6640ce5f0 2024-03-07T06:53:11.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.354+0000 7fa669400640 1 --2- 172.21.15.133:0/4292760955 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa64406b4d0 0x7fa64406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:11.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.355+0000 7fa669400640 1 --2- 172.21.15.133:0/4292760955 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa64406b4d0 0x7fa64406d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa658000f60 tx=0x7fa658007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:11.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.357+0000 7fa661600640 1 -- 172.21.15.133:0/4292760955 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa650041070 con 0x7fa6640ce5f0 2024-03-07T06:53:11.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.552+0000 7fa66be00640 1 -- 172.21.15.133:0/4292760955 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6640cc660 con 0x7fa6640ce5f0 2024-03-07T06:53:11.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.555+0000 7fa661600640 1 -- 172.21.15.133:0/4292760955 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16717 (secure 0 0 0) 0x7fa65007b3e0 con 0x7fa6640ce5f0 2024-03-07T06:53:11.554 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:11.554 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":35,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:52:36.759644+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:52:33.007428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-07T06:52:34.201590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:53:11.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.558+0000 7fa66be00640 1 -- 172.21.15.133:0/4292760955 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa64406b4d0 msgr2=0x7fa64406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:11.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.558+0000 7fa66be00640 1 --2- 172.21.15.133:0/4292760955 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa64406b4d0 0x7fa64406d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa658000f60 tx=0x7fa658007c00 comp rx=0 tx=0).stop 2024-03-07T06:53:11.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.558+0000 7fa66be00640 1 -- 172.21.15.133:0/4292760955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6640ce5f0 msgr2=0x7fa66409b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:11.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.558+0000 7fa66be00640 1 --2- 172.21.15.133:0/4292760955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6640ce5f0 0x7fa66409b240 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa6500097a0 tx=0x7fa6500046c0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.558+0000 7fa66be00640 1 -- 172.21.15.133:0/4292760955 shutdown_connections 2024-03-07T06:53:11.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.558+0000 7fa66be00640 1 --2- 172.21.15.133:0/4292760955 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa66415b1b0 0x7fa66409b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.558+0000 7fa66be00640 1 --2- 172.21.15.133:0/4292760955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6640ce5f0 0x7fa66409b240 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.558+0000 7fa66be00640 1 --2- 172.21.15.133:0/4292760955 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa64406b4d0 0x7fa64406d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.558+0000 7fa66be00640 1 --2- 172.21.15.133:0/4292760955 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa6640af950 0x7fa66414ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.558+0000 7fa66be00640 1 -- 172.21.15.133:0/4292760955 >> 172.21.15.133:0/4292760955 conn(0x7fa6640b3120 msgr2=0x7fa66406dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:11.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.558+0000 7fa66be00640 1 -- 172.21.15.133:0/4292760955 shutdown_connections 2024-03-07T06:53:11.557 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.558+0000 7fa66be00640 1 -- 172.21.15.133:0/4292760955 wait complete. 2024-03-07T06:53:11.574 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-07T06:53:11.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.771+0000 7f3916a00640 1 Processor -- start 2024-03-07T06:53:11.770 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.771+0000 7f3916a00640 1 -- start start 2024-03-07T06:53:11.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.773+0000 7f3916a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39101420d0 0x7f391015d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:11.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.773+0000 7f3916a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3910067020 0x7f39100accc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:11.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.773+0000 7f3916a00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39101424b0 con 0x7f39100cd080 2024-03-07T06:53:11.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.773+0000 7f3916a00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39100aa780 con 0x7f3910067020 2024-03-07T06:53:11.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.773+0000 7f3916a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39100aaa60 con 0x7f39101420d0 2024-03-07T06:53:11.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.773+0000 7f390f400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3910067020 0x7f39100accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:11.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.773+0000 7f390f400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3910067020 0x7f39100accc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48966/0 (socket says 172.21.15.133:48966) 2024-03-07T06:53:11.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.773+0000 7f390f400640 1 -- 172.21.15.133:0/1688519659 learned_addr learned my addr 172.21.15.133:0/1688519659 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:11.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.773+0000 7f3914a00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f39100cd080 0x7f39100ab820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52584/0 (socket says 172.21.15.133:52584) 2024-03-07T06:53:11.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.774+0000 7f390fe00640 1 --2- 172.21.15.133:0/1688519659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39101420d0 0x7f391015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:11.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.774+0000 7f390f400640 1 -- 172.21.15.133:0/1688519659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39101420d0 msgr2=0x7f391015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:11.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.774+0000 7f390f400640 1 --2- 172.21.15.133:0/1688519659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39101420d0 0x7f391015d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.774+0000 7f390f400640 1 -- 172.21.15.133:0/1688519659 >> v1:172.21.15.133:6789/0 conn(0x7f39100cd080 legacy=0x7f39100ab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:11.773 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.774+0000 7f390f400640 1 -- 172.21.15.133:0/1688519659 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39100aad40 con 0x7f3910067020 2024-03-07T06:53:11.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.775+0000 7f390f400640 1 --2- 172.21.15.133:0/1688519659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3910067020 0x7f39100accc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009870 tx=0x7f38fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64349961e1ed619 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:11.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.775+0000 7f390ea00640 1 -- 172.21.15.133:0/1688519659 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc035020 con 0x7f3910067020 2024-03-07T06:53:11.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.775+0000 7f390ea00640 1 -- 172.21.15.133:0/1688519659 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc031640 con 0x7f3910067020 2024-03-07T06:53:11.774 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.775+0000 7f390ea00640 1 -- 172.21.15.133:0/1688519659 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc02d040 con 0x7f3910067020 2024-03-07T06:53:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.776+0000 7f3916a00640 1 -- 172.21.15.133:0/1688519659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3910067020 msgr2=0x7f39100accc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.776+0000 7f3916a00640 1 --2- 172.21.15.133:0/1688519659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3910067020 0x7f39100accc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009870 tx=0x7f38fc02b8f0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.776+0000 7f3916a00640 1 -- 172.21.15.133:0/1688519659 shutdown_connections 2024-03-07T06:53:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.776+0000 7f3916a00640 1 --2- 172.21.15.133:0/1688519659 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3910067020 0x7f39100accc0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.776+0000 7f3916a00640 1 --2- 172.21.15.133:0/1688519659 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39101420d0 0x7f391015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.776+0000 7f3916a00640 1 -- 172.21.15.133:0/1688519659 >> 172.21.15.133:0/1688519659 conn(0x7f3910073030 msgr2=0x7f3910070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.776+0000 7f3916a00640 1 -- 172.21.15.133:0/1688519659 shutdown_connections 2024-03-07T06:53:11.775 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.776+0000 7f3916a00640 1 -- 172.21.15.133:0/1688519659 wait complete. 2024-03-07T06:53:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.778+0000 7f3916a00640 1 Processor -- start 2024-03-07T06:53:11.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.778+0000 7f3916a00640 1 -- start start 2024-03-07T06:53:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.779+0000 7f3916a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3910067020 0x7f39101519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.779+0000 7f3916a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39100cd080 0x7f39100ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.779+0000 7f3916a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39101420d0 0x7f391013aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.779+0000 7f3916a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391015da80 con 0x7f39100cd080 2024-03-07T06:53:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.779+0000 7f3916a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39100ab300 con 0x7f3910067020 2024-03-07T06:53:11.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.779+0000 7f3916a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39100ad200 con 0x7f39101420d0 2024-03-07T06:53:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.779+0000 7f390f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39100cd080 0x7f39100ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.779+0000 7f390f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39100cd080 0x7f39100ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48976/0 (socket says 172.21.15.133:48976) 2024-03-07T06:53:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.779+0000 7f390f400640 1 -- 172.21.15.133:0/3039310877 learned_addr learned my addr 172.21.15.133:0/3039310877 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.779+0000 7f390f400640 1 -- 172.21.15.133:0/3039310877 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39101420d0 msgr2=0x7f391013aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.780+0000 7f390f400640 1 --2- 172.21.15.133:0/3039310877 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39101420d0 0x7f391013aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.780+0000 7f390fe00640 1 --2- 172.21.15.133:0/3039310877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3910067020 0x7f39101519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.780+0000 7f390f400640 1 -- 172.21.15.133:0/3039310877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3910067020 msgr2=0x7f39101519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.780+0000 7f390f400640 1 --2- 172.21.15.133:0/3039310877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3910067020 0x7f39101519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.780+0000 7f390f400640 1 -- 172.21.15.133:0/3039310877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391013b6a0 con 0x7f39100cd080 2024-03-07T06:53:11.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.780+0000 7f390f400640 1 --2- 172.21.15.133:0/3039310877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39100cd080 0x7f39100ceaf0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f38fc02b930 tx=0x7f38fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:11.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.780+0000 7f3907e00640 1 -- 172.21.15.133:0/3039310877 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc040070 con 0x7f39100cd080 2024-03-07T06:53:11.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.780+0000 7f3907e00640 1 -- 172.21.15.133:0/3039310877 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc03e070 con 0x7f39100cd080 2024-03-07T06:53:11.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.780+0000 7f3916a00640 1 -- 172.21.15.133:0/3039310877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391013b980 con 0x7f39100cd080 2024-03-07T06:53:11.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.780+0000 7f3907e00640 1 -- 172.21.15.133:0/3039310877 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc035040 con 0x7f39100cd080 2024-03-07T06:53:11.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.781+0000 7f3916a00640 1 -- 172.21.15.133:0/3039310877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391013bf40 con 0x7f39100cd080 2024-03-07T06:53:11.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.782+0000 7f3916a00640 1 -- 172.21.15.133:0/3039310877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3910062c90 con 0x7f39100cd080 2024-03-07T06:53:11.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.783+0000 7f3907e00640 1 -- 172.21.15.133:0/3039310877 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f38fc038020 con 0x7f39100cd080 2024-03-07T06:53:11.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.784+0000 7f3907e00640 1 --2- 172.21.15.133:0/3039310877 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f38f006b4d0 0x7f38f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:11.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.784+0000 7f3907e00640 1 -- 172.21.15.133:0/3039310877 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38fc0abbe0 con 0x7f39100cd080 2024-03-07T06:53:11.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.785+0000 7f390fe00640 1 --2- 172.21.15.133:0/3039310877 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f38f006b4d0 0x7f38f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:11.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.786+0000 7f390fe00640 1 --2- 172.21.15.133:0/3039310877 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f38f006b4d0 0x7f38f006d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3900006fd0 tx=0x7f3900008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:11.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:11.789+0000 7f3907e00640 1 -- 172.21.15.133:0/3039310877 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f38fc07aa50 con 0x7f39100cd080 2024-03-07T06:53:12.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.035+0000 7f3916a00640 1 -- 172.21.15.133:0/3039310877 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f391006dc90 con 0x7f39100cd080 2024-03-07T06:53:12.034 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-03-07T06:53:12.036+0000 7fb4e8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-07T06:53:12.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.039+0000 7f3907e00640 1 -- 172.21.15.133:0/3039310877 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f38fc043e40 con 0x7f39100cd080 2024-03-07T06:53:12.037 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-07T06:53:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.042+0000 7f3916a00640 1 -- 172.21.15.133:0/3039310877 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f38f006b4d0 msgr2=0x7f38f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.042+0000 7f3916a00640 1 --2- 172.21.15.133:0/3039310877 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f38f006b4d0 0x7f38f006d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3900006fd0 tx=0x7f3900008040 comp rx=0 tx=0).stop 2024-03-07T06:53:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.042+0000 7f3916a00640 1 -- 172.21.15.133:0/3039310877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39100cd080 msgr2=0x7f39100ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.042+0000 7f3916a00640 1 --2- 172.21.15.133:0/3039310877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39100cd080 0x7f39100ceaf0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f38fc02b930 tx=0x7f38fc0046c0 comp rx=0 tx=0).stop 2024-03-07T06:53:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.042+0000 7f3916a00640 1 -- 172.21.15.133:0/3039310877 shutdown_connections 2024-03-07T06:53:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.042+0000 7f3916a00640 1 --2- 172.21.15.133:0/3039310877 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39101420d0 0x7f391013aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.042+0000 7f3916a00640 1 --2- 172.21.15.133:0/3039310877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39100cd080 0x7f39100ceaf0 secure :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f38fc02b930 tx=0x7f38fc0046c0 comp rx=0 tx=0).stop 2024-03-07T06:53:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.042+0000 7f3916a00640 1 --2- 172.21.15.133:0/3039310877 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f38f006b4d0 0x7f38f006d990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:12.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.042+0000 7f3916a00640 1 --2- 172.21.15.133:0/3039310877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3910067020 0x7f39101519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:12.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.042+0000 7f3916a00640 1 -- 172.21.15.133:0/3039310877 >> 172.21.15.133:0/3039310877 conn(0x7f3910073030 msgr2=0x7f3910070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:12.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.042+0000 7f3916a00640 1 -- 172.21.15.133:0/3039310877 shutdown_connections 2024-03-07T06:53:12.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.042+0000 7f3916a00640 1 -- 172.21.15.133:0/3039310877 wait complete. 2024-03-07T06:53:12.060 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-07T06:53:12.131 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-03-07T06:53:12.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.247+0000 7fda8a200640 1 Processor -- start 2024-03-07T06:53:12.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.247+0000 7fda8a200640 1 -- start start 2024-03-07T06:53:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.248+0000 7fda8a200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fda84143870 0x7fda840b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.249+0000 7fda8a200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda8415f070 0x7fda8406d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.249+0000 7fda8a200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda840b3630 con 0x7fda840b31b0 2024-03-07T06:53:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.249+0000 7fda8a200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda840b3910 con 0x7fda84143870 2024-03-07T06:53:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.249+0000 7fda8a200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda840b3bf0 con 0x7fda8415f070 2024-03-07T06:53:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.249+0000 7fda88e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fda84143870 0x7fda840b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:12.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.249+0000 7fda88e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fda84143870 0x7fda840b2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48998/0 (socket says 172.21.15.133:48998) 2024-03-07T06:53:12.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.249+0000 7fda88e00640 1 -- 172.21.15.133:0/2740157993 learned_addr learned my addr 172.21.15.133:0/2740157993 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:12.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.249+0000 7fda89800640 1 --2- 172.21.15.133:0/2740157993 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda8415f070 0x7fda8406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:12.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.249+0000 7fda88e00640 1 -- 172.21.15.133:0/2740157993 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda8415f070 msgr2=0x7fda8406d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:12.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.249+0000 7fda88e00640 1 --2- 172.21.15.133:0/2740157993 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda8415f070 0x7fda8406d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:12.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.249+0000 7fda88e00640 1 -- 172.21.15.133:0/2740157993 >> v1:172.21.15.161:6789/0 conn(0x7fda840b31b0 legacy=0x7fda8415e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:12.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.249+0000 7fda88e00640 1 -- 172.21.15.133:0/2740157993 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda840b3ed0 con 0x7fda84143870 2024-03-07T06:53:12.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.250+0000 7fda88e00640 1 --2- 172.21.15.133:0/2740157993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda84143870 0x7fda840b2c70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fda74009870 tx=0x7fda7402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b174a4c3fc30a3e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:12.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.251+0000 7fda83400640 1 -- 172.21.15.133:0/2740157993 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74038070 con 0x7fda84143870 2024-03-07T06:53:12.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.251+0000 7fda83400640 1 -- 172.21.15.133:0/2740157993 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda74031740 con 0x7fda84143870 2024-03-07T06:53:12.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.251+0000 7fda83400640 1 -- 172.21.15.133:0/2740157993 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7402f050 con 0x7fda84143870 2024-03-07T06:53:12.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.251+0000 7fda8a200640 1 -- 172.21.15.133:0/2740157993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda84143870 msgr2=0x7fda840b2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:12.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.251+0000 7fda8a200640 1 --2- 172.21.15.133:0/2740157993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda84143870 0x7fda840b2c70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fda74009870 tx=0x7fda7402b7e0 comp rx=0 tx=0).stop 2024-03-07T06:53:12.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.251+0000 7fda8a200640 1 -- 172.21.15.133:0/2740157993 shutdown_connections 2024-03-07T06:53:12.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.251+0000 7fda8a200640 1 --2- 172.21.15.133:0/2740157993 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda8415f070 0x7fda8406d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:12.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.252+0000 7fda8a200640 1 --2- 172.21.15.133:0/2740157993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda84143870 0x7fda840b2c70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:12.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.252+0000 7fda8a200640 1 -- 172.21.15.133:0/2740157993 >> 172.21.15.133:0/2740157993 conn(0x7fda840b1d90 msgr2=0x7fda840b21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:12.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.252+0000 7fda8a200640 1 -- 172.21.15.133:0/2740157993 shutdown_connections 2024-03-07T06:53:12.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.252+0000 7fda8a200640 1 -- 172.21.15.133:0/2740157993 wait complete. 2024-03-07T06:53:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.253+0000 7fda8a200640 1 Processor -- start 2024-03-07T06:53:12.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.253+0000 7fda8a200640 1 -- start start 2024-03-07T06:53:12.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.254+0000 7fda8a200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda840b31b0 0x7fda841576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:12.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.254+0000 7fda8a200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda84143870 0x7fda841efce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:12.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.254+0000 7fda8a200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda8415f070 0x7fda8413ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:12.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.254+0000 7fda8a200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda84143c50 con 0x7fda840b31b0 2024-03-07T06:53:12.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.254+0000 7fda8a200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8415f7f0 con 0x7fda8415f070 2024-03-07T06:53:12.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.254+0000 7fda8a200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda84143dd0 con 0x7fda84143870 2024-03-07T06:53:12.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.255+0000 7fda88e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda840b31b0 0x7fda841576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:12.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.255+0000 7fda88e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda840b31b0 0x7fda841576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49014/0 (socket says 172.21.15.133:49014) 2024-03-07T06:53:12.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.255+0000 7fda88e00640 1 -- 172.21.15.133:0/308463086 learned_addr learned my addr 172.21.15.133:0/308463086 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:12.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.255+0000 7fda83e00640 1 --2- 172.21.15.133:0/308463086 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda84143870 0x7fda841efce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:12.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.255+0000 7fda88e00640 1 -- 172.21.15.133:0/308463086 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda84143870 msgr2=0x7fda841efce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:12.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.255+0000 7fda88e00640 1 --2- 172.21.15.133:0/308463086 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda84143870 0x7fda841efce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:12.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.255+0000 7fda88e00640 1 -- 172.21.15.133:0/308463086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda8415f070 msgr2=0x7fda8413ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:12.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.255+0000 7fda88e00640 1 --2- 172.21.15.133:0/308463086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda8415f070 0x7fda8413ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:12.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.255+0000 7fda88e00640 1 -- 172.21.15.133:0/308463086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda84157c20 con 0x7fda840b31b0 2024-03-07T06:53:12.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.255+0000 7fda88e00640 1 --2- 172.21.15.133:0/308463086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda840b31b0 0x7fda841576e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fda74009870 tx=0x7fda740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:12.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.256+0000 7fda80c00640 1 -- 172.21.15.133:0/308463086 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74042070 con 0x7fda840b31b0 2024-03-07T06:53:12.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.256+0000 7fda80c00640 1 -- 172.21.15.133:0/308463086 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda74033c10 con 0x7fda840b31b0 2024-03-07T06:53:12.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.256+0000 7fda80c00640 1 -- 172.21.15.133:0/308463086 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74038040 con 0x7fda840b31b0 2024-03-07T06:53:12.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.256+0000 7fda8a200640 1 -- 172.21.15.133:0/308463086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda841f07d0 con 0x7fda840b31b0 2024-03-07T06:53:12.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.256+0000 7fda8a200640 1 -- 172.21.15.133:0/308463086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda841f0d90 con 0x7fda840b31b0 2024-03-07T06:53:12.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.257+0000 7fda8a200640 1 -- 172.21.15.133:0/308463086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda84067c80 con 0x7fda840b31b0 2024-03-07T06:53:12.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.258+0000 7fda80c00640 1 -- 172.21.15.133:0/308463086 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fda74045070 con 0x7fda840b31b0 2024-03-07T06:53:12.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.259+0000 7fda80c00640 1 --2- 172.21.15.133:0/308463086 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fda6406b4d0 0x7fda6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:12.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.259+0000 7fda80c00640 1 -- 172.21.15.133:0/308463086 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fda740ac110 con 0x7fda840b31b0 2024-03-07T06:53:12.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.259+0000 7fda83e00640 1 --2- 172.21.15.133:0/308463086 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fda6406b4d0 0x7fda6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:12.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.261+0000 7fda83e00640 1 --2- 172.21.15.133:0/308463086 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fda6406b4d0 0x7fda6406d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fda84140730 tx=0x7fda70009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:12.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.264+0000 7fda80c00640 1 -- 172.21.15.133:0/308463086 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fda7404c030 con 0x7fda840b31b0 2024-03-07T06:53:12.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:12.497+0000 7fda8a200640 1 -- 172.21.15.133:0/308463086 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fda84062c90 con 0x7fda840b31b0 2024-03-07T06:53:13.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.042+0000 7fda80c00640 1 -- 172.21.15.133:0/308463086 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7fda7407af20 con 0x7fda840b31b0 2024-03-07T06:53:13.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.046+0000 7fda8a200640 1 -- 172.21.15.133:0/308463086 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fda6406b4d0 msgr2=0x7fda6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:13.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.046+0000 7fda8a200640 1 --2- 172.21.15.133:0/308463086 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fda6406b4d0 0x7fda6406d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fda84140730 tx=0x7fda70009040 comp rx=0 tx=0).stop 2024-03-07T06:53:13.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.046+0000 7fda8a200640 1 -- 172.21.15.133:0/308463086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda840b31b0 msgr2=0x7fda841576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:13.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.046+0000 7fda8a200640 1 --2- 172.21.15.133:0/308463086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda840b31b0 0x7fda841576e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fda74009870 tx=0x7fda740046c0 comp rx=0 tx=0).stop 2024-03-07T06:53:13.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.046+0000 7fda8a200640 1 -- 172.21.15.133:0/308463086 shutdown_connections 2024-03-07T06:53:13.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.046+0000 7fda8a200640 1 --2- 172.21.15.133:0/308463086 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fda6406b4d0 0x7fda6406d990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:13.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.046+0000 7fda8a200640 1 --2- 172.21.15.133:0/308463086 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fda8415f070 0x7fda8413ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:13.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.046+0000 7fda8a200640 1 --2- 172.21.15.133:0/308463086 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fda84143870 0x7fda841efce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:13.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.046+0000 7fda8a200640 1 --2- 172.21.15.133:0/308463086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda840b31b0 0x7fda841576e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:13.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.046+0000 7fda8a200640 1 -- 172.21.15.133:0/308463086 >> 172.21.15.133:0/308463086 conn(0x7fda840b1d90 msgr2=0x7fda84072310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:13.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.046+0000 7fda8a200640 1 -- 172.21.15.133:0/308463086 shutdown_connections 2024-03-07T06:53:13.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.046+0000 7fda8a200640 1 -- 172.21.15.133:0/308463086 wait complete. 2024-03-07T06:53:13.067 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-07T06:53:13.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.281+0000 7fe78ac00640 1 Processor -- start 2024-03-07T06:53:13.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.281+0000 7fe78ac00640 1 -- start start 2024-03-07T06:53:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.283+0000 7fe78ac00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe784162a60 0x7fe78406cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.283+0000 7fe78ac00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe78406d150 0x7fe7840b6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.283+0000 7fe78ac00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7840d1510 con 0x7fe784147940 2024-03-07T06:53:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.283+0000 7fe78ac00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7840b52c0 con 0x7fe784162a60 2024-03-07T06:53:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.283+0000 7fe78ac00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7840b55a0 con 0x7fe78406d150 2024-03-07T06:53:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.283+0000 7fe783400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe784162a60 0x7fe78406cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.283+0000 7fe783400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe784162a60 0x7fe78406cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49020/0 (socket says 172.21.15.133:49020) 2024-03-07T06:53:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.283+0000 7fe783e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe784147940 0x7fe7840d0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52588/0 (socket says 172.21.15.133:52588) 2024-03-07T06:53:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.283+0000 7fe783400640 1 -- 172.21.15.133:0/1249916297 learned_addr learned my addr 172.21.15.133:0/1249916297 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:13.282 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.283+0000 7fe783400640 1 -- 172.21.15.133:0/1249916297 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe78406d150 msgr2=0x7fe7840b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:13.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.283+0000 7fe783400640 1 --2- 172.21.15.133:0/1249916297 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe78406d150 0x7fe7840b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:13.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.283+0000 7fe783400640 1 -- 172.21.15.133:0/1249916297 >> v1:172.21.15.133:6789/0 conn(0x7fe784147940 legacy=0x7fe7840d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:13.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.283+0000 7fe783400640 1 -- 172.21.15.133:0/1249916297 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7840b5880 con 0x7fe784162a60 2024-03-07T06:53:13.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.284+0000 7fe783400640 1 --2- 172.21.15.133:0/1249916297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe784162a60 0x7fe78406cc10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe770009870 tx=0x7fe77002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24ffe6d7a54583f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:13.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.285+0000 7fe782a00640 1 -- 172.21.15.133:0/1249916297 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770034070 con 0x7fe784162a60 2024-03-07T06:53:13.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.285+0000 7fe782a00640 1 -- 172.21.15.133:0/1249916297 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe770030840 con 0x7fe784162a60 2024-03-07T06:53:13.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.285+0000 7fe782a00640 1 -- 172.21.15.133:0/1249916297 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe770030c10 con 0x7fe784162a60 2024-03-07T06:53:13.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.285+0000 7fe78ac00640 1 -- 172.21.15.133:0/1249916297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe784162a60 msgr2=0x7fe78406cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:13.284 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.285+0000 7fe78ac00640 1 --2- 172.21.15.133:0/1249916297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe784162a60 0x7fe78406cc10 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe770009870 tx=0x7fe77002b980 comp rx=0 tx=0).stop 2024-03-07T06:53:13.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.285+0000 7fe78ac00640 1 -- 172.21.15.133:0/1249916297 shutdown_connections 2024-03-07T06:53:13.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.285+0000 7fe78ac00640 1 --2- 172.21.15.133:0/1249916297 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe78406d150 0x7fe7840b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:13.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.285+0000 7fe78ac00640 1 --2- 172.21.15.133:0/1249916297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe784162a60 0x7fe78406cc10 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:13.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.285+0000 7fe78ac00640 1 -- 172.21.15.133:0/1249916297 >> 172.21.15.133:0/1249916297 conn(0x7fe784070400 msgr2=0x7fe7840707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:13.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.286+0000 7fe78ac00640 1 -- 172.21.15.133:0/1249916297 shutdown_connections 2024-03-07T06:53:13.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.286+0000 7fe78ac00640 1 -- 172.21.15.133:0/1249916297 wait complete. 2024-03-07T06:53:13.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.287+0000 7fe78ac00640 1 Processor -- start 2024-03-07T06:53:13.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.287+0000 7fe78ac00640 1 -- start start 2024-03-07T06:53:13.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.288+0000 7fe78ac00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe78406d150 0x7fe78415f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:13.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.288+0000 7fe78ac00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe784147940 0x7fe7840ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:13.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.288+0000 7fe78ac00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe784162a60 0x7fe784141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:13.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.288+0000 7fe78ac00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7840b4f70 con 0x7fe784162a60 2024-03-07T06:53:13.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.288+0000 7fe78ac00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe784147d20 con 0x7fe784147940 2024-03-07T06:53:13.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.288+0000 7fe78ac00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7840b65c0 con 0x7fe78406d150 2024-03-07T06:53:13.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.288+0000 7fe788c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe784162a60 0x7fe784141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:13.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe788c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe784162a60 0x7fe784141b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49024/0 (socket says 172.21.15.133:49024) 2024-03-07T06:53:13.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe788c00640 1 -- 172.21.15.133:0/1622326460 learned_addr learned my addr 172.21.15.133:0/1622326460 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:13.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe783e00640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe78406d150 0x7fe78415f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:13.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe783400640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe784147940 0x7fe7840ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:13.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe788c00640 1 -- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe78406d150 msgr2=0x7fe78415f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:13.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe788c00640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe78406d150 0x7fe78415f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:13.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe788c00640 1 -- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe784147940 msgr2=0x7fe7840ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:13.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe788c00640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe784147940 0x7fe7840ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:13.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe788c00640 1 -- 172.21.15.133:0/1622326460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7840ceab0 con 0x7fe784162a60 2024-03-07T06:53:13.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe783e00640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe78406d150 0x7fe78415f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:13.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe788c00640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe784162a60 0x7fe784141b00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00b970 tx=0x7fe77c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:13.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe77be00640 1 -- 172.21.15.133:0/1622326460 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c019070 con 0x7fe784162a60 2024-03-07T06:53:13.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe77be00640 1 -- 172.21.15.133:0/1622326460 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c00cb00 con 0x7fe784162a60 2024-03-07T06:53:13.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe77be00640 1 -- 172.21.15.133:0/1622326460 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c014440 con 0x7fe784162a60 2024-03-07T06:53:13.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.289+0000 7fe78ac00640 1 -- 172.21.15.133:0/1622326460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7840ced90 con 0x7fe784162a60 2024-03-07T06:53:13.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.290+0000 7fe78ac00640 1 -- 172.21.15.133:0/1622326460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7840cf350 con 0x7fe784162a60 2024-03-07T06:53:13.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.291+0000 7fe78ac00640 1 -- 172.21.15.133:0/1622326460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe784062c90 con 0x7fe784162a60 2024-03-07T06:53:13.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.291+0000 7fe77be00640 1 -- 172.21.15.133:0/1622326460 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe77c01c070 con 0x7fe784162a60 2024-03-07T06:53:13.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.292+0000 7fe77be00640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe76406b500 0x7fe76406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:13.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.293+0000 7fe77be00640 1 -- 172.21.15.133:0/1622326460 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe77c089ff0 con 0x7fe784162a60 2024-03-07T06:53:13.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.293+0000 7fe783e00640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe76406b500 0x7fe76406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:13.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.294+0000 7fe783e00640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe76406b500 0x7fe76406d9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe774000f60 tx=0x7fe774009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:13.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.297+0000 7fe77be00640 1 -- 172.21.15.133:0/1622326460 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe77c058d20 con 0x7fe784162a60 2024-03-07T06:53:13.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:13.495+0000 7fe78ac00640 1 -- 172.21.15.133:0/1622326460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe7840b1fa0 con 0x7fe784162a60 2024-03-07T06:53:14.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.053+0000 7fe77be00640 1 -- 172.21.15.133:0/1622326460 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe77c058620 con 0x7fe784162a60 2024-03-07T06:53:14.052 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-03-07T06:53:14.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.057+0000 7fe78ac00640 1 -- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe76406b500 msgr2=0x7fe76406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:14.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.057+0000 7fe78ac00640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe76406b500 0x7fe76406d9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe774000f60 tx=0x7fe774009040 comp rx=0 tx=0).stop 2024-03-07T06:53:14.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.057+0000 7fe78ac00640 1 -- 172.21.15.133:0/1622326460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe784162a60 msgr2=0x7fe784141b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:14.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.057+0000 7fe78ac00640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe784162a60 0x7fe784141b00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00b970 tx=0x7fe77c00be30 comp rx=0 tx=0).stop 2024-03-07T06:53:14.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.057+0000 7fe78ac00640 1 -- 172.21.15.133:0/1622326460 shutdown_connections 2024-03-07T06:53:14.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.057+0000 7fe78ac00640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe784162a60 0x7fe784141b00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:14.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.057+0000 7fe78ac00640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe76406b500 0x7fe76406d9c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:14.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.057+0000 7fe78ac00640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe784147940 0x7fe7840ce280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:14.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.057+0000 7fe78ac00640 1 --2- 172.21.15.133:0/1622326460 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe78406d150 0x7fe78415f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:14.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.058+0000 7fe78ac00640 1 -- 172.21.15.133:0/1622326460 >> 172.21.15.133:0/1622326460 conn(0x7fe784070400 msgr2=0x7fe7840d0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:14.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.059+0000 7fe78ac00640 1 -- 172.21.15.133:0/1622326460 shutdown_connections 2024-03-07T06:53:14.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.059+0000 7fe78ac00640 1 -- 172.21.15.133:0/1622326460 wait complete. 2024-03-07T06:53:14.082 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-07T06:53:14.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.284+0000 7fe1d0c00640 1 Processor -- start 2024-03-07T06:53:14.283 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.285+0000 7fe1d0c00640 1 -- start start 2024-03-07T06:53:14.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.286+0000 7fe1d0c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe1cc0cc6f0 0x7fe1cc0b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:14.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.286+0000 7fe1d0c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe1cc0cdf00 0x7fe1cc15ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:14.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.286+0000 7fe1d0c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1cc070820 0x7fe1cc070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:14.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.286+0000 7fe1d0c00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1cc0b2190 con 0x7fe1cc0cdf00 2024-03-07T06:53:14.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.286+0000 7fe1d0c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1cc0b2430 con 0x7fe1cc0cc6f0 2024-03-07T06:53:14.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.286+0000 7fe1d0c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc0b2710 con 0x7fe1cc070820 2024-03-07T06:53:14.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.287+0000 7fe1c1400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe1cc0cdf00 0x7fe1cc15ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:14.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.287+0000 7fe1c1400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe1cc0cdf00 0x7fe1cc15ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49028/0 (socket says 172.21.15.133:49028) 2024-03-07T06:53:14.286 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.287+0000 7fe1c1400640 1 -- 172.21.15.133:0/742436354 learned_addr learned my addr 172.21.15.133:0/742436354 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:14.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.287+0000 7fe1c1400640 1 -- 172.21.15.133:0/742436354 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1cc070820 msgr2=0x7fe1cc070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:14.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.287+0000 7fe1c1400640 1 --2- 172.21.15.133:0/742436354 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1cc070820 0x7fe1cc070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:14.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.287+0000 7fe1c1400640 1 -- 172.21.15.133:0/742436354 >> v2:172.21.15.161:3300/0 conn(0x7fe1cc0cc6f0 msgr2=0x7fe1cc0b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:14.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.287+0000 7fe1c1400640 1 --2- 172.21.15.133:0/742436354 >> v2:172.21.15.161:3300/0 conn(0x7fe1cc0cc6f0 0x7fe1cc0b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:14.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.287+0000 7fe1c1400640 1 -- 172.21.15.133:0/742436354 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc0b29f0 con 0x7fe1cc0cdf00 2024-03-07T06:53:14.287 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.288+0000 7fe1c1400640 1 --2- 172.21.15.133:0/742436354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cdf00 0x7fe1cc15ccd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc071b20 tx=0x7fe1b402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950b9b3205c7d6e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:14.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.288+0000 7fe1c9400640 1 -- 172.21.15.133:0/742436354 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4035020 con 0x7fe1cc0cdf00 2024-03-07T06:53:14.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.288+0000 7fe1c9400640 1 -- 172.21.15.133:0/742436354 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b4031880 con 0x7fe1cc0cdf00 2024-03-07T06:53:14.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.288+0000 7fe1c9400640 1 -- 172.21.15.133:0/742436354 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4031c50 con 0x7fe1cc0cdf00 2024-03-07T06:53:14.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.289+0000 7fe1d0c00640 1 -- 172.21.15.133:0/742436354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cdf00 msgr2=0x7fe1cc15ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:14.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.289+0000 7fe1d0c00640 1 --2- 172.21.15.133:0/742436354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cdf00 0x7fe1cc15ccd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc071b20 tx=0x7fe1b402b920 comp rx=0 tx=0).stop 2024-03-07T06:53:14.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.289+0000 7fe1d0c00640 1 -- 172.21.15.133:0/742436354 shutdown_connections 2024-03-07T06:53:14.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.289+0000 7fe1d0c00640 1 --2- 172.21.15.133:0/742436354 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1cc070820 0x7fe1cc070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:14.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.289+0000 7fe1d0c00640 1 --2- 172.21.15.133:0/742436354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cdf00 0x7fe1cc15ccd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:14.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.289+0000 7fe1d0c00640 1 --2- 172.21.15.133:0/742436354 >> v2:172.21.15.161:3300/0 conn(0x7fe1cc0cc6f0 0x7fe1cc0b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:14.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.289+0000 7fe1d0c00640 1 -- 172.21.15.133:0/742436354 >> 172.21.15.133:0/742436354 conn(0x7fe1cc066eb0 msgr2=0x7fe1cc067290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:14.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.289+0000 7fe1d0c00640 1 -- 172.21.15.133:0/742436354 shutdown_connections 2024-03-07T06:53:14.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.289+0000 7fe1d0c00640 1 -- 172.21.15.133:0/742436354 wait complete. 2024-03-07T06:53:14.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.291+0000 7fe1d0c00640 1 Processor -- start 2024-03-07T06:53:14.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.291+0000 7fe1d0c00640 1 -- start start 2024-03-07T06:53:14.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.292+0000 7fe1d0c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1cc070820 0x7fe1cc0af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:14.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.292+0000 7fe1d0c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cc6f0 0x7fe1cc14d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:14.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.292+0000 7fe1d0c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1cc0cdf00 0x7fe1cc13f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:14.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.292+0000 7fe1d0c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc15d210 con 0x7fe1cc0cc6f0 2024-03-07T06:53:14.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.292+0000 7fe1d0c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc15c460 con 0x7fe1cc0cdf00 2024-03-07T06:53:14.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.292+0000 7fe1d0c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1cc14e7e0 con 0x7fe1cc070820 2024-03-07T06:53:14.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.292+0000 7fe1c1400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cc6f0 0x7fe1cc14d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:14.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.292+0000 7fe1c1400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cc6f0 0x7fe1cc14d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49030/0 (socket says 172.21.15.133:49030) 2024-03-07T06:53:14.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.292+0000 7fe1c1400640 1 -- 172.21.15.133:0/1493465647 learned_addr learned my addr 172.21.15.133:0/1493465647 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:14.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.293+0000 7fe1c9e00640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1cc070820 0x7fe1cc0af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:14.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.293+0000 7fe1c1400640 1 -- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1cc070820 msgr2=0x7fe1cc0af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:14.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.293+0000 7fe1ca800640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1cc0cdf00 0x7fe1cc13f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:14.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.293+0000 7fe1c1400640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1cc070820 0x7fe1cc0af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:14.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.293+0000 7fe1c1400640 1 -- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1cc0cdf00 msgr2=0x7fe1cc13f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:14.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.293+0000 7fe1c1400640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1cc0cdf00 0x7fe1cc13f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:14.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.293+0000 7fe1c1400640 1 -- 172.21.15.133:0/1493465647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1cc14dc60 con 0x7fe1cc0cc6f0 2024-03-07T06:53:14.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.293+0000 7fe1c9e00640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1cc070820 0x7fe1cc0af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:14.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.293+0000 7fe1c1400640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cc6f0 0x7fe1cc14d3e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe1b402b960 tx=0x7fe1b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:14.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.293+0000 7fe1c2a00640 1 -- 172.21.15.133:0/1493465647 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b40453f0 con 0x7fe1cc0cc6f0 2024-03-07T06:53:14.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.293+0000 7fe1c2a00640 1 -- 172.21.15.133:0/1493465647 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b4045590 con 0x7fe1cc0cc6f0 2024-03-07T06:53:14.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.293+0000 7fe1c2a00640 1 -- 172.21.15.133:0/1493465647 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b4045980 con 0x7fe1cc0cc6f0 2024-03-07T06:53:14.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.293+0000 7fe1d0c00640 1 -- 172.21.15.133:0/1493465647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1cc14df40 con 0x7fe1cc0cc6f0 2024-03-07T06:53:14.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.294+0000 7fe1d0c00640 1 -- 172.21.15.133:0/1493465647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1cc14e500 con 0x7fe1cc0cc6f0 2024-03-07T06:53:14.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.295+0000 7fe1d0c00640 1 -- 172.21.15.133:0/1493465647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1cc062c90 con 0x7fe1cc0cc6f0 2024-03-07T06:53:14.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.296+0000 7fe1c2a00640 1 -- 172.21.15.133:0/1493465647 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe1b404c060 con 0x7fe1cc0cc6f0 2024-03-07T06:53:14.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.296+0000 7fe1c2a00640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe1a806b500 0x7fe1a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:14.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.297+0000 7fe1c2a00640 1 -- 172.21.15.133:0/1493465647 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fe1b40ac310 con 0x7fe1cc0cc6f0 2024-03-07T06:53:14.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.297+0000 7fe1c9e00640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe1a806b500 0x7fe1a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:14.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.298+0000 7fe1c9e00640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe1a806b500 0x7fe1a806d9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00a870 tx=0x7fe1bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:14.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.301+0000 7fe1c2a00640 1 -- 172.21.15.133:0/1493465647 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe1b4040070 con 0x7fe1cc0cc6f0 2024-03-07T06:53:14.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:14.504+0000 7fe1d0c00640 1 -- 172.21.15.133:0/1493465647 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe1cc14f120 con 0x7fe1cc0cc6f0 2024-03-07T06:53:15.065 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.065+0000 7fe1c2a00640 1 -- 172.21.15.133:0/1493465647 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe1b407b380 con 0x7fe1cc0cc6f0 2024-03-07T06:53:15.065 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-03-07T06:53:15.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.068+0000 7fe1d0c00640 1 -- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe1a806b500 msgr2=0x7fe1a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.068+0000 7fe1d0c00640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe1a806b500 0x7fe1a806d9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00a870 tx=0x7fe1bc008040 comp rx=0 tx=0).stop 2024-03-07T06:53:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.068+0000 7fe1d0c00640 1 -- 172.21.15.133:0/1493465647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cc6f0 msgr2=0x7fe1cc14d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.068+0000 7fe1d0c00640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cc6f0 0x7fe1cc14d3e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe1b402b960 tx=0x7fe1b40046c0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.069+0000 7fe1d0c00640 1 -- 172.21.15.133:0/1493465647 shutdown_connections 2024-03-07T06:53:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.069+0000 7fe1d0c00640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe1a806b500 0x7fe1a806d9c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.069+0000 7fe1d0c00640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1cc0cdf00 0x7fe1cc13f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.069+0000 7fe1d0c00640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cc6f0 0x7fe1cc14d3e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.069+0000 7fe1d0c00640 1 --2- 172.21.15.133:0/1493465647 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1cc070820 0x7fe1cc0af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.069+0000 7fe1d0c00640 1 -- 172.21.15.133:0/1493465647 >> 172.21.15.133:0/1493465647 conn(0x7fe1cc066eb0 msgr2=0x7fe1cc0b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.069+0000 7fe1d0c00640 1 -- 172.21.15.133:0/1493465647 shutdown_connections 2024-03-07T06:53:15.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.069+0000 7fe1d0c00640 1 -- 172.21.15.133:0/1493465647 wait complete. 2024-03-07T06:53:15.090 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-07T06:53:15.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.295+0000 7f4321000640 1 Processor -- start 2024-03-07T06:53:15.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.295+0000 7f4321000640 1 -- start start 2024-03-07T06:53:15.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.296+0000 7f4321000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f431c0cf4c0 0x7f431c166aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:15.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.296+0000 7f4321000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431c15c470 con 0x7f431c159c70 2024-03-07T06:53:15.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.297+0000 7f4321000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431c15c6f0 con 0x7f431c070b10 2024-03-07T06:53:15.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.297+0000 7f4321000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431c15c9d0 con 0x7f431c0cf4c0 2024-03-07T06:53:15.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.297+0000 7f4319e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f431c159c70 0x7f431c070400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52596/0 (socket says 172.21.15.133:52596) 2024-03-07T06:53:15.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.297+0000 7f4319e00640 1 -- 172.21.15.133:0/3678432181 learned_addr learned my addr 172.21.15.133:0/3678432181 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:15.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.297+0000 7f431a800640 1 --2- 172.21.15.133:0/3678432181 >> v2:172.21.15.133:3300/0 conn(0x7f431c0cf4c0 0x7f431c166aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:15.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.297+0000 7f4318a00640 1 -- 172.21.15.133:0/3678432181 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549950154 0 0) 0x7f430c001110 con 0x7f431c159c70 2024-03-07T06:53:15.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.297+0000 7f431a800640 1 -- 172.21.15.133:0/3678432181 >> v1:172.21.15.161:6789/0 conn(0x7f431c070b10 legacy=0x7f431c0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:15.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.297+0000 7f431a800640 1 -- 172.21.15.133:0/3678432181 >> v1:172.21.15.133:6789/0 conn(0x7f431c159c70 legacy=0x7f431c070400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:15.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.297+0000 7f431a800640 1 -- 172.21.15.133:0/3678432181 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c15ccb0 con 0x7f431c0cf4c0 2024-03-07T06:53:15.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.298+0000 7f431a800640 1 --2- 172.21.15.133:0/3678432181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f431c0cf4c0 0x7f431c166aa0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4314009d90 tx=0x7f431402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6830d5671b5317e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:15.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.299+0000 7f4318a00640 1 -- 172.21.15.133:0/3678432181 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314039070 con 0x7f431c0cf4c0 2024-03-07T06:53:15.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.299+0000 7f4318a00640 1 -- 172.21.15.133:0/3678432181 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431402cc60 con 0x7f431c0cf4c0 2024-03-07T06:53:15.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.299+0000 7f4318a00640 1 -- 172.21.15.133:0/3678432181 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314034660 con 0x7f431c0cf4c0 2024-03-07T06:53:15.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.299+0000 7f4321000640 1 -- 172.21.15.133:0/3678432181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f431c0cf4c0 msgr2=0x7f431c166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:15.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.299+0000 7f4321000640 1 --2- 172.21.15.133:0/3678432181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f431c0cf4c0 0x7f431c166aa0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4314009d90 tx=0x7f431402beb0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.299+0000 7f4321000640 1 -- 172.21.15.133:0/3678432181 shutdown_connections 2024-03-07T06:53:15.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.299+0000 7f4321000640 1 --2- 172.21.15.133:0/3678432181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f431c0cf4c0 0x7f431c166aa0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.299+0000 7f4321000640 1 -- 172.21.15.133:0/3678432181 >> 172.21.15.133:0/3678432181 conn(0x7f431c0cfb00 msgr2=0x7f431c0cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:15.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.299+0000 7f4321000640 1 -- 172.21.15.133:0/3678432181 shutdown_connections 2024-03-07T06:53:15.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.299+0000 7f4321000640 1 -- 172.21.15.133:0/3678432181 wait complete. 2024-03-07T06:53:15.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.301+0000 7f4321000640 1 Processor -- start 2024-03-07T06:53:15.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.301+0000 7f4321000640 1 -- start start 2024-03-07T06:53:15.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.302+0000 7f4321000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c070b10 0x7f431c0ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:15.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.302+0000 7f4321000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f431c0cf4c0 0x7f431c155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.302+0000 7f4321000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f431c159c70 0x7f431c14ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.302+0000 7f4321000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0b4e50 con 0x7f431c159c70 2024-03-07T06:53:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.302+0000 7f4321000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0b4cd0 con 0x7f431c070b10 2024-03-07T06:53:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.302+0000 7f4321000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c166fe0 con 0x7f431c0cf4c0 2024-03-07T06:53:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.302+0000 7f431a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f431c159c70 0x7f431c14ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.303+0000 7f431a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f431c159c70 0x7f431c14ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49052/0 (socket says 172.21.15.133:49052) 2024-03-07T06:53:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.303+0000 7f431a800640 1 -- 172.21.15.133:0/2336208521 learned_addr learned my addr 172.21.15.133:0/2336208521 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.303+0000 7f4319e00640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c070b10 0x7f431c0ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.303+0000 7f431a800640 1 -- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f431c0cf4c0 msgr2=0x7f431c155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.303+0000 7f4319400640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f431c0cf4c0 0x7f431c155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.303+0000 7f431a800640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f431c0cf4c0 0x7f431c155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.303+0000 7f431a800640 1 -- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c070b10 msgr2=0x7f431c0ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.303+0000 7f431a800640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c070b10 0x7f431c0ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.303+0000 7f431a800640 1 -- 172.21.15.133:0/2336208521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c0aecc0 con 0x7f431c159c70 2024-03-07T06:53:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.303+0000 7f4319400640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f431c0cf4c0 0x7f431c155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.303+0000 7f431a800640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f431c159c70 0x7f431c14ded0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4314037040 tx=0x7f4314004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.304+0000 7f4312000640 1 -- 172.21.15.133:0/2336208521 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314044030 con 0x7f431c159c70 2024-03-07T06:53:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.304+0000 7f4312000640 1 -- 172.21.15.133:0/2336208521 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431402ca60 con 0x7f431c159c70 2024-03-07T06:53:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.304+0000 7f4312000640 1 -- 172.21.15.133:0/2336208521 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314039040 con 0x7f431c159c70 2024-03-07T06:53:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.304+0000 7f4321000640 1 -- 172.21.15.133:0/2336208521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c0ad9c0 con 0x7f431c159c70 2024-03-07T06:53:15.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.304+0000 7f4321000640 1 -- 172.21.15.133:0/2336208521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c0adf80 con 0x7f431c159c70 2024-03-07T06:53:15.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.305+0000 7f4321000640 1 -- 172.21.15.133:0/2336208521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c062c90 con 0x7f431c159c70 2024-03-07T06:53:15.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.306+0000 7f4312000640 1 -- 172.21.15.133:0/2336208521 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4314047070 con 0x7f431c159c70 2024-03-07T06:53:15.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.307+0000 7f4312000640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f42fc06b500 0x7f42fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:15.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.307+0000 7f4312000640 1 -- 172.21.15.133:0/2336208521 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f43140ac880 con 0x7f431c159c70 2024-03-07T06:53:15.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.307+0000 7f4319e00640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f42fc06b500 0x7f42fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:15.308 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.309+0000 7f4319e00640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f42fc06b500 0x7f42fc06d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f430c000f60 tx=0x7f430c007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:15.311 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.312+0000 7f4312000640 1 -- 172.21.15.133:0/2336208521 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f431404e070 con 0x7f431c159c70 2024-03-07T06:53:15.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.508+0000 7f4321000640 1 -- 172.21.15.133:0/2336208521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f431c155c70 con 0x7f431c159c70 2024-03-07T06:53:15.515 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.515+0000 7f4312000640 1 -- 172.21.15.133:0/2336208521 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f431407b960 con 0x7f431c159c70 2024-03-07T06:53:15.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.518+0000 7f4321000640 1 -- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f42fc06b500 msgr2=0x7f42fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:15.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.518+0000 7f4321000640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f42fc06b500 0x7f42fc06d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f430c000f60 tx=0x7f430c007c00 comp rx=0 tx=0).stop 2024-03-07T06:53:15.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.518+0000 7f4321000640 1 -- 172.21.15.133:0/2336208521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f431c159c70 msgr2=0x7f431c14ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:15.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.518+0000 7f4321000640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f431c159c70 0x7f431c14ded0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4314037040 tx=0x7f4314004a40 comp rx=0 tx=0).stop 2024-03-07T06:53:15.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.518+0000 7f4321000640 1 -- 172.21.15.133:0/2336208521 shutdown_connections 2024-03-07T06:53:15.518 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.518+0000 7f4321000640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f431c159c70 0x7f431c14ded0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.518+0000 7f4321000640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f431c0cf4c0 0x7f431c155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.518+0000 7f4321000640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f42fc06b500 0x7f42fc06d9c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.518+0000 7f4321000640 1 --2- 172.21.15.133:0/2336208521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f431c070b10 0x7f431c0ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.518+0000 7f4321000640 1 -- 172.21.15.133:0/2336208521 >> 172.21.15.133:0/2336208521 conn(0x7f431c0cfb00 msgr2=0x7f431c0afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.518+0000 7f4321000640 1 -- 172.21.15.133:0/2336208521 shutdown_connections 2024-03-07T06:53:15.519 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.518+0000 7f4321000640 1 -- 172.21.15.133:0/2336208521 wait complete. 2024-03-07T06:53:15.544 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_strays.TestStrays.test_dir_deletion' 2024-03-07T06:53:15.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.728+0000 7f4a6f200640 1 Processor -- start 2024-03-07T06:53:15.728 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.728+0000 7f4a6f200640 1 -- start start 2024-03-07T06:53:15.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.729+0000 7f4a6f200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4a68073050 0x7f4a680724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:15.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.729+0000 7f4a6f200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a680cdf80 con 0x7f4a6815e390 2024-03-07T06:53:15.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.729+0000 7f4a6f200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a6815e770 con 0x7f4a68071d20 2024-03-07T06:53:15.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.729+0000 7f4a6f200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a680ce4d0 con 0x7f4a68073050 2024-03-07T06:53:15.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.730+0000 7f4a67e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f4a6815e390 0x7f4a68071610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52602/0 (socket says 172.21.15.133:52602) 2024-03-07T06:53:15.729 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.730+0000 7f4a67e00640 1 -- 172.21.15.133:0/1464115411 learned_addr learned my addr 172.21.15.133:0/1464115411 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:15.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.730+0000 7f4a67400640 1 -- 172.21.15.133:0/1464115411 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239689 0 0) 0x7f4a680cdf80 con 0x7f4a6815e390 2024-03-07T06:53:15.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.730+0000 7f4a6d200640 1 --2- 172.21.15.133:0/1464115411 >> v2:172.21.15.161:3300/0 conn(0x7f4a68073050 0x7f4a680724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.730+0000 7f4a67400640 1 -- 172.21.15.133:0/1464115411 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a4c003520 con 0x7f4a6815e390 2024-03-07T06:53:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a67400640 1 -- 172.21.15.133:0/1464115411 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2775918170 0 0) 0x7f4a4c003520 con 0x7f4a6815e390 2024-03-07T06:53:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a67400640 1 -- 172.21.15.133:0/1464115411 >> v2:172.21.15.161:3300/0 conn(0x7f4a68073050 msgr2=0x7f4a680724b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a67400640 1 --2- 172.21.15.133:0/1464115411 >> v2:172.21.15.161:3300/0 conn(0x7f4a68073050 0x7f4a680724b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a67400640 1 -- 172.21.15.133:0/1464115411 >> v1:172.21.15.161:6789/0 conn(0x7f4a68071d20 legacy=0x7f4a6806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a67400640 1 -- 172.21.15.133:0/1464115411 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a680b3d90 con 0x7f4a6815e390 2024-03-07T06:53:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a67400640 1 -- 172.21.15.133:0/1464115411 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f4a58001670 con 0x7f4a6815e390 2024-03-07T06:53:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a67400640 1 -- 172.21.15.133:0/1464115411 >> v1:172.21.15.133:6789/0 conn(0x7f4a6815e390 legacy=0x7f4a68071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a67400640 1 --2- 172.21.15.133:0/1464115411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a4c003a00 0x7f4a4c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a67400640 1 --2- 172.21.15.133:0/1464115411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a4c004800 0x7f4a4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a67400640 1 --2- 172.21.15.133:0/1464115411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c0051c0 0x7f4a4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a67400640 1 -- 172.21.15.133:0/1464115411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a680ce4d0 con 0x7f4a4c0051c0 2024-03-07T06:53:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a67400640 1 -- 172.21.15.133:0/1464115411 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c00c0c0 con 0x7f4a4c004800 2024-03-07T06:53:15.732 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a67400640 1 -- 172.21.15.133:0/1464115411 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c00c2c0 con 0x7f4a4c003a00 2024-03-07T06:53:15.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a6d200640 1 --2- 172.21.15.133:0/1464115411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c0051c0 0x7f4a4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:15.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a6d200640 1 -- 172.21.15.133:0/1464115411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a4c003a00 msgr2=0x7f4a4c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:15.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a6d200640 1 --2- 172.21.15.133:0/1464115411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a4c003a00 0x7f4a4c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a6d200640 1 -- 172.21.15.133:0/1464115411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a4c004800 msgr2=0x7f4a4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:15.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a6d200640 1 --2- 172.21.15.133:0/1464115411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a4c004800 0x7f4a4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.731+0000 7f4a6d200640 1 -- 172.21.15.133:0/1464115411 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c003520 con 0x7f4a4c0051c0 2024-03-07T06:53:15.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.732+0000 7f4a6d200640 1 -- 172.21.15.133:0/1464115411 reap_dead start 2024-03-07T06:53:15.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.733+0000 7f4a6d200640 1 --2- 172.21.15.133:0/1464115411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c0051c0 0x7f4a4c00b9c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009fc0 tx=0x7f4a4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710f99ceaa38dd68 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:15.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.733+0000 7f4a67400640 1 -- 172.21.15.133:0/1464115411 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60009c10 con 0x7f4a4c0051c0 2024-03-07T06:53:15.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.733+0000 7f4a67400640 1 -- 172.21.15.133:0/1464115411 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6002cca0 con 0x7f4a4c0051c0 2024-03-07T06:53:15.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.733+0000 7f4a67400640 1 -- 172.21.15.133:0/1464115411 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a60003bd0 con 0x7f4a4c0051c0 2024-03-07T06:53:15.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.733+0000 7f4a6f200640 1 -- 172.21.15.133:0/1464115411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c0051c0 msgr2=0x7f4a4c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:15.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.733+0000 7f4a6f200640 1 --2- 172.21.15.133:0/1464115411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c0051c0 0x7f4a4c00b9c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4a60009fc0 tx=0x7f4a4c005830 comp rx=0 tx=0).stop 2024-03-07T06:53:15.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.733+0000 7f4a6f200640 1 -- 172.21.15.133:0/1464115411 shutdown_connections 2024-03-07T06:53:15.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.733+0000 7f4a6f200640 1 --2- 172.21.15.133:0/1464115411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c0051c0 0x7f4a4c00b9c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.733+0000 7f4a6f200640 1 -- 172.21.15.133:0/1464115411 >> 172.21.15.133:0/1464115411 conn(0x7f4a680b3120 msgr2=0x7f4a680b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:15.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.734+0000 7f4a6f200640 1 -- 172.21.15.133:0/1464115411 shutdown_connections 2024-03-07T06:53:15.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.734+0000 7f4a6f200640 1 -- 172.21.15.133:0/1464115411 wait complete. 2024-03-07T06:53:15.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.735+0000 7f4a6f200640 1 Processor -- start 2024-03-07T06:53:15.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.735+0000 7f4a6f200640 1 -- start start 2024-03-07T06:53:15.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.736+0000 7f4a6f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c0051c0 0x7f4a680b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:15.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.736+0000 7f4a6f200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a681599f0 0x7f4a68152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:15.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.736+0000 7f4a6f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a68159e70 0x7f4a68152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:15.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.736+0000 7f4a6f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a680724b0 con 0x7f4a4c0051c0 2024-03-07T06:53:15.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.736+0000 7f4a6f200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a680b4070 con 0x7f4a68159e70 2024-03-07T06:53:15.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.736+0000 7f4a6f200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a68072630 con 0x7f4a681599f0 2024-03-07T06:53:15.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.737+0000 7f4a67e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c0051c0 0x7f4a680b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:15.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.737+0000 7f4a67e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c0051c0 0x7f4a680b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49068/0 (socket says 172.21.15.133:49068) 2024-03-07T06:53:15.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.737+0000 7f4a67e00640 1 -- 172.21.15.133:0/1272496868 learned_addr learned my addr 172.21.15.133:0/1272496868 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:15.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.737+0000 7f4a6d200640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a68159e70 0x7f4a68152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:15.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.737+0000 7f4a5fe00640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a681599f0 0x7f4a68152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:15.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.737+0000 7f4a67e00640 1 -- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a681599f0 msgr2=0x7f4a68152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:15.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.737+0000 7f4a67e00640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a681599f0 0x7f4a68152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.737+0000 7f4a67e00640 1 -- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a68159e70 msgr2=0x7f4a68152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:15.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.737+0000 7f4a67e00640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a68159e70 0x7f4a68152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:15.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.737+0000 7f4a67e00640 1 -- 172.21.15.133:0/1272496868 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6809c260 con 0x7f4a4c0051c0 2024-03-07T06:53:15.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.737+0000 7f4a6d200640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a68159e70 0x7f4a68152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:15.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.737+0000 7f4a5fe00640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a681599f0 0x7f4a68152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:15.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.737+0000 7f4a67e00640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c0051c0 0x7f4a680b5e30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c730 tx=0x7f4a5800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:15.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.738+0000 7f4a64c00640 1 -- 172.21.15.133:0/1272496868 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a58018070 con 0x7f4a4c0051c0 2024-03-07T06:53:15.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.738+0000 7f4a64c00640 1 -- 172.21.15.133:0/1272496868 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a580072d0 con 0x7f4a4c0051c0 2024-03-07T06:53:15.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.738+0000 7f4a64c00640 1 -- 172.21.15.133:0/1272496868 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a580047e0 con 0x7f4a4c0051c0 2024-03-07T06:53:15.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.738+0000 7f4a6f200640 1 -- 172.21.15.133:0/1272496868 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6809c540 con 0x7f4a4c0051c0 2024-03-07T06:53:15.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.738+0000 7f4a6f200640 1 -- 172.21.15.133:0/1272496868 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6809cb00 con 0x7f4a4c0051c0 2024-03-07T06:53:15.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.739+0000 7f4a6f200640 1 -- 172.21.15.133:0/1272496868 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a680b2df0 con 0x7f4a4c0051c0 2024-03-07T06:53:15.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.740+0000 7f4a64c00640 1 -- 172.21.15.133:0/1272496868 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4a5801b070 con 0x7f4a4c0051c0 2024-03-07T06:53:15.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.741+0000 7f4a64c00640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f4a4406b4d0 0x7f4a4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:15.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.741+0000 7f4a64c00640 1 -- 172.21.15.133:0/1272496868 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4a58089420 con 0x7f4a4c0051c0 2024-03-07T06:53:15.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.741+0000 7f4a5fe00640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f4a4406b4d0 0x7f4a4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:15.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.742+0000 7f4a5fe00640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f4a4406b4d0 0x7f4a4406d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4a50001660 tx=0x7f4a50009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:15.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:15.745+0000 7f4a64c00640 1 -- 172.21.15.133:0/1272496868 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4a680b2df0 con 0x7f4a4c0051c0 2024-03-07T06:53:16.008 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.009+0000 7f4a6f200640 1 -- 172.21.15.133:0/1272496868 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_strays.TestStrays.test_dir_deletion"]} v 0) v1 -- 0x7f4a68062c90 con 0x7f4a4c0051c0 2024-03-07T06:53:16.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.068+0000 7f4a64c00640 1 -- 172.21.15.133:0/1272496868 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_strays.TestStrays.test_dir_deletion"]}]=0 v131) v1 ==== 132+0+0 (secure 0 0 0) 0x7f4a58058630 con 0x7f4a4c0051c0 2024-03-07T06:53:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.071+0000 7f4a6f200640 1 -- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f4a4406b4d0 msgr2=0x7f4a4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.071+0000 7f4a6f200640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f4a4406b4d0 0x7f4a4406d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4a50001660 tx=0x7f4a50009040 comp rx=0 tx=0).stop 2024-03-07T06:53:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.071+0000 7f4a6f200640 1 -- 172.21.15.133:0/1272496868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c0051c0 msgr2=0x7f4a680b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.071+0000 7f4a6f200640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c0051c0 0x7f4a680b5e30 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800c730 tx=0x7f4a5800cbf0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.071+0000 7f4a6f200640 1 -- 172.21.15.133:0/1272496868 shutdown_connections 2024-03-07T06:53:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.071+0000 7f4a6f200640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f4a4406b4d0 0x7f4a4406d990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.071+0000 7f4a6f200640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a68159e70 0x7f4a68152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.071+0000 7f4a6f200640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a681599f0 0x7f4a68152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.071+0000 7f4a6f200640 1 --2- 172.21.15.133:0/1272496868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a4c0051c0 0x7f4a680b5e30 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.071+0000 7f4a6f200640 1 -- 172.21.15.133:0/1272496868 >> 172.21.15.133:0/1272496868 conn(0x7f4a680b3120 msgr2=0x7f4a6815c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:16.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.071+0000 7f4a6f200640 1 -- 172.21.15.133:0/1272496868 shutdown_connections 2024-03-07T06:53:16.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.071+0000 7f4a6f200640 1 -- 172.21.15.133:0/1272496868 wait complete. 2024-03-07T06:53:16.088 INFO:tasks.cephfs_test_runner:That when deleting a bunch of dentries and the containing ... ok 2024-03-07T06:53:16.089 INFO:tasks.cephfs_test_runner:Starting test: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) That the directory fragment size cannot exceed mds_bal_fragment_size_max (using a limit of 50 in all configurations). 2024-03-07T06:53:16.089 INFO:tasks.cephfs_test_runner:test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) 2024-03-07T06:53:16.089 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:53:16.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.285+0000 7fddb8a00640 1 Processor -- start 2024-03-07T06:53:16.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.285+0000 7fddb8a00640 1 -- start start 2024-03-07T06:53:16.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.287+0000 7fddb8a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb4071410 0x7fddb415cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:16.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.287+0000 7fddb8a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fddb406ee20 0x7fddb406f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:16.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.287+0000 7fddb8a00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb415d270 con 0x7fddb40717f0 2024-03-07T06:53:16.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.287+0000 7fddb8a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb415db80 con 0x7fddb406ee20 2024-03-07T06:53:16.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.287+0000 7fddb8a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb406dae0 con 0x7fddb4071410 2024-03-07T06:53:16.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.287+0000 7fddb1e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fddb40717f0 0x7fddb415d470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52614/0 (socket says 172.21.15.133:52614) 2024-03-07T06:53:16.289 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.287+0000 7fddb1e00640 1 -- 172.21.15.133:0/692706560 learned_addr learned my addr 172.21.15.133:0/692706560 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:16.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 -- 172.21.15.133:0/692706560 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947374637 0 0) 0x7fdda0001110 con 0x7fddb40717f0 2024-03-07T06:53:16.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb2800640 1 --2- 172.21.15.133:0/692706560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb4071410 0x7fddb415cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:16.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 -- 172.21.15.133:0/692706560 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd98003520 con 0x7fddb40717f0 2024-03-07T06:53:16.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb3200640 1 --2- 172.21.15.133:0/692706560 >> v2:172.21.15.161:3300/0 conn(0x7fddb406ee20 0x7fddb406f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:16.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 -- 172.21.15.133:0/692706560 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1947093046 0 0) 0x7fdd98003520 con 0x7fddb40717f0 2024-03-07T06:53:16.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 -- 172.21.15.133:0/692706560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb4071410 msgr2=0x7fddb415cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:16.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 --2- 172.21.15.133:0/692706560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb4071410 0x7fddb415cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 -- 172.21.15.133:0/692706560 >> v2:172.21.15.161:3300/0 conn(0x7fddb406ee20 msgr2=0x7fddb406f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:16.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 --2- 172.21.15.133:0/692706560 >> v2:172.21.15.161:3300/0 conn(0x7fddb406ee20 0x7fddb406f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 -- 172.21.15.133:0/692706560 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb406dd80 con 0x7fddb40717f0 2024-03-07T06:53:16.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 -- 172.21.15.133:0/692706560 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fdda0001840 con 0x7fddb40717f0 2024-03-07T06:53:16.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 -- 172.21.15.133:0/692706560 >> v1:172.21.15.133:6789/0 conn(0x7fddb40717f0 legacy=0x7fddb415d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:16.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 --2- 172.21.15.133:0/692706560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd980039b0 0x7fdd98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:16.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 --2- 172.21.15.133:0/692706560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb4071410 0x7fdd98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:16.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 --2- 172.21.15.133:0/692706560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd98004d40 0x7fdd980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:16.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 -- 172.21.15.133:0/692706560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb415db80 con 0x7fdd98004d40 2024-03-07T06:53:16.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 -- 172.21.15.133:0/692706560 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb406dae0 con 0x7fdd980039b0 2024-03-07T06:53:16.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.288+0000 7fddb1400640 1 -- 172.21.15.133:0/692706560 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9800b9c0 con 0x7fddb4071410 2024-03-07T06:53:16.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.289+0000 7fddb3200640 1 --2- 172.21.15.133:0/692706560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd98004d40 0x7fdd980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:16.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.289+0000 7fddb1e00640 1 --2- 172.21.15.133:0/692706560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb4071410 0x7fdd98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:16.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.289+0000 7fddb2800640 1 --2- 172.21.15.133:0/692706560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd980039b0 0x7fdd98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:16.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.289+0000 7fddb3200640 1 -- 172.21.15.133:0/692706560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb4071410 msgr2=0x7fdd98004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:16.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.289+0000 7fddb3200640 1 --2- 172.21.15.133:0/692706560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb4071410 0x7fdd98004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.289+0000 7fddb3200640 1 -- 172.21.15.133:0/692706560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd980039b0 msgr2=0x7fdd98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:16.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.289+0000 7fddb3200640 1 --2- 172.21.15.133:0/692706560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdd980039b0 0x7fdd98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.291 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.289+0000 7fddb3200640 1 -- 172.21.15.133:0/692706560 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd98003520 con 0x7fdd98004d40 2024-03-07T06:53:16.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.291+0000 7fddb3200640 1 --2- 172.21.15.133:0/692706560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd98004d40 0x7fdd980051c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fddac007390 tx=0x7fdd98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bf36788226b679 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:16.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.291+0000 7fddb1400640 1 -- 172.21.15.133:0/692706560 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac039070 con 0x7fdd98004d40 2024-03-07T06:53:16.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.291+0000 7fddb1400640 1 -- 172.21.15.133:0/692706560 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddac02ccf0 con 0x7fdd98004d40 2024-03-07T06:53:16.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.291+0000 7fddb1400640 1 -- 172.21.15.133:0/692706560 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac009be0 con 0x7fdd98004d40 2024-03-07T06:53:16.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.291+0000 7fddb8a00640 1 -- 172.21.15.133:0/692706560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd98004d40 msgr2=0x7fdd980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:16.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.291+0000 7fddb8a00640 1 --2- 172.21.15.133:0/692706560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd98004d40 0x7fdd980051c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fddac007390 tx=0x7fdd98005830 comp rx=0 tx=0).stop 2024-03-07T06:53:16.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.291+0000 7fddb3200640 1 -- 172.21.15.133:0/692706560 reap_dead start 2024-03-07T06:53:16.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.292+0000 7fddb8a00640 1 -- 172.21.15.133:0/692706560 shutdown_connections 2024-03-07T06:53:16.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.292+0000 7fddb8a00640 1 -- 172.21.15.133:0/692706560 >> 172.21.15.133:0/692706560 conn(0x7fddb4072330 msgr2=0x7fddb4072710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:16.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.292+0000 7fddb8a00640 1 -- 172.21.15.133:0/692706560 shutdown_connections 2024-03-07T06:53:16.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.292+0000 7fddb8a00640 1 -- 172.21.15.133:0/692706560 wait complete. 2024-03-07T06:53:16.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.293+0000 7fddb8a00640 1 Processor -- start 2024-03-07T06:53:16.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.293+0000 7fddb8a00640 1 -- start start 2024-03-07T06:53:16.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.294+0000 7fddb8a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddb415aa50 0x7fddb40cec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:16.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.294+0000 7fddb8a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb415ae30 0x7fddb41f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:16.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.294+0000 7fddb8a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb41f81f0 0x7fddb413f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:16.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.294+0000 7fddb8a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb406e620 con 0x7fddb41f81f0 2024-03-07T06:53:16.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.294+0000 7fddb8a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb406e340 con 0x7fddb415aa50 2024-03-07T06:53:16.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.294+0000 7fddb8a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb406e900 con 0x7fddb415ae30 2024-03-07T06:53:16.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.295+0000 7fddb3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb41f81f0 0x7fddb413f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:16.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.295+0000 7fddb3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb41f81f0 0x7fddb413f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49082/0 (socket says 172.21.15.133:49082) 2024-03-07T06:53:16.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.295+0000 7fddb3200640 1 -- 172.21.15.133:0/1979510018 learned_addr learned my addr 172.21.15.133:0/1979510018 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:16.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.295+0000 7fddb1e00640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb415ae30 0x7fddb41f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:16.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.295+0000 7fddb2800640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddb415aa50 0x7fddb40cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:16.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.295+0000 7fddb3200640 1 -- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb415ae30 msgr2=0x7fddb41f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:16.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.295+0000 7fddb3200640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb415ae30 0x7fddb41f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.295+0000 7fddb3200640 1 -- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddb415aa50 msgr2=0x7fddb40cec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:16.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.295+0000 7fddb3200640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddb415aa50 0x7fddb40cec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.295+0000 7fddb3200640 1 -- 172.21.15.133:0/1979510018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb40ce1b0 con 0x7fddb41f81f0 2024-03-07T06:53:16.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.295+0000 7fddb1e00640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb415ae30 0x7fddb41f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:16.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.295+0000 7fddb2800640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddb415aa50 0x7fddb40cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:16.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.296+0000 7fddb3200640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb41f81f0 0x7fddb413f910 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fddac002230 tx=0x7fddac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:16.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.296+0000 7fddaaa00640 1 -- 172.21.15.133:0/1979510018 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac039030 con 0x7fddb41f81f0 2024-03-07T06:53:16.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.296+0000 7fddaaa00640 1 -- 172.21.15.133:0/1979510018 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddac043070 con 0x7fddb41f81f0 2024-03-07T06:53:16.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.296+0000 7fddaaa00640 1 -- 172.21.15.133:0/1979510018 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac009be0 con 0x7fddb41f81f0 2024-03-07T06:53:16.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.296+0000 7fddb8a00640 1 -- 172.21.15.133:0/1979510018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb41f8730 con 0x7fddb41f81f0 2024-03-07T06:53:16.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.296+0000 7fddb8a00640 1 -- 172.21.15.133:0/1979510018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb40ce450 con 0x7fddb41f81f0 2024-03-07T06:53:16.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.297+0000 7fddb8a00640 1 -- 172.21.15.133:0/1979510018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb40aa2a0 con 0x7fddb41f81f0 2024-03-07T06:53:16.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.298+0000 7fddaaa00640 1 -- 172.21.15.133:0/1979510018 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fddac03d7f0 con 0x7fddb41f81f0 2024-03-07T06:53:16.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.299+0000 7fddaaa00640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdd9406b500 0x7fdd9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:16.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.300+0000 7fddaaa00640 1 -- 172.21.15.133:0/1979510018 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fddac0ac200 con 0x7fddb41f81f0 2024-03-07T06:53:16.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.300+0000 7fddb2800640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdd9406b500 0x7fdd9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:16.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.301+0000 7fddb2800640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdd9406b500 0x7fdd9406d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdda4009980 tx=0x7fdda4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:16.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.304+0000 7fddaaa00640 1 -- 172.21.15.133:0/1979510018 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fddb40aa2a0 con 0x7fddb41f81f0 2024-03-07T06:53:16.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.499+0000 7fddb8a00640 1 -- 172.21.15.133:0/1979510018 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddb4145370 con 0x7fddb41f81f0 2024-03-07T06:53:16.500 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.501+0000 7fddaaa00640 1 -- 172.21.15.133:0/1979510018 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+13070 (secure 0 0 0) 0x7fddac07b250 con 0x7fddb41f81f0 2024-03-07T06:53:16.501 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:16.501 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":38,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:53:15.054074+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6835/4067860709":"2024-03-08T06:53:12.036162+0000","172.21.15.133:6833/4067860709":"2024-03-08T06:53:12.036162+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:53:16.503 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.503+0000 7fddb8a00640 1 -- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdd9406b500 msgr2=0x7fdd9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:16.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.503+0000 7fddb8a00640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdd9406b500 0x7fdd9406d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdda4009980 tx=0x7fdda4008040 comp rx=0 tx=0).stop 2024-03-07T06:53:16.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.503+0000 7fddb8a00640 1 -- 172.21.15.133:0/1979510018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb41f81f0 msgr2=0x7fddb413f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:16.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.503+0000 7fddb8a00640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb41f81f0 0x7fddb413f910 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fddac002230 tx=0x7fddac004ab0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.504+0000 7fddb8a00640 1 -- 172.21.15.133:0/1979510018 shutdown_connections 2024-03-07T06:53:16.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.504+0000 7fddb8a00640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb41f81f0 0x7fddb413f910 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.504+0000 7fddb8a00640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fddb415ae30 0x7fddb41f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.504+0000 7fddb8a00640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fdd9406b500 0x7fdd9406d9c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.504+0000 7fddb8a00640 1 --2- 172.21.15.133:0/1979510018 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fddb415aa50 0x7fddb40cec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.504+0000 7fddb8a00640 1 -- 172.21.15.133:0/1979510018 >> 172.21.15.133:0/1979510018 conn(0x7fddb4072330 msgr2=0x7fddb406f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:16.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.504+0000 7fddb8a00640 1 -- 172.21.15.133:0/1979510018 shutdown_connections 2024-03-07T06:53:16.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.504+0000 7fddb8a00640 1 -- 172.21.15.133:0/1979510018 wait complete. 2024-03-07T06:53:16.521 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:53:16.522 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:53:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.731+0000 7f538c200640 1 Processor -- start 2024-03-07T06:53:16.731 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.731+0000 7f538c200640 1 -- start start 2024-03-07T06:53:16.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.733+0000 7f538c200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f538406eb00 0x7f53840b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:16.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.733+0000 7f538c200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538406f450 con 0x7f53840cf620 2024-03-07T06:53:16.733 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.733+0000 7f538c200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538406f170 con 0x7f53840aec80 2024-03-07T06:53:16.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.733+0000 7f538c200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538406ef10 con 0x7f538406eb00 2024-03-07T06:53:16.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.733+0000 7f538a200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f538406eb00 0x7f53840b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:16.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.733+0000 7f538a200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f538406eb00 0x7f53840b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49096/0 (socket says 172.21.15.133:49096) 2024-03-07T06:53:16.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.733+0000 7f538a200640 1 -- 172.21.15.133:0/689446640 learned_addr learned my addr 172.21.15.133:0/689446640 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:16.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.733+0000 7f5389800640 1 --1- 172.21.15.133:0/689446640 >> v1:172.21.15.133:6789/0 conn(0x7f53840cf620 0x7f53840ae570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52624/0 (socket says 172.21.15.133:52624) 2024-03-07T06:53:16.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.734+0000 7f538a200640 1 -- 172.21.15.133:0/689446640 >> v1:172.21.15.161:6789/0 conn(0x7f53840aec80 legacy=0x7f538406e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:16.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.734+0000 7f538a200640 1 -- 172.21.15.133:0/689446640 >> v1:172.21.15.133:6789/0 conn(0x7f53840cf620 legacy=0x7f53840ae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:16.734 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.734+0000 7f538a200640 1 -- 172.21.15.133:0/689446640 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5384071ad0 con 0x7f538406eb00 2024-03-07T06:53:16.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.735+0000 7f538a200640 1 --2- 172.21.15.133:0/689446640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f538406eb00 0x7f53840b2e50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f537c009e50 tx=0x7f537c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97647fdfb07efd96 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:16.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.736+0000 7f5383e00640 1 -- 172.21.15.133:0/689446640 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c02cb10 con 0x7f538406eb00 2024-03-07T06:53:16.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.736+0000 7f5383e00640 1 -- 172.21.15.133:0/689446640 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537c02ccb0 con 0x7f538406eb00 2024-03-07T06:53:16.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.736+0000 7f5383e00640 1 -- 172.21.15.133:0/689446640 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c02ce60 con 0x7f538406eb00 2024-03-07T06:53:16.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.736+0000 7f538c200640 1 -- 172.21.15.133:0/689446640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f538406eb00 msgr2=0x7f53840b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:16.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.736+0000 7f538c200640 1 --2- 172.21.15.133:0/689446640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f538406eb00 0x7f53840b2e50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f537c009e50 tx=0x7f537c02bf70 comp rx=0 tx=0).stop 2024-03-07T06:53:16.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.736+0000 7f538c200640 1 -- 172.21.15.133:0/689446640 shutdown_connections 2024-03-07T06:53:16.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.736+0000 7f538c200640 1 --2- 172.21.15.133:0/689446640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f538406eb00 0x7f53840b2e50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.736+0000 7f538c200640 1 -- 172.21.15.133:0/689446640 >> 172.21.15.133:0/689446640 conn(0x7f53840b1d30 msgr2=0x7f53840b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:16.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.736+0000 7f538c200640 1 -- 172.21.15.133:0/689446640 shutdown_connections 2024-03-07T06:53:16.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.736+0000 7f538c200640 1 -- 172.21.15.133:0/689446640 wait complete. 2024-03-07T06:53:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.738+0000 7f538c200640 1 Processor -- start 2024-03-07T06:53:16.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.739+0000 7f538c200640 1 -- start start 2024-03-07T06:53:16.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.740+0000 7f538c200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f538406eb00 0x7f538413d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:16.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.740+0000 7f538c200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53840aec80 0x7f53841e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:16.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.740+0000 7f538c200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53840cf620 0x7f53841e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:16.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.740+0000 7f538c200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53840725b0 con 0x7f53840aec80 2024-03-07T06:53:16.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.740+0000 7f538c200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5384072090 con 0x7f538406eb00 2024-03-07T06:53:16.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.740+0000 7f538c200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5384072730 con 0x7f53840cf620 2024-03-07T06:53:16.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.740+0000 7f5388e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53840aec80 0x7f53841e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:16.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.740+0000 7f5388e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53840aec80 0x7f53841e7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49102/0 (socket says 172.21.15.133:49102) 2024-03-07T06:53:16.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.740+0000 7f5388e00640 1 -- 172.21.15.133:0/2190113681 learned_addr learned my addr 172.21.15.133:0/2190113681 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:16.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.741+0000 7f538a200640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53840cf620 0x7f53841e7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:16.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.741+0000 7f5388e00640 1 -- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53840cf620 msgr2=0x7f53841e7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:16.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.741+0000 7f5388e00640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53840cf620 0x7f53841e7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.741+0000 7f5388e00640 1 -- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f538406eb00 msgr2=0x7f538413d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:16.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.741+0000 7f5389800640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f538406eb00 0x7f538413d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:16.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.741+0000 7f5388e00640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f538406eb00 0x7f538413d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.741+0000 7f5388e00640 1 -- 172.21.15.133:0/2190113681 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5384153650 con 0x7f53840aec80 2024-03-07T06:53:16.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.741+0000 7f5389800640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f538406eb00 0x7f538413d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:16.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.741+0000 7f5388e00640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53840aec80 0x7f53841e7a10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f537000c400 tx=0x7f537000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:16.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.741+0000 7f5381600640 1 -- 172.21.15.133:0/2190113681 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53700052d0 con 0x7f53840aec80 2024-03-07T06:53:16.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.741+0000 7f5381600640 1 -- 172.21.15.133:0/2190113681 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5370005490 con 0x7f53840aec80 2024-03-07T06:53:16.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.741+0000 7f5381600640 1 -- 172.21.15.133:0/2190113681 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5370007260 con 0x7f53840aec80 2024-03-07T06:53:16.742 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.741+0000 7f538c200640 1 -- 172.21.15.133:0/2190113681 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538413f7c0 con 0x7f53840aec80 2024-03-07T06:53:16.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.742+0000 7f538c200640 1 -- 172.21.15.133:0/2190113681 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538413fd80 con 0x7f53840aec80 2024-03-07T06:53:16.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.743+0000 7f538c200640 1 -- 172.21.15.133:0/2190113681 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5384062c90 con 0x7f53840aec80 2024-03-07T06:53:16.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.744+0000 7f5381600640 1 -- 172.21.15.133:0/2190113681 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5370007400 con 0x7f53840aec80 2024-03-07T06:53:16.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.745+0000 7f5381600640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f536406b4d0 0x7f536406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:16.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.745+0000 7f5381600640 1 -- 172.21.15.133:0/2190113681 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5370089580 con 0x7f53840aec80 2024-03-07T06:53:16.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.745+0000 7f5389800640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f536406b4d0 0x7f536406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:16.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.746+0000 7f5389800640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f536406b4d0 0x7f536406d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5378000f60 tx=0x7f5378009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:16.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.750+0000 7f5381600640 1 -- 172.21.15.133:0/2190113681 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f53700585d0 con 0x7f53840aec80 2024-03-07T06:53:16.931 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.931+0000 7f538c200640 1 -- 172.21.15.133:0/2190113681 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53840ae350 con 0x7f53840aec80 2024-03-07T06:53:16.932 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.932+0000 7f5381600640 1 -- 172.21.15.133:0/2190113681 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5370057ed0 con 0x7f53840aec80 2024-03-07T06:53:16.932 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:53:16.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.935+0000 7f538c200640 1 -- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f536406b4d0 msgr2=0x7f536406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:16.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.935+0000 7f538c200640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f536406b4d0 0x7f536406d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5378000f60 tx=0x7f5378009040 comp rx=0 tx=0).stop 2024-03-07T06:53:16.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.935+0000 7f538c200640 1 -- 172.21.15.133:0/2190113681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53840aec80 msgr2=0x7f53841e7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:16.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.935+0000 7f538c200640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53840aec80 0x7f53841e7a10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f537000c400 tx=0x7f537000c8c0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.935+0000 7f538c200640 1 -- 172.21.15.133:0/2190113681 shutdown_connections 2024-03-07T06:53:16.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.935+0000 7f538c200640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53840cf620 0x7f53841e7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.935+0000 7f538c200640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f53840aec80 0x7f53841e7a10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.935+0000 7f538c200640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f536406b4d0 0x7f536406d990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.935+0000 7f538c200640 1 --2- 172.21.15.133:0/2190113681 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f538406eb00 0x7f538413d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:16.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.935+0000 7f538c200640 1 -- 172.21.15.133:0/2190113681 >> 172.21.15.133:0/2190113681 conn(0x7f53840b1d30 msgr2=0x7f5384160020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:16.935 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.935+0000 7f538c200640 1 -- 172.21.15.133:0/2190113681 shutdown_connections 2024-03-07T06:53:16.936 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:16.935+0000 7f538c200640 1 -- 172.21.15.133:0/2190113681 wait complete. 2024-03-07T06:53:16.952 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_strays.TestStrays.test_dirfrag_limit' 2024-03-07T06:53:17.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.152+0000 7f6265e00640 1 Processor -- start 2024-03-07T06:53:17.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.153+0000 7f6265e00640 1 -- start start 2024-03-07T06:53:17.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.154+0000 7f6265e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6260072950 0x7f62600cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:17.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.154+0000 7f6265e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f62600cdee0 0x7f626015c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:17.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.154+0000 7f6265e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62600731b0 con 0x7f626015fbb0 2024-03-07T06:53:17.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.154+0000 7f6265e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6260072f60 con 0x7f62600cdee0 2024-03-07T06:53:17.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.154+0000 7f6265e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6260073490 con 0x7f6260072950 2024-03-07T06:53:17.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.154+0000 7f625f400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f62600cdee0 0x7f626015c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:17.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.155+0000 7f625f400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f62600cdee0 0x7f626015c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49110/0 (socket says 172.21.15.133:49110) 2024-03-07T06:53:17.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.155+0000 7f625f400640 1 -- 172.21.15.133:0/35822706 learned_addr learned my addr 172.21.15.133:0/35822706 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:17.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.155+0000 7f625e000640 1 --2- 172.21.15.133:0/35822706 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6260072950 0x7f62600cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:17.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.155+0000 7f625f400640 1 -- 172.21.15.133:0/35822706 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6260072950 msgr2=0x7f62600cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:17.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.155+0000 7f625f400640 1 --2- 172.21.15.133:0/35822706 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6260072950 0x7f62600cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:17.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.155+0000 7f625f400640 1 -- 172.21.15.133:0/35822706 >> v1:172.21.15.161:6789/0 conn(0x7f626015fbb0 legacy=0x7f626015ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:17.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.155+0000 7f625f400640 1 -- 172.21.15.133:0/35822706 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6260072150 con 0x7f62600cdee0 2024-03-07T06:53:17.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.156+0000 7f625f400640 1 --2- 172.21.15.133:0/35822706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62600cdee0 0x7f626015c550 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f62600730e0 tx=0x7f625802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3c0e298e9caa51 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:17.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.156+0000 7f625d600640 1 -- 172.21.15.133:0/35822706 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62580317c0 con 0x7f62600cdee0 2024-03-07T06:53:17.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.156+0000 7f625d600640 1 -- 172.21.15.133:0/35822706 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6258037070 con 0x7f62600cdee0 2024-03-07T06:53:17.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.156+0000 7f625d600640 1 -- 172.21.15.133:0/35822706 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258031b80 con 0x7f62600cdee0 2024-03-07T06:53:17.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.157+0000 7f6265e00640 1 -- 172.21.15.133:0/35822706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62600cdee0 msgr2=0x7f626015c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:17.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.157+0000 7f6265e00640 1 --2- 172.21.15.133:0/35822706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62600cdee0 0x7f626015c550 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f62600730e0 tx=0x7f625802b860 comp rx=0 tx=0).stop 2024-03-07T06:53:17.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.157+0000 7f6265e00640 1 -- 172.21.15.133:0/35822706 shutdown_connections 2024-03-07T06:53:17.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.157+0000 7f6265e00640 1 --2- 172.21.15.133:0/35822706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62600cdee0 0x7f626015c550 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:17.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.157+0000 7f6265e00640 1 --2- 172.21.15.133:0/35822706 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6260072950 0x7f62600cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:17.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.157+0000 7f6265e00640 1 -- 172.21.15.133:0/35822706 >> 172.21.15.133:0/35822706 conn(0x7f62600b3120 msgr2=0x7f62600b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:17.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.157+0000 7f6265e00640 1 -- 172.21.15.133:0/35822706 shutdown_connections 2024-03-07T06:53:17.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.157+0000 7f6265e00640 1 -- 172.21.15.133:0/35822706 wait complete. 2024-03-07T06:53:17.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.159+0000 7f6265e00640 1 Processor -- start 2024-03-07T06:53:17.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.159+0000 7f6265e00640 1 -- start start 2024-03-07T06:53:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.160+0000 7f6265e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62600cdee0 0x7f62601e59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.160+0000 7f6265e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f626015fbb0 0x7f62601e7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.160+0000 7f6265e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6260150450 0x7f62601e7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.160+0000 7f6265e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6260160980 con 0x7f62600cdee0 2024-03-07T06:53:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.160+0000 7f6265e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62600ce2f0 con 0x7f6260150450 2024-03-07T06:53:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.160+0000 7f6265e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62600b1a00 con 0x7f626015fbb0 2024-03-07T06:53:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.160+0000 7f625ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62600cdee0 0x7f62601e59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.160+0000 7f625ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62600cdee0 0x7f62601e59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49112/0 (socket says 172.21.15.133:49112) 2024-03-07T06:53:17.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.160+0000 7f625ea00640 1 -- 172.21.15.133:0/3716335227 learned_addr learned my addr 172.21.15.133:0/3716335227 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:17.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.161+0000 7f625ea00640 1 -- 172.21.15.133:0/3716335227 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f626015fbb0 msgr2=0x7f62601e7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:17.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.161+0000 7f625e000640 1 --2- 172.21.15.133:0/3716335227 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f626015fbb0 0x7f62601e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:17.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.161+0000 7f625ea00640 1 --2- 172.21.15.133:0/3716335227 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f626015fbb0 0x7f62601e7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:17.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.161+0000 7f625ea00640 1 -- 172.21.15.133:0/3716335227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6260150450 msgr2=0x7f62601e7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:17.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.161+0000 7f625ea00640 1 --2- 172.21.15.133:0/3716335227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6260150450 0x7f62601e7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:17.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.161+0000 7f625ea00640 1 -- 172.21.15.133:0/3716335227 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62601e5fb0 con 0x7f62600cdee0 2024-03-07T06:53:17.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.161+0000 7f625ea00640 1 --2- 172.21.15.133:0/3716335227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62600cdee0 0x7f62601e59d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f625000c7c0 tx=0x7f625000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:17.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.161+0000 7f6257400640 1 -- 172.21.15.133:0/3716335227 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250016020 con 0x7f62600cdee0 2024-03-07T06:53:17.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.161+0000 7f6257400640 1 -- 172.21.15.133:0/3716335227 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625000aa50 con 0x7f62600cdee0 2024-03-07T06:53:17.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.161+0000 7f6265e00640 1 -- 172.21.15.133:0/3716335227 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62601e6290 con 0x7f62600cdee0 2024-03-07T06:53:17.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.161+0000 7f6257400640 1 -- 172.21.15.133:0/3716335227 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6250012420 con 0x7f62600cdee0 2024-03-07T06:53:17.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.162+0000 7f6265e00640 1 -- 172.21.15.133:0/3716335227 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62601e6850 con 0x7f62600cdee0 2024-03-07T06:53:17.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.163+0000 7f6265e00640 1 -- 172.21.15.133:0/3716335227 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6260062c90 con 0x7f62600cdee0 2024-03-07T06:53:17.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.164+0000 7f6257400640 1 -- 172.21.15.133:0/3716335227 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6250019020 con 0x7f62600cdee0 2024-03-07T06:53:17.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.165+0000 7f6257400640 1 --2- 172.21.15.133:0/3716335227 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f623c06b4d0 0x7f623c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:17.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.165+0000 7f6257400640 1 -- 172.21.15.133:0/3716335227 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6250089ff0 con 0x7f62600cdee0 2024-03-07T06:53:17.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.166+0000 7f625e000640 1 --2- 172.21.15.133:0/3716335227 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f623c06b4d0 0x7f623c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:17.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.167+0000 7f625e000640 1 --2- 172.21.15.133:0/3716335227 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f623c06b4d0 0x7f623c06d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6248006fd0 tx=0x7f6248008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:17.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.170+0000 7f6257400640 1 -- 172.21.15.133:0/3716335227 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6250020030 con 0x7f62600cdee0 2024-03-07T06:53:17.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:17.436+0000 7f6265e00640 1 -- 172.21.15.133:0/3716335227 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_strays.TestStrays.test_dirfrag_limit"]} v 0) v1 -- 0x7f62601e7ca0 con 0x7f62600cdee0 2024-03-07T06:53:18.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.083+0000 7f6257400640 1 -- 172.21.15.133:0/3716335227 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_strays.TestStrays.test_dirfrag_limit"]}]=0 v133) v1 ==== 136+0+0 (secure 0 0 0) 0x7f6250059130 con 0x7f62600cdee0 2024-03-07T06:53:18.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.086+0000 7f6265e00640 1 -- 172.21.15.133:0/3716335227 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f623c06b4d0 msgr2=0x7f623c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:18.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.087+0000 7f6265e00640 1 --2- 172.21.15.133:0/3716335227 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f623c06b4d0 0x7f623c06d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6248006fd0 tx=0x7f6248008040 comp rx=0 tx=0).stop 2024-03-07T06:53:18.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.087+0000 7f6265e00640 1 -- 172.21.15.133:0/3716335227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62600cdee0 msgr2=0x7f62601e59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:18.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.087+0000 7f6265e00640 1 --2- 172.21.15.133:0/3716335227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62600cdee0 0x7f62601e59d0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f625000c7c0 tx=0x7f625000cc80 comp rx=0 tx=0).stop 2024-03-07T06:53:18.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.087+0000 7f6265e00640 1 -- 172.21.15.133:0/3716335227 shutdown_connections 2024-03-07T06:53:18.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.087+0000 7f6265e00640 1 --2- 172.21.15.133:0/3716335227 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f623c06b4d0 0x7f623c06d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.087+0000 7f6265e00640 1 --2- 172.21.15.133:0/3716335227 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6260150450 0x7f62601e7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.087+0000 7f6265e00640 1 --2- 172.21.15.133:0/3716335227 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f626015fbb0 0x7f62601e7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.087+0000 7f6265e00640 1 --2- 172.21.15.133:0/3716335227 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62600cdee0 0x7f62601e59d0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.087+0000 7f6265e00640 1 -- 172.21.15.133:0/3716335227 >> 172.21.15.133:0/3716335227 conn(0x7f62600b3120 msgr2=0x7f62600cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:18.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.087+0000 7f6265e00640 1 -- 172.21.15.133:0/3716335227 shutdown_connections 2024-03-07T06:53:18.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.087+0000 7f6265e00640 1 -- 172.21.15.133:0/3716335227 wait complete. 2024-03-07T06:53:18.108 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-07T06:53:18.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.319+0000 7f5691400640 1 Processor -- start 2024-03-07T06:53:18.319 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.319+0000 7f5691400640 1 -- start start 2024-03-07T06:53:18.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.321+0000 7f5691400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f568c159880 0x7f568c0cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:18.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.321+0000 7f5691400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b4f10 0x7f568c0b5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:18.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.321+0000 7f5691400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568c160720 con 0x7f568c0ccff0 2024-03-07T06:53:18.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.321+0000 7f5691400640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c0b58c0 con 0x7f568c159880 2024-03-07T06:53:18.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.321+0000 7f5691400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0cde00 con 0x7f568c0b4f10 2024-03-07T06:53:18.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.321+0000 7f568be00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f568c159880 0x7f568c0cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:18.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.321+0000 7f568be00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f568c159880 0x7f568c0cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49128/0 (socket says 172.21.15.133:49128) 2024-03-07T06:53:18.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.321+0000 7f568be00640 1 -- 172.21.15.133:0/3605051294 learned_addr learned my addr 172.21.15.133:0/3605051294 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:18.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.321+0000 7f5690a00640 1 --2- 172.21.15.133:0/3605051294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b4f10 0x7f568c0b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:18.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.322+0000 7f568be00640 1 -- 172.21.15.133:0/3605051294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b4f10 msgr2=0x7f568c0b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:18.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.322+0000 7f568be00640 1 --2- 172.21.15.133:0/3605051294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b4f10 0x7f568c0b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.322+0000 7f568be00640 1 -- 172.21.15.133:0/3605051294 >> v1:172.21.15.161:6789/0 conn(0x7f568c0ccff0 legacy=0x7f568c160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:18.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.322+0000 7f568be00640 1 -- 172.21.15.133:0/3605051294 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c0cd560 con 0x7f568c159880 2024-03-07T06:53:18.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.323+0000 7f568be00640 1 --2- 172.21.15.133:0/3605051294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f568c159880 0x7f568c0cf2b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f567c009870 tx=0x7f567c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b0281a69738c3f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:18.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.323+0000 7f568b400640 1 -- 172.21.15.133:0/3605051294 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c035020 con 0x7f568c159880 2024-03-07T06:53:18.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.323+0000 7f568b400640 1 -- 172.21.15.133:0/3605051294 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c031640 con 0x7f568c159880 2024-03-07T06:53:18.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.323+0000 7f568b400640 1 -- 172.21.15.133:0/3605051294 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c02d040 con 0x7f568c159880 2024-03-07T06:53:18.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.323+0000 7f5691400640 1 -- 172.21.15.133:0/3605051294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f568c159880 msgr2=0x7f568c0cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:18.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.323+0000 7f5691400640 1 --2- 172.21.15.133:0/3605051294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f568c159880 0x7f568c0cf2b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f567c009870 tx=0x7f567c02b8f0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.324+0000 7f5691400640 1 -- 172.21.15.133:0/3605051294 shutdown_connections 2024-03-07T06:53:18.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.324+0000 7f5691400640 1 --2- 172.21.15.133:0/3605051294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b4f10 0x7f568c0b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.324+0000 7f5691400640 1 --2- 172.21.15.133:0/3605051294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f568c159880 0x7f568c0cf2b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.324+0000 7f5691400640 1 -- 172.21.15.133:0/3605051294 >> 172.21.15.133:0/3605051294 conn(0x7f568c071860 msgr2=0x7f568c071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:18.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.324+0000 7f5691400640 1 -- 172.21.15.133:0/3605051294 shutdown_connections 2024-03-07T06:53:18.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.324+0000 7f5691400640 1 -- 172.21.15.133:0/3605051294 wait complete. 2024-03-07T06:53:18.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.325+0000 7f5691400640 1 Processor -- start 2024-03-07T06:53:18.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.326+0000 7f5691400640 1 -- start start 2024-03-07T06:53:18.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.326+0000 7f5691400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b4f10 0x7f568c14c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:18.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.326+0000 7f5691400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f568c0ccff0 0x7f568c0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:18.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.326+0000 7f5691400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f568c159880 0x7f568c0ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:18.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.326+0000 7f5691400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c15faf0 con 0x7f568c159880 2024-03-07T06:53:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.326+0000 7f5691400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0cdb20 con 0x7f568c0ccff0 2024-03-07T06:53:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.326+0000 7f5691400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c159c60 con 0x7f568c0b4f10 2024-03-07T06:53:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.327+0000 7f5690a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f568c159880 0x7f568c0ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.327+0000 7f5690a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f568c159880 0x7f568c0ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49140/0 (socket says 172.21.15.133:49140) 2024-03-07T06:53:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.327+0000 7f5690a00640 1 -- 172.21.15.133:0/426085721 learned_addr learned my addr 172.21.15.133:0/426085721 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.327+0000 7f568be00640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b4f10 0x7f568c14c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.327+0000 7f5683e00640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f568c0ccff0 0x7f568c0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.327+0000 7f5690a00640 1 -- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b4f10 msgr2=0x7f568c14c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.327+0000 7f5690a00640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b4f10 0x7f568c14c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.327+0000 7f5690a00640 1 -- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f568c0ccff0 msgr2=0x7f568c0adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.327+0000 7f5690a00640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f568c0ccff0 0x7f568c0adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.327+0000 7f5690a00640 1 -- 172.21.15.133:0/426085721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c0aadc0 con 0x7f568c159880 2024-03-07T06:53:18.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.327+0000 7f568be00640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b4f10 0x7f568c14c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:18.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.327+0000 7f5683e00640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f568c0ccff0 0x7f568c0adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:18.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.328+0000 7f5690a00640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f568c159880 0x7f568c0ae2e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f568400cc30 tx=0x7f568400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:18.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.328+0000 7f5688c00640 1 -- 172.21.15.133:0/426085721 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568400ce40 con 0x7f568c159880 2024-03-07T06:53:18.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.328+0000 7f5688c00640 1 -- 172.21.15.133:0/426085721 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5684003990 con 0x7f568c159880 2024-03-07T06:53:18.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.328+0000 7f5691400640 1 -- 172.21.15.133:0/426085721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c0aaae0 con 0x7f568c159880 2024-03-07T06:53:18.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.328+0000 7f5688c00640 1 -- 172.21.15.133:0/426085721 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684003d40 con 0x7f568c159880 2024-03-07T06:53:18.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.328+0000 7f5691400640 1 -- 172.21.15.133:0/426085721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c0ab7e0 con 0x7f568c159880 2024-03-07T06:53:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.329+0000 7f5691400640 1 -- 172.21.15.133:0/426085721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c067c80 con 0x7f568c159880 2024-03-07T06:53:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.330+0000 7f5688c00640 1 -- 172.21.15.133:0/426085721 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5684004000 con 0x7f568c159880 2024-03-07T06:53:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.331+0000 7f5688c00640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f566806b4d0 0x7f566806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:18.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.331+0000 7f5688c00640 1 -- 172.21.15.133:0/426085721 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f568408a440 con 0x7f568c159880 2024-03-07T06:53:18.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.332+0000 7f568be00640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f566806b4d0 0x7f566806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:18.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.333+0000 7f568be00640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f566806b4d0 0x7f566806d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f567c036000 tx=0x7f567c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:18.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.335+0000 7f5688c00640 1 -- 172.21.15.133:0/426085721 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5684014030 con 0x7f568c159880 2024-03-07T06:53:18.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.526+0000 7f5691400640 1 -- 172.21.15.133:0/426085721 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f568c0b4cd0 con 0x7f568c159880 2024-03-07T06:53:18.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.531+0000 7f5688c00640 1 -- 172.21.15.133:0/426085721 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5684059520 con 0x7f568c159880 2024-03-07T06:53:18.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.534+0000 7f5691400640 1 -- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f566806b4d0 msgr2=0x7f566806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:18.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.534+0000 7f5691400640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f566806b4d0 0x7f566806d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f567c036000 tx=0x7f567c008030 comp rx=0 tx=0).stop 2024-03-07T06:53:18.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.534+0000 7f5691400640 1 -- 172.21.15.133:0/426085721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f568c159880 msgr2=0x7f568c0ae2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:18.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.534+0000 7f5691400640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f568c159880 0x7f568c0ae2e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f568400cc30 tx=0x7f568400a5c0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.534+0000 7f5691400640 1 -- 172.21.15.133:0/426085721 shutdown_connections 2024-03-07T06:53:18.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.534+0000 7f5691400640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f568c159880 0x7f568c0ae2e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.534+0000 7f5691400640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f566806b4d0 0x7f566806d990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.534+0000 7f5691400640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f568c0ccff0 0x7f568c0adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.534 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.534+0000 7f5691400640 1 --2- 172.21.15.133:0/426085721 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f568c0b4f10 0x7f568c14c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:18.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.534+0000 7f5691400640 1 -- 172.21.15.133:0/426085721 >> 172.21.15.133:0/426085721 conn(0x7f568c071860 msgr2=0x7f568c0afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:18.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.534+0000 7f5691400640 1 -- 172.21.15.133:0/426085721 shutdown_connections 2024-03-07T06:53:18.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:18.535+0000 7f5691400640 1 -- 172.21.15.133:0/426085721 wait complete. 2024-03-07T06:53:18.553 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-07T06:53:18.553 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-07T06:53:18.661 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-07T06:53:18.795 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-07T06:53:18.795 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:53:18.795 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:18.852 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053417,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-07T06:53:18.852 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:53:18.852 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:18.910 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053429,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-07T06:53:18.910 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:53:18.910 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:18.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053417,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-07T06:53:18.965 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:53:18.965 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:19.022 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053429,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-07T06:53:19.022 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-07T06:53:19.022 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:53:19.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.238+0000 7f5c9ec00640 1 Processor -- start 2024-03-07T06:53:19.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.238+0000 7f5c9ec00640 1 -- start start 2024-03-07T06:53:19.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.240+0000 7f5c9ec00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5c98072950 0x7f5c980cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:19.239 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.240+0000 7f5c9ec00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c980731b0 con 0x7f5c980cdee0 2024-03-07T06:53:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.240+0000 7f5c9ec00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c98072150 con 0x7f5c9815fbb0 2024-03-07T06:53:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.240+0000 7f5c9ec00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c98072f60 con 0x7f5c98072950 2024-03-07T06:53:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.240+0000 7f5c97400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5c98072950 0x7f5c980cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.240+0000 7f5c9cc00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f5c980cdee0 0x7f5c9815c550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52636/0 (socket says 172.21.15.133:52636) 2024-03-07T06:53:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.240+0000 7f5c9cc00640 1 -- 172.21.15.133:0/3403713899 learned_addr learned my addr 172.21.15.133:0/3403713899 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:19.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.241+0000 7f5c96a00640 1 -- 172.21.15.133:0/3403713899 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802558646 0 0) 0x7f5c980731b0 con 0x7f5c980cdee0 2024-03-07T06:53:19.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.241+0000 7f5c96a00640 1 -- 172.21.15.133:0/3403713899 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c7c003520 con 0x7f5c980cdee0 2024-03-07T06:53:19.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.241+0000 7f5c97400640 1 -- 172.21.15.133:0/3403713899 >> v1:172.21.15.161:6789/0 conn(0x7f5c9815fbb0 legacy=0x7f5c9815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:19.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.241+0000 7f5c97400640 1 -- 172.21.15.133:0/3403713899 >> v1:172.21.15.133:6789/0 conn(0x7f5c980cdee0 legacy=0x7f5c9815c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:19.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.241+0000 7f5c97400640 1 -- 172.21.15.133:0/3403713899 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c98072430 con 0x7f5c98072950 2024-03-07T06:53:19.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.242+0000 7f5c97400640 1 --2- 172.21.15.133:0/3403713899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c98072950 0x7f5c980cf7d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5c980730e0 tx=0x7f5c8402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1ff970f3e5bacc server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:19.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.242+0000 7f5c96a00640 1 -- 172.21.15.133:0/3403713899 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84031780 con 0x7f5c98072950 2024-03-07T06:53:19.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.242+0000 7f5c96a00640 1 -- 172.21.15.133:0/3403713899 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c84037070 con 0x7f5c98072950 2024-03-07T06:53:19.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.242+0000 7f5c96a00640 1 -- 172.21.15.133:0/3403713899 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84031b20 con 0x7f5c98072950 2024-03-07T06:53:19.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.243+0000 7f5c9ec00640 1 -- 172.21.15.133:0/3403713899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c98072950 msgr2=0x7f5c980cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.243+0000 7f5c9ec00640 1 --2- 172.21.15.133:0/3403713899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c98072950 0x7f5c980cf7d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5c980730e0 tx=0x7f5c8402b820 comp rx=0 tx=0).stop 2024-03-07T06:53:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.243+0000 7f5c9ec00640 1 -- 172.21.15.133:0/3403713899 shutdown_connections 2024-03-07T06:53:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.243+0000 7f5c9ec00640 1 --2- 172.21.15.133:0/3403713899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c98072950 0x7f5c980cf7d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.243+0000 7f5c9ec00640 1 -- 172.21.15.133:0/3403713899 >> 172.21.15.133:0/3403713899 conn(0x7f5c980b3120 msgr2=0x7f5c980b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.243+0000 7f5c9ec00640 1 -- 172.21.15.133:0/3403713899 shutdown_connections 2024-03-07T06:53:19.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.243+0000 7f5c9ec00640 1 -- 172.21.15.133:0/3403713899 wait complete. 2024-03-07T06:53:19.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.245+0000 7f5c9ec00640 1 Processor -- start 2024-03-07T06:53:19.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.245+0000 7f5c9ec00640 1 -- start start 2024-03-07T06:53:19.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.246+0000 7f5c9ec00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c980cdee0 0x7f5c9814c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:19.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.246+0000 7f5c9ec00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9815fbb0 0x7f5c980c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.246+0000 7f5c9ec00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c980c94a0 0x7f5c980cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.246+0000 7f5c9ec00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c98160980 con 0x7f5c9815fbb0 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.246+0000 7f5c9ec00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c98073490 con 0x7f5c980cdee0 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.246+0000 7f5c9ec00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c980b1a00 con 0x7f5c980c94a0 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.246+0000 7f5c97400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9815fbb0 0x7f5c980c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.246+0000 7f5c97400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9815fbb0 0x7f5c980c9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49148/0 (socket says 172.21.15.133:49148) 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.246+0000 7f5c97400640 1 -- 172.21.15.133:0/2676503189 learned_addr learned my addr 172.21.15.133:0/2676503189 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.247+0000 7f5c97e00640 1 --2- 172.21.15.133:0/2676503189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c980cdee0 0x7f5c9814c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.247+0000 7f5c9cc00640 1 --2- 172.21.15.133:0/2676503189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c980c94a0 0x7f5c980cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.247+0000 7f5c97400640 1 -- 172.21.15.133:0/2676503189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c980c94a0 msgr2=0x7f5c980cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.247+0000 7f5c97400640 1 --2- 172.21.15.133:0/2676503189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c980c94a0 0x7f5c980cba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.247+0000 7f5c97400640 1 -- 172.21.15.133:0/2676503189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c980cdee0 msgr2=0x7f5c9814c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.247+0000 7f5c97400640 1 --2- 172.21.15.133:0/2676503189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c980cdee0 0x7f5c9814c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.247+0000 7f5c97400640 1 -- 172.21.15.133:0/2676503189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c980c9a60 con 0x7f5c9815fbb0 2024-03-07T06:53:19.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.247+0000 7f5c97400640 1 --2- 172.21.15.133:0/2676503189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9815fbb0 0x7f5c980c9f80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5c8402b860 tx=0x7f5c840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:19.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.247+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2676503189 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84032610 con 0x7f5c9815fbb0 2024-03-07T06:53:19.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.247+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2676503189 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c840327b0 con 0x7f5c9815fbb0 2024-03-07T06:53:19.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.247+0000 7f5c9ec00640 1 -- 172.21.15.133:0/2676503189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9809b8f0 con 0x7f5c9815fbb0 2024-03-07T06:53:19.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.248+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2676503189 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84032ba0 con 0x7f5c9815fbb0 2024-03-07T06:53:19.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.248+0000 7f5c9ec00640 1 -- 172.21.15.133:0/2676503189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9809beb0 con 0x7f5c9815fbb0 2024-03-07T06:53:19.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.249+0000 7f5c9ec00640 1 -- 172.21.15.133:0/2676503189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c98062c90 con 0x7f5c9815fbb0 2024-03-07T06:53:19.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.250+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2676503189 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5c840453f0 con 0x7f5c9815fbb0 2024-03-07T06:53:19.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.251+0000 7f5c8fe00640 1 --2- 172.21.15.133:0/2676503189 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5c7806b4d0 0x7f5c7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:19.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.251+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2676503189 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5c840abd30 con 0x7f5c9815fbb0 2024-03-07T06:53:19.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.251+0000 7f5c97e00640 1 --2- 172.21.15.133:0/2676503189 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5c7806b4d0 0x7f5c7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:19.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.252+0000 7f5c97e00640 1 --2- 172.21.15.133:0/2676503189 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5c7806b4d0 0x7f5c7806d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5c88004ea0 tx=0x7f5c88009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:19.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.256+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2676503189 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5c84040070 con 0x7f5c9815fbb0 2024-03-07T06:53:19.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.491+0000 7f5c9ec00640 1 -- 172.21.15.133:0/2676503189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c9806bb10 con 0x7f5c9815fbb0 2024-03-07T06:53:19.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.492+0000 7f5c8fe00640 1 -- 172.21.15.133:0/2676503189 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f5c8402f050 con 0x7f5c9815fbb0 2024-03-07T06:53:19.492 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:19.492 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4637,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3269394125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3269394125},{"type":"v1","addr":"172.21.15.161:6835","nonce":3269394125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4822,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/72243172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":72243172},{"type":"v1","addr":"172.21.15.161:6837","nonce":72243172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5097,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6834/3714291615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":3714291615},{"type":"v1","addr":"172.21.15.133:6834","nonce":3714291615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5490,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/686057584","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6833","nonce":686057584},{"type":"v1","addr":"172.21.15.133:6835","nonce":686057584}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2024-03-07T06:53:19.492 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 20 2024-03-07T06:53:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.495+0000 7f5c9ec00640 1 -- 172.21.15.133:0/2676503189 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5c7806b4d0 msgr2=0x7f5c7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.495+0000 7f5c9ec00640 1 --2- 172.21.15.133:0/2676503189 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5c7806b4d0 0x7f5c7806d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5c88004ea0 tx=0x7f5c88009040 comp rx=0 tx=0).stop 2024-03-07T06:53:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.495+0000 7f5c9ec00640 1 -- 172.21.15.133:0/2676503189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9815fbb0 msgr2=0x7f5c980c9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.495+0000 7f5c9ec00640 1 --2- 172.21.15.133:0/2676503189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9815fbb0 0x7f5c980c9f80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5c8402b860 tx=0x7f5c840046c0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.495+0000 7f5c9ec00640 1 -- 172.21.15.133:0/2676503189 shutdown_connections 2024-03-07T06:53:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.495+0000 7f5c9ec00640 1 --2- 172.21.15.133:0/2676503189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5c980c94a0 0x7f5c980cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.495+0000 7f5c9ec00640 1 --2- 172.21.15.133:0/2676503189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c9815fbb0 0x7f5c980c9f80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.495+0000 7f5c9ec00640 1 --2- 172.21.15.133:0/2676503189 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5c7806b4d0 0x7f5c7806d990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.495+0000 7f5c9ec00640 1 --2- 172.21.15.133:0/2676503189 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5c980cdee0 0x7f5c9814c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.495+0000 7f5c9ec00640 1 -- 172.21.15.133:0/2676503189 >> 172.21.15.133:0/2676503189 conn(0x7f5c980b3120 msgr2=0x7f5c9815e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.495+0000 7f5c9ec00640 1 -- 172.21.15.133:0/2676503189 shutdown_connections 2024-03-07T06:53:19.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.495+0000 7f5c9ec00640 1 -- 172.21.15.133:0/2676503189 wait complete. 2024-03-07T06:53:19.515 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-07T06:53:19.515 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-07T06:53:19.515 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-07T06:53:19.515 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:53:19.515 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-07T06:53:19.515 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-07T06:53:19.516 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-07T06:53:19.516 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:53:19.516 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-07T06:53:19.516 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-07T06:53:19.516 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-07T06:53:19.516 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:53:19.516 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-07T06:53:19.516 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-07T06:53:19.517 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-07T06:53:19.517 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:53:19.533 INFO:tasks.ceph.mds.c:Stopped 2024-03-07T06:53:19.534 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-07T06:53:19.538 INFO:tasks.ceph.mds.a:Stopped 2024-03-07T06:53:19.538 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-07T06:53:19.539 INFO:tasks.ceph.mds.d:Stopped 2024-03-07T06:53:19.539 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-07T06:53:19.540 INFO:tasks.ceph.mds.b:Stopped 2024-03-07T06:53:19.540 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-07T06:53:19.584 INFO:tasks.ceph.mds.c:Started 2024-03-07T06:53:19.587 DEBUG:teuthology.parallel:result is None 2024-03-07T06:53:19.587 INFO:tasks.ceph.mds.d:Started 2024-03-07T06:53:19.587 DEBUG:teuthology.parallel:result is None 2024-03-07T06:53:19.590 INFO:tasks.ceph.mds.a:Started 2024-03-07T06:53:19.590 DEBUG:teuthology.parallel:result is None 2024-03-07T06:53:19.593 INFO:tasks.ceph.mds.b:Started 2024-03-07T06:53:19.593 DEBUG:teuthology.parallel:result is None 2024-03-07T06:53:19.594 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-07T06:53:19.737 INFO:tasks.ceph.mds.c.smithi133.stdout:starting mds.c at 2024-03-07T06:53:19.745 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-03-07T06:53:19.752 INFO:tasks.ceph.mds.d.smithi161.stdout:starting mds.d at 2024-03-07T06:53:19.760 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2024-03-07T06:53:19.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.780+0000 7fb7b0800640 1 Processor -- start 2024-03-07T06:53:19.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.780+0000 7fb7b0800640 1 -- start start 2024-03-07T06:53:19.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7b0800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb7a80b6120 0x7fb7a8070e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:19.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7b0800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7a80b43f0 0x7fb7a80b6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:19.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7b0800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7a806e3c0 0x7fb7a806e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:19.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7b0800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a8071390 con 0x7fb7a80b43f0 2024-03-07T06:53:19.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7b0800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a8147ea0 con 0x7fb7a80b6120 2024-03-07T06:53:19.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7b0800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80b6a40 con 0x7fb7a806e3c0 2024-03-07T06:53:19.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7ad400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7a80b43f0 0x7fb7a80b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:19.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7ad400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb7a80b43f0 0x7fb7a80b6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49174/0 (socket says 172.21.15.133:49174) 2024-03-07T06:53:19.782 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7ad400640 1 -- 172.21.15.133:0/2857573596 learned_addr learned my addr 172.21.15.133:0/2857573596 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:19.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7ae800640 1 --2- 172.21.15.133:0/2857573596 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7a806e3c0 0x7fb7a806e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:19.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7ade00640 1 --2- 172.21.15.133:0/2857573596 >> v2:172.21.15.161:3300/0 conn(0x7fb7a80b6120 0x7fb7a8070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:19.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7ad400640 1 -- 172.21.15.133:0/2857573596 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7a806e3c0 msgr2=0x7fb7a806e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:19.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7ad400640 1 --2- 172.21.15.133:0/2857573596 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7a806e3c0 0x7fb7a806e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7ad400640 1 -- 172.21.15.133:0/2857573596 >> v2:172.21.15.161:3300/0 conn(0x7fb7a80b6120 msgr2=0x7fb7a8070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:19.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7ad400640 1 --2- 172.21.15.133:0/2857573596 >> v2:172.21.15.161:3300/0 conn(0x7fb7a80b6120 0x7fb7a8070e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.782+0000 7fb7ad400640 1 -- 172.21.15.133:0/2857573596 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a80b4960 con 0x7fb7a80b43f0 2024-03-07T06:53:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.783+0000 7fb7ad400640 1 --2- 172.21.15.133:0/2857573596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a80b43f0 0x7fb7a80b6500 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8073c90 tx=0x7fb79802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc21ce2e2e4a331c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.784+0000 7fb7aca00640 1 -- 172.21.15.133:0/2857573596 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798035020 con 0x7fb7a80b43f0 2024-03-07T06:53:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.784+0000 7fb7aca00640 1 -- 172.21.15.133:0/2857573596 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79802d040 con 0x7fb7a80b43f0 2024-03-07T06:53:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.784+0000 7fb7aca00640 1 -- 172.21.15.133:0/2857573596 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798031a80 con 0x7fb7a80b43f0 2024-03-07T06:53:19.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.784+0000 7fb7b0800640 1 -- 172.21.15.133:0/2857573596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a80b43f0 msgr2=0x7fb7a80b6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:19.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.784+0000 7fb7b0800640 1 --2- 172.21.15.133:0/2857573596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a80b43f0 0x7fb7a80b6500 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8073c90 tx=0x7fb79802b900 comp rx=0 tx=0).stop 2024-03-07T06:53:19.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.784+0000 7fb7b0800640 1 -- 172.21.15.133:0/2857573596 shutdown_connections 2024-03-07T06:53:19.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.784+0000 7fb7b0800640 1 --2- 172.21.15.133:0/2857573596 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7a806e3c0 0x7fb7a806e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.784+0000 7fb7b0800640 1 --2- 172.21.15.133:0/2857573596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a80b43f0 0x7fb7a80b6500 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.784+0000 7fb7b0800640 1 --2- 172.21.15.133:0/2857573596 >> v2:172.21.15.161:3300/0 conn(0x7fb7a80b6120 0x7fb7a8070e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.784+0000 7fb7b0800640 1 -- 172.21.15.133:0/2857573596 >> 172.21.15.133:0/2857573596 conn(0x7fb7a8071550 msgr2=0x7fb7a8071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:19.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.785+0000 7fb7b0800640 1 -- 172.21.15.133:0/2857573596 shutdown_connections 2024-03-07T06:53:19.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.785+0000 7fb7b0800640 1 -- 172.21.15.133:0/2857573596 wait complete. 2024-03-07T06:53:19.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.786+0000 7fb7b0800640 1 Processor -- start 2024-03-07T06:53:19.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.786+0000 7fb7b0800640 1 -- start start 2024-03-07T06:53:19.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.787+0000 7fb7b0800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a806e3c0 0x7fb7a80b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:19.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.787+0000 7fb7b0800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7a80b43f0 0x7fb7a80b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:19.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.787+0000 7fb7b0800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7a80b6120 0x7fb7a80c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:19.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.787+0000 7fb7b0800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80b5200 con 0x7fb7a806e3c0 2024-03-07T06:53:19.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.787+0000 7fb7b0800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80b4f20 con 0x7fb7a80b43f0 2024-03-07T06:53:19.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.787+0000 7fb7b0800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a806ed90 con 0x7fb7a80b6120 2024-03-07T06:53:19.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.788+0000 7fb7ade00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a806e3c0 0x7fb7a80b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:19.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.788+0000 7fb7ade00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a806e3c0 0x7fb7a80b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49176/0 (socket says 172.21.15.133:49176) 2024-03-07T06:53:19.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.788+0000 7fb7ade00640 1 -- 172.21.15.133:0/3081174641 learned_addr learned my addr 172.21.15.133:0/3081174641 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:19.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.788+0000 7fb7ae800640 1 --2- 172.21.15.133:0/3081174641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7a80b6120 0x7fb7a80c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:19.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.788+0000 7fb7ad400640 1 --2- 172.21.15.133:0/3081174641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7a80b43f0 0x7fb7a80b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:19.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.788+0000 7fb7ade00640 1 -- 172.21.15.133:0/3081174641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7a80b6120 msgr2=0x7fb7a80c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:19.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.788+0000 7fb7ade00640 1 --2- 172.21.15.133:0/3081174641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7a80b6120 0x7fb7a80c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.788+0000 7fb7ade00640 1 -- 172.21.15.133:0/3081174641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7a80b43f0 msgr2=0x7fb7a80b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:19.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.788+0000 7fb7ade00640 1 --2- 172.21.15.133:0/3081174641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7a80b43f0 0x7fb7a80b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:19.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.788+0000 7fb7ade00640 1 -- 172.21.15.133:0/3081174641 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a80b1ef0 con 0x7fb7a806e3c0 2024-03-07T06:53:19.789 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.788+0000 7fb7ade00640 1 --2- 172.21.15.133:0/3081174641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a806e3c0 0x7fb7a80b3290 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00d5f0 tx=0x7fb79c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:19.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.789+0000 7fb7a2000640 1 -- 172.21.15.133:0/3081174641 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c0125f0 con 0x7fb7a806e3c0 2024-03-07T06:53:19.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.789+0000 7fb7a2000640 1 -- 172.21.15.133:0/3081174641 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c019070 con 0x7fb7a806e3c0 2024-03-07T06:53:19.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.789+0000 7fb7a2000640 1 -- 172.21.15.133:0/3081174641 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c0129d0 con 0x7fb7a806e3c0 2024-03-07T06:53:19.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.789+0000 7fb7b0800640 1 -- 172.21.15.133:0/3081174641 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a80b2750 con 0x7fb7a806e3c0 2024-03-07T06:53:19.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.789+0000 7fb7b0800640 1 -- 172.21.15.133:0/3081174641 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a80b2470 con 0x7fb7a806e3c0 2024-03-07T06:53:19.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.790+0000 7fb7b0800640 1 -- 172.21.15.133:0/3081174641 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a8062c90 con 0x7fb7a806e3c0 2024-03-07T06:53:19.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.791+0000 7fb7a2000640 1 -- 172.21.15.133:0/3081174641 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb79c012b70 con 0x7fb7a806e3c0 2024-03-07T06:53:19.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.792+0000 7fb7a2000640 1 --2- 172.21.15.133:0/3081174641 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb78c06b520 0x7fb78c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:19.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.792+0000 7fb7a2000640 1 -- 172.21.15.133:0/3081174641 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb79c08a180 con 0x7fb7a806e3c0 2024-03-07T06:53:19.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.792+0000 7fb7ad400640 1 --2- 172.21.15.133:0/3081174641 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb78c06b520 0x7fb78c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:19.792 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.793+0000 7fb7ad400640 1 --2- 172.21.15.133:0/3081174641 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb78c06b520 0x7fb78c06d9e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb798036000 tx=0x7fb79803c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:19.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:19.796+0000 7fb7a2000640 1 -- 172.21.15.133:0/3081174641 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb79c058d20 con 0x7fb7a806e3c0 2024-03-07T06:53:20.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.002+0000 7fb7b0800640 1 -- 172.21.15.133:0/3081174641 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb7a806c940 con 0x7fb7a806e3c0 2024-03-07T06:53:20.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.113+0000 7fb7a2000640 1 -- 172.21.15.133:0/3081174641 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb79c05cdc0 con 0x7fb7a806e3c0 2024-03-07T06:53:20.113 INFO:teuthology.orchestra.run.smithi133.stderr: removed all blocklist entries 2024-03-07T06:53:20.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.117+0000 7fb7b0800640 1 -- 172.21.15.133:0/3081174641 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb78c06b520 msgr2=0x7fb78c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:20.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.117+0000 7fb7b0800640 1 --2- 172.21.15.133:0/3081174641 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb78c06b520 0x7fb78c06d9e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb798036000 tx=0x7fb79803c040 comp rx=0 tx=0).stop 2024-03-07T06:53:20.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.117+0000 7fb7b0800640 1 -- 172.21.15.133:0/3081174641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a806e3c0 msgr2=0x7fb7a80b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:20.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.117+0000 7fb7b0800640 1 --2- 172.21.15.133:0/3081174641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a806e3c0 0x7fb7a80b3290 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fb79c00d5f0 tx=0x7fb79c00dab0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.117+0000 7fb7b0800640 1 -- 172.21.15.133:0/3081174641 shutdown_connections 2024-03-07T06:53:20.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.117+0000 7fb7b0800640 1 --2- 172.21.15.133:0/3081174641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7a80b6120 0x7fb7a80c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.117+0000 7fb7b0800640 1 --2- 172.21.15.133:0/3081174641 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb78c06b520 0x7fb78c06d9e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.117+0000 7fb7b0800640 1 --2- 172.21.15.133:0/3081174641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7a80b43f0 0x7fb7a80b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.117+0000 7fb7b0800640 1 --2- 172.21.15.133:0/3081174641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7a806e3c0 0x7fb7a80b3290 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.117+0000 7fb7b0800640 1 -- 172.21.15.133:0/3081174641 >> 172.21.15.133:0/3081174641 conn(0x7fb7a8071550 msgr2=0x7fb7a8146920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:20.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.118+0000 7fb7b0800640 1 -- 172.21.15.133:0/3081174641 shutdown_connections 2024-03-07T06:53:20.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.118+0000 7fb7b0800640 1 -- 172.21.15.133:0/3081174641 wait complete. 2024-03-07T06:53:20.139 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-07T06:53:20.337 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.337+0000 7fe451000640 1 Processor -- start 2024-03-07T06:53:20.338 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.338+0000 7fe451000640 1 -- start start 2024-03-07T06:53:20.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.339+0000 7fe451000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe44c15fbb0 0x7fe44c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.339+0000 7fe451000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe44c072950 0x7fe44c0cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.339+0000 7fe451000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44c0731b0 con 0x7fe44c15ff90 2024-03-07T06:53:20.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.339+0000 7fe451000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe44c072f60 con 0x7fe44c15fbb0 2024-03-07T06:53:20.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.339+0000 7fe451000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c073490 con 0x7fe44c072950 2024-03-07T06:53:20.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.340+0000 7fe44a800640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe44c15ff90 0x7fe44c0acd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:52658/0 (socket says 172.21.15.133:52658) 2024-03-07T06:53:20.340 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.340+0000 7fe44a800640 1 -- 172.21.15.133:0/2482865152 learned_addr learned my addr 172.21.15.133:0/2482865152 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:20.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.340+0000 7fe449400640 1 --2- 172.21.15.133:0/2482865152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe44c072950 0x7fe44c0cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:20.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.340+0000 7fe449e00640 1 --2- 172.21.15.133:0/2482865152 >> v2:172.21.15.161:3300/0 conn(0x7fe44c15fbb0 0x7fe44c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:20.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.340+0000 7fe448a00640 1 -- 172.21.15.133:0/2482865152 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647289595 0 0) 0x7fe44c0731b0 con 0x7fe44c15ff90 2024-03-07T06:53:20.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.340+0000 7fe448a00640 1 -- 172.21.15.133:0/2482865152 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe42c003520 con 0x7fe44c15ff90 2024-03-07T06:53:20.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe448a00640 1 -- 172.21.15.133:0/2482865152 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1760665142 0 0) 0x7fe42c003520 con 0x7fe44c15ff90 2024-03-07T06:53:20.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe448a00640 1 -- 172.21.15.133:0/2482865152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe44c072950 msgr2=0x7fe44c0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe448a00640 1 --2- 172.21.15.133:0/2482865152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe44c072950 0x7fe44c0cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe448a00640 1 -- 172.21.15.133:0/2482865152 >> v2:172.21.15.161:3300/0 conn(0x7fe44c15fbb0 msgr2=0x7fe44c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe448a00640 1 --2- 172.21.15.133:0/2482865152 >> v2:172.21.15.161:3300/0 conn(0x7fe44c15fbb0 0x7fe44c0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe448a00640 1 -- 172.21.15.133:0/2482865152 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c072150 con 0x7fe44c15ff90 2024-03-07T06:53:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe448a00640 1 -- 172.21.15.133:0/2482865152 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fe444001670 con 0x7fe44c15ff90 2024-03-07T06:53:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe448a00640 1 -- 172.21.15.133:0/2482865152 >> v1:172.21.15.133:6789/0 conn(0x7fe44c15ff90 legacy=0x7fe44c0acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe448a00640 1 --2- 172.21.15.133:0/2482865152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42c0039b0 0x7fe42c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe448a00640 1 --2- 172.21.15.133:0/2482865152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42c004800 0x7fe42c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe448a00640 1 --2- 172.21.15.133:0/2482865152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe42c0051c0 0x7fe42c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe448a00640 1 -- 172.21.15.133:0/2482865152 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c072f60 con 0x7fe42c0039b0 2024-03-07T06:53:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe448a00640 1 -- 172.21.15.133:0/2482865152 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c073490 con 0x7fe42c004800 2024-03-07T06:53:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe448a00640 1 -- 172.21.15.133:0/2482865152 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42c00bf00 con 0x7fe42c0051c0 2024-03-07T06:53:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe449e00640 1 --2- 172.21.15.133:0/2482865152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42c0039b0 0x7fe42c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:20.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe449e00640 1 -- 172.21.15.133:0/2482865152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe42c0051c0 msgr2=0x7fe42c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:20.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe449400640 1 --2- 172.21.15.133:0/2482865152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42c004800 0x7fe42c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:20.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.341+0000 7fe44a800640 1 --2- 172.21.15.133:0/2482865152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe42c0051c0 0x7fe42c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:20.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.342+0000 7fe449e00640 1 --2- 172.21.15.133:0/2482865152 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe42c0051c0 0x7fe42c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.342+0000 7fe449e00640 1 -- 172.21.15.133:0/2482865152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42c004800 msgr2=0x7fe42c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:20.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.342+0000 7fe449e00640 1 --2- 172.21.15.133:0/2482865152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42c004800 0x7fe42c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.342+0000 7fe449e00640 1 -- 172.21.15.133:0/2482865152 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c003520 con 0x7fe42c0039b0 2024-03-07T06:53:20.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.342+0000 7fe449400640 1 --2- 172.21.15.133:0/2482865152 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe42c004800 0x7fe42c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:20.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.343+0000 7fe449e00640 1 --2- 172.21.15.133:0/2482865152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42c0039b0 0x7fe42c003df0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009ac0 tx=0x7fe42c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1beb2571665695f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:20.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.343+0000 7fe448a00640 1 -- 172.21.15.133:0/2482865152 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c039070 con 0x7fe42c0039b0 2024-03-07T06:53:20.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.343+0000 7fe448a00640 1 -- 172.21.15.133:0/2482865152 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43c02c710 con 0x7fe42c0039b0 2024-03-07T06:53:20.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.343+0000 7fe448a00640 1 -- 172.21.15.133:0/2482865152 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c009690 con 0x7fe42c0039b0 2024-03-07T06:53:20.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.343+0000 7fe451000640 1 -- 172.21.15.133:0/2482865152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42c0039b0 msgr2=0x7fe42c003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:20.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.343+0000 7fe451000640 1 --2- 172.21.15.133:0/2482865152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42c0039b0 0x7fe42c003df0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009ac0 tx=0x7fe42c005830 comp rx=0 tx=0).stop 2024-03-07T06:53:20.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.343+0000 7fe44a800640 1 -- 172.21.15.133:0/2482865152 reap_dead start 2024-03-07T06:53:20.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.343+0000 7fe451000640 1 -- 172.21.15.133:0/2482865152 shutdown_connections 2024-03-07T06:53:20.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.343+0000 7fe451000640 1 -- 172.21.15.133:0/2482865152 >> 172.21.15.133:0/2482865152 conn(0x7fe44c0b3120 msgr2=0x7fe44c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:20.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.343+0000 7fe451000640 1 -- 172.21.15.133:0/2482865152 shutdown_connections 2024-03-07T06:53:20.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.344+0000 7fe451000640 1 -- 172.21.15.133:0/2482865152 wait complete. 2024-03-07T06:53:20.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.345+0000 7fe451000640 1 Processor -- start 2024-03-07T06:53:20.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.345+0000 7fe451000640 1 -- start start 2024-03-07T06:53:20.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.346+0000 7fe451000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42c0039b0 0x7fe44c1f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.346+0000 7fe451000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe44c09d4c0 0x7fe44c154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.346+0000 7fe451000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe44c156380 0x7fe44c0acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.346+0000 7fe451000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c0b1a00 con 0x7fe42c0039b0 2024-03-07T06:53:20.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.346+0000 7fe451000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c160980 con 0x7fe44c09d4c0 2024-03-07T06:53:20.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.346+0000 7fe451000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c1603e0 con 0x7fe44c156380 2024-03-07T06:53:20.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.347+0000 7fe449e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42c0039b0 0x7fe44c1f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:20.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.347+0000 7fe449e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42c0039b0 0x7fe44c1f4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49192/0 (socket says 172.21.15.133:49192) 2024-03-07T06:53:20.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.347+0000 7fe449e00640 1 -- 172.21.15.133:0/3361468268 learned_addr learned my addr 172.21.15.133:0/3361468268 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:20.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.347+0000 7fe449e00640 1 -- 172.21.15.133:0/3361468268 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe44c156380 msgr2=0x7fe44c0acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:20.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.347+0000 7fe449e00640 1 --2- 172.21.15.133:0/3361468268 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe44c156380 0x7fe44c0acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.347+0000 7fe449e00640 1 -- 172.21.15.133:0/3361468268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe44c09d4c0 msgr2=0x7fe44c154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:20.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.347+0000 7fe449e00640 1 --2- 172.21.15.133:0/3361468268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe44c09d4c0 0x7fe44c154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.347+0000 7fe449e00640 1 -- 172.21.15.133:0/3361468268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c1f5560 con 0x7fe42c0039b0 2024-03-07T06:53:20.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.347+0000 7fe449e00640 1 --2- 172.21.15.133:0/3361468268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42c0039b0 0x7fe44c1f4e10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe43c037040 tx=0x7fe43c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:20.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.347+0000 7fe442000640 1 -- 172.21.15.133:0/3361468268 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c042020 con 0x7fe42c0039b0 2024-03-07T06:53:20.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.348+0000 7fe442000640 1 -- 172.21.15.133:0/3361468268 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43c03a040 con 0x7fe42c0039b0 2024-03-07T06:53:20.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.348+0000 7fe442000640 1 -- 172.21.15.133:0/3361468268 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c039070 con 0x7fe42c0039b0 2024-03-07T06:53:20.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.348+0000 7fe451000640 1 -- 172.21.15.133:0/3361468268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe44c1f5840 con 0x7fe42c0039b0 2024-03-07T06:53:20.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.348+0000 7fe451000640 1 -- 172.21.15.133:0/3361468268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe44c09bc30 con 0x7fe42c0039b0 2024-03-07T06:53:20.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.349+0000 7fe451000640 1 -- 172.21.15.133:0/3361468268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe44c09ba20 con 0x7fe42c0039b0 2024-03-07T06:53:20.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.350+0000 7fe442000640 1 -- 172.21.15.133:0/3361468268 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe43c045020 con 0x7fe42c0039b0 2024-03-07T06:53:20.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.350+0000 7fe442000640 1 --2- 172.21.15.133:0/3361468268 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe42806b500 0x7fe42806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.351+0000 7fe442000640 1 -- 172.21.15.133:0/3361468268 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe43c0ab3a0 con 0x7fe42c0039b0 2024-03-07T06:53:20.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.351+0000 7fe449400640 1 --2- 172.21.15.133:0/3361468268 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe42806b500 0x7fe42806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:20.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.352+0000 7fe449400640 1 --2- 172.21.15.133:0/3361468268 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe42806b500 0x7fe42806d9c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe44c1f4d70 tx=0x7fe434009750 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:20.354 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.355+0000 7fe442000640 1 -- 172.21.15.133:0/3361468268 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe43c04c070 con 0x7fe42c0039b0 2024-03-07T06:53:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.609+0000 7fe451000640 1 -- 172.21.15.133:0/3361468268 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe44c062c90 con 0x7fe42c0039b0 2024-03-07T06:53:20.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.610+0000 7fe442000640 1 -- 172.21.15.133:0/3361468268 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe43c07a4d0 con 0x7fe42c0039b0 2024-03-07T06:53:20.610 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:20.610 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:53:20.610 INFO:teuthology.orchestra.run.smithi133.stdout: "auth_dump": [ 2024-03-07T06:53:20.611 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.611 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.a", 2024-03-07T06:53:20.611 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCaY+llrqSIMBAAy1d9K8CCdERld7xCmKOutQ==", 2024-03-07T06:53:20.611 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.611 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:53:20.611 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-03-07T06:53:20.611 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-03-07T06:53:20.611 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.611 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.611 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.611 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.b", 2024-03-07T06:53:20.611 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCbY+llx7hIFxAAaEpBpdI7eaGAU9GuxPPfhg==", 2024-03-07T06:53:20.612 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.612 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:53:20.612 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-03-07T06:53:20.612 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-03-07T06:53:20.612 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.612 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.612 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.612 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.c", 2024-03-07T06:53:20.612 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCbY+ll0XgoBhAAXJiu001WO6mHv/UaVuHoJA==", 2024-03-07T06:53:20.612 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.612 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:53:20.612 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mds.d", 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCbY+llrDLKJxAAASZPgaXrsJYHB3pzksPcSA==", 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.613 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.0", 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCgY+llgbaKFxAAUNMCRhWDof+qhuedrCY57Q==", 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.1", 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCjY+llQx/1IRAA5yMfKhH4CxK5751ymJYSzQ==", 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.614 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.2", 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCmY+lluw4uOhAAvy/SoCA7TWJGid53bj1fhA==", 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.3", 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCqY+llf5L0FRAA/WlR9r+5bSmu+Y4ggoobdw==", 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.615 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:53:20.616 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:53:20.616 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.616 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.616 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.616 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.616 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.4", 2024-03-07T06:53:20.616 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCxY+llYmATHxAA+eGqKm4rlXZtcYgV3bZDDg==", 2024-03-07T06:53:20.616 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.616 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:53:20.616 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:53:20.616 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.616 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.616 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.616 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.5", 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQC0Y+llQJ+EOhAAs7gC9YWEap3xhX/Qiyt0lg==", 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.6", 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQC4Y+llpADECBAA42RfVF05S45UiaHJyPCRzA==", 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:53:20.617 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "osd.7", 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQC7Y+llC4N/IRAAcnaB9/yZj2fz9nL6hS5mSA==", 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow profile osd", 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile osd", 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.0", 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCbY+ll6xWENRAArKbw8ID7PLMFVCO3UlKDOg==", 2024-03-07T06:53:20.618 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow r", 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.1", 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCcY+llCvgxEBAA+PC1o+JD9S0G7IUJVLVh+g==", 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.619 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.620 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.2", 2024-03-07T06:53:20.620 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCcY+llvYWxBBAA32ACXxhIhSDI+wI6zZiafA==", 2024-03-07T06:53:20.620 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.620 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:53:20.620 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-03-07T06:53:20.620 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-03-07T06:53:20.620 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.3", 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCcY+llwgZYGxAAtbCo73WlwL2ftJ4yyFEzXA==", 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow", 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow r", 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow rw", 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow rwx" 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "client.admin", 2024-03-07T06:53:20.621 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCZY+lluFVeJBAA30ba1sDHilXgfvKFTiofwg==", 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr": "allow *", 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow *", 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.x", 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCaY+llqK68JRAAuPudks7z1/uL593N8/VyGA==", 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.622 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.623 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2024-03-07T06:53:20.623 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:20.623 INFO:teuthology.orchestra.run.smithi133.stdout: "entity": "mgr.y", 2024-03-07T06:53:20.623 INFO:teuthology.orchestra.run.smithi133.stdout: "key": "AQCaY+llosjXGBAAjyuWL3GnYWP6nHe9jnyUNg==", 2024-03-07T06:53:20.623 INFO:teuthology.orchestra.run.smithi133.stdout: "caps": { 2024-03-07T06:53:20.623 INFO:teuthology.orchestra.run.smithi133.stdout: "mds": "allow *", 2024-03-07T06:53:20.623 INFO:teuthology.orchestra.run.smithi133.stdout: "mon": "allow profile mgr", 2024-03-07T06:53:20.623 INFO:teuthology.orchestra.run.smithi133.stdout: "osd": "allow *" 2024-03-07T06:53:20.623 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.623 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:20.623 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2024-03-07T06:53:20.623 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:53:20.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.613+0000 7fe451000640 1 -- 172.21.15.133:0/3361468268 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe42806b500 msgr2=0x7fe42806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:20.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.613+0000 7fe451000640 1 --2- 172.21.15.133:0/3361468268 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe42806b500 0x7fe42806d9c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe44c1f4d70 tx=0x7fe434009750 comp rx=0 tx=0).stop 2024-03-07T06:53:20.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.613+0000 7fe451000640 1 -- 172.21.15.133:0/3361468268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42c0039b0 msgr2=0x7fe44c1f4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:20.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.613+0000 7fe451000640 1 --2- 172.21.15.133:0/3361468268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42c0039b0 0x7fe44c1f4e10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe43c037040 tx=0x7fe43c004730 comp rx=0 tx=0).stop 2024-03-07T06:53:20.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.613+0000 7fe451000640 1 -- 172.21.15.133:0/3361468268 shutdown_connections 2024-03-07T06:53:20.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.613+0000 7fe451000640 1 --2- 172.21.15.133:0/3361468268 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe44c156380 0x7fe44c0acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.613+0000 7fe451000640 1 --2- 172.21.15.133:0/3361468268 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe42806b500 0x7fe42806d9c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.613+0000 7fe451000640 1 --2- 172.21.15.133:0/3361468268 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe44c09d4c0 0x7fe44c154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.613+0000 7fe451000640 1 --2- 172.21.15.133:0/3361468268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe42c0039b0 0x7fe44c1f4e10 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.613+0000 7fe451000640 1 -- 172.21.15.133:0/3361468268 >> 172.21.15.133:0/3361468268 conn(0x7fe44c0b3120 msgr2=0x7fe44c0b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:20.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.613+0000 7fe451000640 1 -- 172.21.15.133:0/3361468268 shutdown_connections 2024-03-07T06:53:20.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.613+0000 7fe451000640 1 -- 172.21.15.133:0/3361468268 wait complete. 2024-03-07T06:53:20.630 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:53:20.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.833+0000 7fc821400640 1 Processor -- start 2024-03-07T06:53:20.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.834+0000 7fc821400640 1 -- start start 2024-03-07T06:53:20.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.835+0000 7fc821400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc81c15dd60 0x7fc81c0ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.835+0000 7fc821400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc81c162900 0x7fc81c070640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.835+0000 7fc821400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81c0af130 con 0x7fc81c0aebf0 2024-03-07T06:53:20.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.835+0000 7fc821400640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81c161bd0 con 0x7fc81c15dd60 2024-03-07T06:53:20.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.835+0000 7fc821400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c161e20 con 0x7fc81c162900 2024-03-07T06:53:20.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.836+0000 7fc81be00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc81c15dd60 0x7fc81c0ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:20.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.836+0000 7fc81be00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc81c15dd60 0x7fc81c0ae6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54448/0 (socket says 172.21.15.133:54448) 2024-03-07T06:53:20.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.836+0000 7fc81be00640 1 -- 172.21.15.133:0/416838750 learned_addr learned my addr 172.21.15.133:0/416838750 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:20.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.836+0000 7fc81b400640 1 --2- 172.21.15.133:0/416838750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc81c162900 0x7fc81c070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:20.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.836+0000 7fc81be00640 1 -- 172.21.15.133:0/416838750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc81c162900 msgr2=0x7fc81c070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:20.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.836+0000 7fc81be00640 1 --2- 172.21.15.133:0/416838750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc81c162900 0x7fc81c070640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.836+0000 7fc81be00640 1 -- 172.21.15.133:0/416838750 >> v1:172.21.15.161:6789/0 conn(0x7fc81c0aebf0 legacy=0x7fc81c06eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:20.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.836+0000 7fc81be00640 1 -- 172.21.15.133:0/416838750 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c162100 con 0x7fc81c15dd60 2024-03-07T06:53:20.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.837+0000 7fc81be00640 1 --2- 172.21.15.133:0/416838750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c15dd60 0x7fc81c0ae6b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc804009870 tx=0x7fc80402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868f6390c5680ed6 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:20.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.838+0000 7fc81aa00640 1 -- 172.21.15.133:0/416838750 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8040317c0 con 0x7fc81c15dd60 2024-03-07T06:53:20.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.838+0000 7fc81aa00640 1 -- 172.21.15.133:0/416838750 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc804037070 con 0x7fc81c15dd60 2024-03-07T06:53:20.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.838+0000 7fc81aa00640 1 -- 172.21.15.133:0/416838750 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804031b80 con 0x7fc81c15dd60 2024-03-07T06:53:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.838+0000 7fc821400640 1 -- 172.21.15.133:0/416838750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c15dd60 msgr2=0x7fc81c0ae6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:20.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.838+0000 7fc821400640 1 --2- 172.21.15.133:0/416838750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c15dd60 0x7fc81c0ae6b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc804009870 tx=0x7fc80402b860 comp rx=0 tx=0).stop 2024-03-07T06:53:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.838+0000 7fc821400640 1 -- 172.21.15.133:0/416838750 shutdown_connections 2024-03-07T06:53:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.838+0000 7fc821400640 1 --2- 172.21.15.133:0/416838750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc81c162900 0x7fc81c070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.838+0000 7fc821400640 1 --2- 172.21.15.133:0/416838750 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c15dd60 0x7fc81c0ae6b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.838+0000 7fc821400640 1 -- 172.21.15.133:0/416838750 >> 172.21.15.133:0/416838750 conn(0x7fc81c0b2500 msgr2=0x7fc81c0b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.838+0000 7fc821400640 1 -- 172.21.15.133:0/416838750 shutdown_connections 2024-03-07T06:53:20.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.839+0000 7fc821400640 1 -- 172.21.15.133:0/416838750 wait complete. 2024-03-07T06:53:20.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.840+0000 7fc821400640 1 Processor -- start 2024-03-07T06:53:20.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.840+0000 7fc821400640 1 -- start start 2024-03-07T06:53:20.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.841+0000 7fc821400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c0aebf0 0x7fc81c1e8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.841+0000 7fc821400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc81c15dd60 0x7fc81c1e8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.841+0000 7fc821400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc81c162900 0x7fc81c149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.841+0000 7fc821400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c15e140 con 0x7fc81c0aebf0 2024-03-07T06:53:20.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.841+0000 7fc821400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c0d00b0 con 0x7fc81c162900 2024-03-07T06:53:20.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.841+0000 7fc821400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c15e2c0 con 0x7fc81c15dd60 2024-03-07T06:53:20.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.841+0000 7fc81be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c0aebf0 0x7fc81c1e8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:20.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.841+0000 7fc81be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c0aebf0 0x7fc81c1e8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54456/0 (socket says 172.21.15.133:54456) 2024-03-07T06:53:20.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.841+0000 7fc81be00640 1 -- 172.21.15.133:0/3142827239 learned_addr learned my addr 172.21.15.133:0/3142827239 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:20.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.842+0000 7fc81be00640 1 -- 172.21.15.133:0/3142827239 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc81c15dd60 msgr2=0x7fc81c1e8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:20.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.842+0000 7fc81be00640 1 --2- 172.21.15.133:0/3142827239 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc81c15dd60 0x7fc81c1e8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.842+0000 7fc81be00640 1 -- 172.21.15.133:0/3142827239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc81c162900 msgr2=0x7fc81c149cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:20.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.842+0000 7fc81be00640 1 --2- 172.21.15.133:0/3142827239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc81c162900 0x7fc81c149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:20.842 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.842+0000 7fc81be00640 1 -- 172.21.15.133:0/3142827239 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c14a210 con 0x7fc81c0aebf0 2024-03-07T06:53:20.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.842+0000 7fc81be00640 1 --2- 172.21.15.133:0/3142827239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c0aebf0 0x7fc81c1e8e10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc804009870 tx=0x7fc8040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:20.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.842+0000 7fc813e00640 1 -- 172.21.15.133:0/3142827239 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804032860 con 0x7fc81c0aebf0 2024-03-07T06:53:20.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.842+0000 7fc813e00640 1 -- 172.21.15.133:0/3142827239 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc804032a00 con 0x7fc81c0aebf0 2024-03-07T06:53:20.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.842+0000 7fc813e00640 1 -- 172.21.15.133:0/3142827239 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804032df0 con 0x7fc81c0aebf0 2024-03-07T06:53:20.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.843+0000 7fc821400640 1 -- 172.21.15.133:0/3142827239 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81c1e94d0 con 0x7fc81c0aebf0 2024-03-07T06:53:20.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.843+0000 7fc821400640 1 -- 172.21.15.133:0/3142827239 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c1e9a90 con 0x7fc81c0aebf0 2024-03-07T06:53:20.843 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.844+0000 7fc821400640 1 -- 172.21.15.133:0/3142827239 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81c062c90 con 0x7fc81c0aebf0 2024-03-07T06:53:20.844 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.845+0000 7fc813e00640 1 -- 172.21.15.133:0/3142827239 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc80403b3f0 con 0x7fc81c0aebf0 2024-03-07T06:53:20.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.845+0000 7fc813e00640 1 --2- 172.21.15.133:0/3142827239 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc7f806b500 0x7fc7f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:20.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.846+0000 7fc813e00640 1 -- 172.21.15.133:0/3142827239 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc8040abff0 con 0x7fc81c0aebf0 2024-03-07T06:53:20.845 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.846+0000 7fc81b400640 1 --2- 172.21.15.133:0/3142827239 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc7f806b500 0x7fc7f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:20.846 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.847+0000 7fc81b400640 1 --2- 172.21.15.133:0/3142827239 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc7f806b500 0x7fc7f806d9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc81c161030 tx=0x7fc80c007390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:20.849 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:20.849+0000 7fc813e00640 1 -- 172.21.15.133:0/3142827239 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc804040070 con 0x7fc81c0aebf0 2024-03-07T06:53:21.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.032+0000 7fc821400640 1 -- 172.21.15.133:0/3142827239 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc81c161640 con 0x7fc81c0aebf0 2024-03-07T06:53:21.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.034+0000 7fc813e00640 1 -- 172.21.15.133:0/3142827239 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+12939 (secure 0 0 0) 0x7fc80407b160 con 0x7fc81c0aebf0 2024-03-07T06:53:21.034 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:21.034 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":42,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:53:20.128398+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:53:21.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.037+0000 7fc821400640 1 -- 172.21.15.133:0/3142827239 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc7f806b500 msgr2=0x7fc7f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:21.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.037+0000 7fc821400640 1 --2- 172.21.15.133:0/3142827239 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc7f806b500 0x7fc7f806d9c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc81c161030 tx=0x7fc80c007390 comp rx=0 tx=0).stop 2024-03-07T06:53:21.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.037+0000 7fc821400640 1 -- 172.21.15.133:0/3142827239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c0aebf0 msgr2=0x7fc81c1e8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:21.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.037+0000 7fc821400640 1 --2- 172.21.15.133:0/3142827239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c0aebf0 0x7fc81c1e8e10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc804009870 tx=0x7fc8040046c0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.037+0000 7fc821400640 1 -- 172.21.15.133:0/3142827239 shutdown_connections 2024-03-07T06:53:21.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.037+0000 7fc821400640 1 --2- 172.21.15.133:0/3142827239 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc7f806b500 0x7fc7f806d9c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.037+0000 7fc821400640 1 --2- 172.21.15.133:0/3142827239 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc81c162900 0x7fc81c149cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.037+0000 7fc821400640 1 --2- 172.21.15.133:0/3142827239 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc81c15dd60 0x7fc81c1e8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.037+0000 7fc821400640 1 --2- 172.21.15.133:0/3142827239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc81c0aebf0 0x7fc81c1e8e10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.037+0000 7fc821400640 1 -- 172.21.15.133:0/3142827239 >> 172.21.15.133:0/3142827239 conn(0x7fc81c0b2500 msgr2=0x7fc81c0b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:21.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.037+0000 7fc821400640 1 -- 172.21.15.133:0/3142827239 shutdown_connections 2024-03-07T06:53:21.038 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.037+0000 7fc821400640 1 -- 172.21.15.133:0/3142827239 wait complete. 2024-03-07T06:53:21.054 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:53:21.054 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:53:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.264+0000 7ffa7f200640 1 Processor -- start 2024-03-07T06:53:21.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.264+0000 7ffa7f200640 1 -- start start 2024-03-07T06:53:21.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.265+0000 7ffa7f200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffa780b3ed0 0x7ffa7815b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.265+0000 7ffa7f200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffa7806d180 0x7ffa7815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.265+0000 7ffa7f200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7806d740 con 0x7ffa7815b370 2024-03-07T06:53:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.265+0000 7ffa7f200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7815c1d0 con 0x7ffa780b3ed0 2024-03-07T06:53:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.265+0000 7ffa7f200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7806da20 con 0x7ffa7806d180 2024-03-07T06:53:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.265+0000 7ffa6f400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffa780b3ed0 0x7ffa7815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.265+0000 7ffa6f400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffa780b3ed0 0x7ffa7815b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54462/0 (socket says 172.21.15.133:54462) 2024-03-07T06:53:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.265+0000 7ffa6f400640 1 -- 172.21.15.133:0/1125920036 learned_addr learned my addr 172.21.15.133:0/1125920036 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.266+0000 7ffa7d200640 1 --2- 172.21.15.133:0/1125920036 >> v2:172.21.15.161:3300/0 conn(0x7ffa7806d180 0x7ffa7815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.266+0000 7ffa6f400640 1 -- 172.21.15.133:0/1125920036 >> v2:172.21.15.161:3300/0 conn(0x7ffa7806d180 msgr2=0x7ffa7815bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.266+0000 7ffa6f400640 1 --2- 172.21.15.133:0/1125920036 >> v2:172.21.15.161:3300/0 conn(0x7ffa7806d180 0x7ffa7815bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.266+0000 7ffa6f400640 1 -- 172.21.15.133:0/1125920036 >> v1:172.21.15.161:6789/0 conn(0x7ffa7815b370 legacy=0x7ffa780cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.266+0000 7ffa6f400640 1 -- 172.21.15.133:0/1125920036 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7806dd00 con 0x7ffa780b3ed0 2024-03-07T06:53:21.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.267+0000 7ffa6f400640 1 --2- 172.21.15.133:0/1125920036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa780b3ed0 0x7ffa7815b750 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffa780cd170 tx=0x7ffa6002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6917a7e57bf3e97 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:21.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.267+0000 7ffa77400640 1 -- 172.21.15.133:0/1125920036 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60035020 con 0x7ffa780b3ed0 2024-03-07T06:53:21.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.267+0000 7ffa77400640 1 -- 172.21.15.133:0/1125920036 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa60031870 con 0x7ffa780b3ed0 2024-03-07T06:53:21.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.267+0000 7ffa77400640 1 -- 172.21.15.133:0/1125920036 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60031c40 con 0x7ffa780b3ed0 2024-03-07T06:53:21.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.268+0000 7ffa7f200640 1 -- 172.21.15.133:0/1125920036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa780b3ed0 msgr2=0x7ffa7815b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:21.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.268+0000 7ffa7f200640 1 --2- 172.21.15.133:0/1125920036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa780b3ed0 0x7ffa7815b750 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffa780cd170 tx=0x7ffa6002b910 comp rx=0 tx=0).stop 2024-03-07T06:53:21.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.268+0000 7ffa7f200640 1 -- 172.21.15.133:0/1125920036 shutdown_connections 2024-03-07T06:53:21.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.268+0000 7ffa7f200640 1 --2- 172.21.15.133:0/1125920036 >> v2:172.21.15.161:3300/0 conn(0x7ffa7806d180 0x7ffa7815bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.268+0000 7ffa7f200640 1 --2- 172.21.15.133:0/1125920036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa780b3ed0 0x7ffa7815b750 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.268+0000 7ffa7f200640 1 -- 172.21.15.133:0/1125920036 >> 172.21.15.133:0/1125920036 conn(0x7ffa780b1d30 msgr2=0x7ffa780b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:21.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.268+0000 7ffa7f200640 1 -- 172.21.15.133:0/1125920036 shutdown_connections 2024-03-07T06:53:21.270 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.268+0000 7ffa7f200640 1 -- 172.21.15.133:0/1125920036 wait complete. 2024-03-07T06:53:21.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.270+0000 7ffa7f200640 1 Processor -- start 2024-03-07T06:53:21.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.270+0000 7ffa7f200640 1 -- start start 2024-03-07T06:53:21.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.271+0000 7ffa7f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa7806d180 0x7ffa781435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:21.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.271+0000 7ffa7f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa780b3ed0 0x7ffa7814bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:21.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.271+0000 7ffa7f200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffa7815b370 0x7ffa78142010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:21.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.271+0000 7ffa7f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780cf660 con 0x7ffa780b3ed0 2024-03-07T06:53:21.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.271+0000 7ffa7f200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7806e2c0 con 0x7ffa7806d180 2024-03-07T06:53:21.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.271+0000 7ffa7f200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa780cf7e0 con 0x7ffa7815b370 2024-03-07T06:53:21.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.271+0000 7ffa6f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa780b3ed0 0x7ffa7814bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:21.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.271+0000 7ffa6f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa780b3ed0 0x7ffa7814bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54478/0 (socket says 172.21.15.133:54478) 2024-03-07T06:53:21.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.271+0000 7ffa6f400640 1 -- 172.21.15.133:0/2745313365 learned_addr learned my addr 172.21.15.133:0/2745313365 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:21.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.272+0000 7ffa77e00640 1 --2- 172.21.15.133:0/2745313365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa7806d180 0x7ffa781435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:21.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.272+0000 7ffa6f400640 1 -- 172.21.15.133:0/2745313365 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffa7815b370 msgr2=0x7ffa78142010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:21.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.272+0000 7ffa6f400640 1 --2- 172.21.15.133:0/2745313365 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffa7815b370 0x7ffa78142010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.272+0000 7ffa6f400640 1 -- 172.21.15.133:0/2745313365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa7806d180 msgr2=0x7ffa781435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:21.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.272+0000 7ffa6f400640 1 --2- 172.21.15.133:0/2745313365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa7806d180 0x7ffa781435b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.272+0000 7ffa6f400640 1 -- 172.21.15.133:0/2745313365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa78143cb0 con 0x7ffa780b3ed0 2024-03-07T06:53:21.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.272+0000 7ffa6f400640 1 --2- 172.21.15.133:0/2745313365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa780b3ed0 0x7ffa7814bda0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ffa60000c00 tx=0x7ffa600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:21.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.272+0000 7ffa74c00640 1 -- 172.21.15.133:0/2745313365 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60040070 con 0x7ffa780b3ed0 2024-03-07T06:53:21.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.272+0000 7ffa74c00640 1 -- 172.21.15.133:0/2745313365 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa60030d00 con 0x7ffa780b3ed0 2024-03-07T06:53:21.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.272+0000 7ffa74c00640 1 -- 172.21.15.133:0/2745313365 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60035040 con 0x7ffa780b3ed0 2024-03-07T06:53:21.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.272+0000 7ffa7f200640 1 -- 172.21.15.133:0/2745313365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa78143f90 con 0x7ffa780b3ed0 2024-03-07T06:53:21.275 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.273+0000 7ffa7f200640 1 -- 172.21.15.133:0/2745313365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa78144550 con 0x7ffa780b3ed0 2024-03-07T06:53:21.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.274+0000 7ffa7f200640 1 -- 172.21.15.133:0/2745313365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa78062c90 con 0x7ffa780b3ed0 2024-03-07T06:53:21.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.275+0000 7ffa74c00640 1 -- 172.21.15.133:0/2745313365 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ffa60038020 con 0x7ffa780b3ed0 2024-03-07T06:53:21.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.275+0000 7ffa74c00640 1 --2- 172.21.15.133:0/2745313365 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ffa5406b4d0 0x7ffa5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:21.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.275+0000 7ffa74c00640 1 -- 172.21.15.133:0/2745313365 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ffa600abda0 con 0x7ffa780b3ed0 2024-03-07T06:53:21.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.276+0000 7ffa77e00640 1 --2- 172.21.15.133:0/2745313365 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ffa5406b4d0 0x7ffa5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:21.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.277+0000 7ffa77e00640 1 --2- 172.21.15.133:0/2745313365 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ffa5406b4d0 0x7ffa5406d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800a730 tx=0x7ffa68008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:21.279 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.280+0000 7ffa74c00640 1 -- 172.21.15.133:0/2745313365 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffa6007aed0 con 0x7ffa780b3ed0 2024-03-07T06:53:21.475 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.476+0000 7ffa7f200640 1 -- 172.21.15.133:0/2745313365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa7814c810 con 0x7ffa780b3ed0 2024-03-07T06:53:21.476 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.476+0000 7ffa74c00640 1 -- 172.21.15.133:0/2745313365 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa6007a7d0 con 0x7ffa780b3ed0 2024-03-07T06:53:21.476 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:53:21.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.479+0000 7ffa7f200640 1 -- 172.21.15.133:0/2745313365 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ffa5406b4d0 msgr2=0x7ffa5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:21.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.479+0000 7ffa7f200640 1 --2- 172.21.15.133:0/2745313365 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ffa5406b4d0 0x7ffa5406d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800a730 tx=0x7ffa68008040 comp rx=0 tx=0).stop 2024-03-07T06:53:21.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.479+0000 7ffa7f200640 1 -- 172.21.15.133:0/2745313365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa780b3ed0 msgr2=0x7ffa7814bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:21.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.479+0000 7ffa7f200640 1 --2- 172.21.15.133:0/2745313365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa780b3ed0 0x7ffa7814bda0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ffa60000c00 tx=0x7ffa600046c0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.479+0000 7ffa7f200640 1 -- 172.21.15.133:0/2745313365 shutdown_connections 2024-03-07T06:53:21.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.479+0000 7ffa7f200640 1 --2- 172.21.15.133:0/2745313365 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffa7815b370 0x7ffa78142010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.479+0000 7ffa7f200640 1 --2- 172.21.15.133:0/2745313365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa780b3ed0 0x7ffa7814bda0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.479+0000 7ffa7f200640 1 --2- 172.21.15.133:0/2745313365 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ffa5406b4d0 0x7ffa5406d990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.479+0000 7ffa7f200640 1 --2- 172.21.15.133:0/2745313365 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffa7806d180 0x7ffa781435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.479+0000 7ffa7f200640 1 -- 172.21.15.133:0/2745313365 >> 172.21.15.133:0/2745313365 conn(0x7ffa780b1d30 msgr2=0x7ffa7814cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:21.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.479+0000 7ffa7f200640 1 -- 172.21.15.133:0/2745313365 shutdown_connections 2024-03-07T06:53:21.480 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.479+0000 7ffa7f200640 1 -- 172.21.15.133:0/2745313365 wait complete. 2024-03-07T06:53:21.499 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-07T06:53:21.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.692+0000 7fc9b4000640 1 Processor -- start 2024-03-07T06:53:21.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.692+0000 7fc9b4000640 1 -- start start 2024-03-07T06:53:21.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.694+0000 7fc9b4000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc9ac0ce5f0 0x7fc9ac0b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:21.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.694+0000 7fc9b4000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc9ac0af950 0x7fc9ac071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:21.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.694+0000 7fc9b4000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac06f140 con 0x7fc9ac15b1b0 2024-03-07T06:53:21.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.694+0000 7fc9b4000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ac071610 con 0x7fc9ac0ce5f0 2024-03-07T06:53:21.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.694+0000 7fc9b4000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ac06f420 con 0x7fc9ac0af950 2024-03-07T06:53:21.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.694+0000 7fc9b0c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc9ac0ce5f0 0x7fc9ac0b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:21.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.694+0000 7fc9b0c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fc9ac0ce5f0 0x7fc9ac0b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54480/0 (socket says 172.21.15.133:54480) 2024-03-07T06:53:21.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.694+0000 7fc9b0c00640 1 -- 172.21.15.133:0/1206224053 learned_addr learned my addr 172.21.15.133:0/1206224053 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:21.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.694+0000 7fc9b2000640 1 --2- 172.21.15.133:0/1206224053 >> v2:172.21.15.161:3300/0 conn(0x7fc9ac0af950 0x7fc9ac071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:21.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.694+0000 7fc9b0c00640 1 -- 172.21.15.133:0/1206224053 >> v2:172.21.15.161:3300/0 conn(0x7fc9ac0af950 msgr2=0x7fc9ac071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:21.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.694+0000 7fc9b0c00640 1 --2- 172.21.15.133:0/1206224053 >> v2:172.21.15.161:3300/0 conn(0x7fc9ac0af950 0x7fc9ac071d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.694+0000 7fc9b0c00640 1 -- 172.21.15.133:0/1206224053 >> v1:172.21.15.161:6789/0 conn(0x7fc9ac15b1b0 legacy=0x7fc9ac0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:21.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.694+0000 7fc9b0c00640 1 -- 172.21.15.133:0/1206224053 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac071860 con 0x7fc9ac0ce5f0 2024-03-07T06:53:21.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.694+0000 7fc9b2000640 1 --2- 172.21.15.133:0/1206224053 >> v2:172.21.15.161:3300/0 conn(0x7fc9ac0af950 0x7fc9ac071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:21.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.696+0000 7fc9b0c00640 1 --2- 172.21.15.133:0/1206224053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0ce5f0 0x7fc9ac0b3c10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac09e020 tx=0x7fc99802e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbddff9fa7fe3645 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:21.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.696+0000 7fc9abe00640 1 -- 172.21.15.133:0/1206224053 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99803b070 con 0x7fc9ac0ce5f0 2024-03-07T06:53:21.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.696+0000 7fc9abe00640 1 -- 172.21.15.133:0/1206224053 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99802ee00 con 0x7fc9ac0ce5f0 2024-03-07T06:53:21.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.696+0000 7fc9abe00640 1 -- 172.21.15.133:0/1206224053 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc998036660 con 0x7fc9ac0ce5f0 2024-03-07T06:53:21.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.697+0000 7fc9b4000640 1 -- 172.21.15.133:0/1206224053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0ce5f0 msgr2=0x7fc9ac0b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:21.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.697+0000 7fc9b4000640 1 --2- 172.21.15.133:0/1206224053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0ce5f0 0x7fc9ac0b3c10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac09e020 tx=0x7fc99802e050 comp rx=0 tx=0).stop 2024-03-07T06:53:21.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.697+0000 7fc9b4000640 1 -- 172.21.15.133:0/1206224053 shutdown_connections 2024-03-07T06:53:21.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.697+0000 7fc9b4000640 1 --2- 172.21.15.133:0/1206224053 >> v2:172.21.15.161:3300/0 conn(0x7fc9ac0af950 0x7fc9ac071d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.697+0000 7fc9b4000640 1 --2- 172.21.15.133:0/1206224053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0ce5f0 0x7fc9ac0b3c10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.697+0000 7fc9b4000640 1 -- 172.21.15.133:0/1206224053 >> 172.21.15.133:0/1206224053 conn(0x7fc9ac0b3120 msgr2=0x7fc9ac0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:21.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.697+0000 7fc9b4000640 1 -- 172.21.15.133:0/1206224053 shutdown_connections 2024-03-07T06:53:21.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.697+0000 7fc9b4000640 1 -- 172.21.15.133:0/1206224053 wait complete. 2024-03-07T06:53:21.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.698+0000 7fc9b4000640 1 Processor -- start 2024-03-07T06:53:21.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.699+0000 7fc9b4000640 1 -- start start 2024-03-07T06:53:21.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.699+0000 7fc9b4000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0af950 0x7fc9ac150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:21.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b4000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9ac0ce5f0 0x7fc9ac09c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:21.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b4000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9ac15b1b0 0x7fc9ac1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:21.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b4000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac15b590 con 0x7fc9ac0af950 2024-03-07T06:53:21.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b4000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac06ee30 con 0x7fc9ac15b1b0 2024-03-07T06:53:21.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b4000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac15b710 con 0x7fc9ac0ce5f0 2024-03-07T06:53:21.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b1600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0af950 0x7fc9ac150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:21.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b1600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0af950 0x7fc9ac150ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54494/0 (socket says 172.21.15.133:54494) 2024-03-07T06:53:21.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b1600640 1 -- 172.21.15.133:0/191657808 learned_addr learned my addr 172.21.15.133:0/191657808 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:21.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b2000640 1 --2- 172.21.15.133:0/191657808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9ac15b1b0 0x7fc9ac1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:21.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b0c00640 1 --2- 172.21.15.133:0/191657808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9ac0ce5f0 0x7fc9ac09c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:21.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b1600640 1 -- 172.21.15.133:0/191657808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9ac0ce5f0 msgr2=0x7fc9ac09c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:21.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b1600640 1 --2- 172.21.15.133:0/191657808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9ac0ce5f0 0x7fc9ac09c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b1600640 1 -- 172.21.15.133:0/191657808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9ac15b1b0 msgr2=0x7fc9ac1579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:21.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b1600640 1 --2- 172.21.15.133:0/191657808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9ac15b1b0 0x7fc9ac1579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.700+0000 7fc9b1600640 1 -- 172.21.15.133:0/191657808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac158460 con 0x7fc9ac0af950 2024-03-07T06:53:21.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.701+0000 7fc9b1600640 1 --2- 172.21.15.133:0/191657808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0af950 0x7fc9ac150ce0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00ba80 tx=0x7fc99c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:21.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.701+0000 7fc9a9600640 1 -- 172.21.15.133:0/191657808 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c019070 con 0x7fc9ac0af950 2024-03-07T06:53:21.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.701+0000 7fc9a9600640 1 -- 172.21.15.133:0/191657808 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c00cc90 con 0x7fc9ac0af950 2024-03-07T06:53:21.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.701+0000 7fc9b4000640 1 -- 172.21.15.133:0/191657808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ac09b7f0 con 0x7fc9ac0af950 2024-03-07T06:53:21.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.701+0000 7fc9a9600640 1 -- 172.21.15.133:0/191657808 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c014680 con 0x7fc9ac0af950 2024-03-07T06:53:21.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.701+0000 7fc9b4000640 1 -- 172.21.15.133:0/191657808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac09bdb0 con 0x7fc9ac0af950 2024-03-07T06:53:21.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.702+0000 7fc9b4000640 1 -- 172.21.15.133:0/191657808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ac062c90 con 0x7fc9ac0af950 2024-03-07T06:53:21.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.703+0000 7fc9a9600640 1 -- 172.21.15.133:0/191657808 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc99c01c070 con 0x7fc9ac0af950 2024-03-07T06:53:21.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.704+0000 7fc9a9600640 1 --2- 172.21.15.133:0/191657808 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc98c06b4d0 0x7fc98c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:21.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.704+0000 7fc9a9600640 1 -- 172.21.15.133:0/191657808 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc99c089d40 con 0x7fc9ac0af950 2024-03-07T06:53:21.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.705+0000 7fc9b0c00640 1 --2- 172.21.15.133:0/191657808 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc98c06b4d0 0x7fc98c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:21.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.706+0000 7fc9b0c00640 1 --2- 172.21.15.133:0/191657808 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc98c06b4d0 0x7fc98c06d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc998038000 tx=0x7fc998008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:21.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.708+0000 7fc9a9600640 1 -- 172.21.15.133:0/191657808 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc99c058e70 con 0x7fc9ac0af950 2024-03-07T06:53:21.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.908+0000 7fc9b4000640 1 -- 172.21.15.133:0/191657808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc9ac0afef0 con 0x7fc9ac0af950 2024-03-07T06:53:21.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.908+0000 7fc9a9600640 1 -- 172.21.15.133:0/191657808 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc99c058770 con 0x7fc9ac0af950 2024-03-07T06:53:21.908 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:21.909 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2024-03-07T06:53:21.909 INFO:teuthology.orchestra.run.smithi133.stdout: { 2024-03-07T06:53:21.909 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2024-03-07T06:53:21.909 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2024-03-07T06:53:21.909 INFO:teuthology.orchestra.run.smithi133.stdout: } 2024-03-07T06:53:21.909 INFO:teuthology.orchestra.run.smithi133.stdout:] 2024-03-07T06:53:21.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.911+0000 7fc9b4000640 1 -- 172.21.15.133:0/191657808 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc98c06b4d0 msgr2=0x7fc98c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:21.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.911+0000 7fc9b4000640 1 --2- 172.21.15.133:0/191657808 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc98c06b4d0 0x7fc98c06d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc998038000 tx=0x7fc998008040 comp rx=0 tx=0).stop 2024-03-07T06:53:21.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.911+0000 7fc9b4000640 1 -- 172.21.15.133:0/191657808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0af950 msgr2=0x7fc9ac150ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:21.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.911+0000 7fc9b4000640 1 --2- 172.21.15.133:0/191657808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0af950 0x7fc9ac150ce0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00ba80 tx=0x7fc99c00bf40 comp rx=0 tx=0).stop 2024-03-07T06:53:21.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.911+0000 7fc9b4000640 1 -- 172.21.15.133:0/191657808 shutdown_connections 2024-03-07T06:53:21.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.911+0000 7fc9b4000640 1 --2- 172.21.15.133:0/191657808 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc98c06b4d0 0x7fc98c06d990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.911+0000 7fc9b4000640 1 --2- 172.21.15.133:0/191657808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc9ac15b1b0 0x7fc9ac1579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.911+0000 7fc9b4000640 1 --2- 172.21.15.133:0/191657808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc9ac0ce5f0 0x7fc9ac09c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.911+0000 7fc9b4000640 1 --2- 172.21.15.133:0/191657808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0af950 0x7fc9ac150ce0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:21.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.911+0000 7fc9b4000640 1 -- 172.21.15.133:0/191657808 >> 172.21.15.133:0/191657808 conn(0x7fc9ac0b3120 msgr2=0x7fc9ac0cd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:21.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.911+0000 7fc9b4000640 1 -- 172.21.15.133:0/191657808 shutdown_connections 2024-03-07T06:53:21.913 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:21.911+0000 7fc9b4000640 1 -- 172.21.15.133:0/191657808 wait complete. 2024-03-07T06:53:21.930 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-07T06:53:21.930 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-07T06:53:22.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.132+0000 7f5811e00640 1 Processor -- start 2024-03-07T06:53:22.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.132+0000 7f5811e00640 1 -- start start 2024-03-07T06:53:22.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.134+0000 7f5811e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f580c070cb0 0x7f580c0b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:22.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.134+0000 7f5811e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580c0b3f20 con 0x7f580c147800 2024-03-07T06:53:22.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.134+0000 7f5811e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580c06d2f0 con 0x7f580c070820 2024-03-07T06:53:22.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.134+0000 7f5811e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580c162bb0 con 0x7f580c070cb0 2024-03-07T06:53:22.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.134+0000 7f580aa00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f580c147800 0x7f580c15f070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55172/0 (socket says 172.21.15.133:55172) 2024-03-07T06:53:22.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.134+0000 7f580aa00640 1 -- 172.21.15.133:0/4287051890 learned_addr learned my addr 172.21.15.133:0/4287051890 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:22.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.134+0000 7f580b400640 1 --2- 172.21.15.133:0/4287051890 >> v2:172.21.15.133:3300/0 conn(0x7f580c070cb0 0x7f580c0b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:22.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.135+0000 7f5809600640 1 -- 172.21.15.133:0/4287051890 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699143492 0 0) 0x7f580c0b3f20 con 0x7f580c147800 2024-03-07T06:53:22.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.135+0000 7f5809600640 1 -- 172.21.15.133:0/4287051890 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57f0003520 con 0x7f580c147800 2024-03-07T06:53:22.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.135+0000 7f580b400640 1 -- 172.21.15.133:0/4287051890 >> v1:172.21.15.161:6789/0 conn(0x7f580c070820 legacy=0x7f580c0b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:22.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.135+0000 7f580b400640 1 -- 172.21.15.133:0/4287051890 >> v1:172.21.15.133:6789/0 conn(0x7f580c147800 legacy=0x7f580c15f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:22.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.135+0000 7f580b400640 1 -- 172.21.15.133:0/4287051890 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c0aea70 con 0x7f580c070cb0 2024-03-07T06:53:22.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.136+0000 7f580b400640 1 --2- 172.21.15.133:0/4287051890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c070cb0 0x7f580c0b4ad0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5804009d90 tx=0x7f580402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb252d15fed50db2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:22.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.136+0000 7f5809600640 1 -- 172.21.15.133:0/4287051890 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804039070 con 0x7f580c070cb0 2024-03-07T06:53:22.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.136+0000 7f5809600640 1 -- 172.21.15.133:0/4287051890 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580402cc60 con 0x7f580c070cb0 2024-03-07T06:53:22.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.136+0000 7f5809600640 1 -- 172.21.15.133:0/4287051890 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804034660 con 0x7f580c070cb0 2024-03-07T06:53:22.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.137+0000 7f5811e00640 1 -- 172.21.15.133:0/4287051890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c070cb0 msgr2=0x7f580c0b4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:22.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.137+0000 7f5811e00640 1 --2- 172.21.15.133:0/4287051890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c070cb0 0x7f580c0b4ad0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f5804009d90 tx=0x7f580402beb0 comp rx=0 tx=0).stop 2024-03-07T06:53:22.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.137+0000 7f5811e00640 1 -- 172.21.15.133:0/4287051890 shutdown_connections 2024-03-07T06:53:22.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.137+0000 7f5811e00640 1 --2- 172.21.15.133:0/4287051890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c070cb0 0x7f580c0b4ad0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:22.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.137+0000 7f5811e00640 1 -- 172.21.15.133:0/4287051890 >> 172.21.15.133:0/4287051890 conn(0x7f580c066eb0 msgr2=0x7f580c067290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:22.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.137+0000 7f5811e00640 1 -- 172.21.15.133:0/4287051890 shutdown_connections 2024-03-07T06:53:22.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.137+0000 7f5811e00640 1 -- 172.21.15.133:0/4287051890 wait complete. 2024-03-07T06:53:22.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.138+0000 7f5811e00640 1 Processor -- start 2024-03-07T06:53:22.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.139+0000 7f5811e00640 1 -- start start 2024-03-07T06:53:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.139+0000 7f5811e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f580c070820 0x7f580c13d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.139+0000 7f5811e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580c070cb0 0x7f580c1e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.139+0000 7f5811e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c147800 0x7f580c1417d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.139+0000 7f5811e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c1625f0 con 0x7f580c147800 2024-03-07T06:53:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.139+0000 7f5811e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c162310 con 0x7f580c070cb0 2024-03-07T06:53:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.139+0000 7f5811e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c1628d0 con 0x7f580c070820 2024-03-07T06:53:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.140+0000 7f580b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c147800 0x7f580c1417d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.140+0000 7f580b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c147800 0x7f580c1417d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54512/0 (socket says 172.21.15.133:54512) 2024-03-07T06:53:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.140+0000 7f580b400640 1 -- 172.21.15.133:0/689425780 learned_addr learned my addr 172.21.15.133:0/689425780 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.140+0000 7f580aa00640 1 --2- 172.21.15.133:0/689425780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f580c070820 0x7f580c13d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.140+0000 7f580b400640 1 -- 172.21.15.133:0/689425780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f580c070820 msgr2=0x7f580c13d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.140+0000 7f580b400640 1 --2- 172.21.15.133:0/689425780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f580c070820 0x7f580c13d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.140+0000 7f580b400640 1 -- 172.21.15.133:0/689425780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580c070cb0 msgr2=0x7f580c1e94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:22.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.140+0000 7f580a000640 1 --2- 172.21.15.133:0/689425780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580c070cb0 0x7f580c1e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:22.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.140+0000 7f580b400640 1 --2- 172.21.15.133:0/689425780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580c070cb0 0x7f580c1e94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:22.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.140+0000 7f580b400640 1 -- 172.21.15.133:0/689425780 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c14b190 con 0x7f580c147800 2024-03-07T06:53:22.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.141+0000 7f580b400640 1 --2- 172.21.15.133:0/689425780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c147800 0x7f580c1417d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5804037040 tx=0x7f5804004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:22.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.141+0000 7f5803400640 1 -- 172.21.15.133:0/689425780 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804039050 con 0x7f580c147800 2024-03-07T06:53:22.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.141+0000 7f5803400640 1 -- 172.21.15.133:0/689425780 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5804045070 con 0x7f580c147800 2024-03-07T06:53:22.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.141+0000 7f5803400640 1 -- 172.21.15.133:0/689425780 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804009450 con 0x7f580c147800 2024-03-07T06:53:22.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.141+0000 7f5811e00640 1 -- 172.21.15.133:0/689425780 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580c14b470 con 0x7f580c147800 2024-03-07T06:53:22.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.141+0000 7f5811e00640 1 -- 172.21.15.133:0/689425780 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580c14ba30 con 0x7f580c147800 2024-03-07T06:53:22.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.142+0000 7f5811e00640 1 -- 172.21.15.133:0/689425780 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f580c06f360 con 0x7f580c147800 2024-03-07T06:53:22.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.143+0000 7f5803400640 1 -- 172.21.15.133:0/689425780 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f580403e850 con 0x7f580c147800 2024-03-07T06:53:22.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.144+0000 7f5803400640 1 --2- 172.21.15.133:0/689425780 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57ec06b4d0 0x7f57ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:22.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.144+0000 7f5803400640 1 -- 172.21.15.133:0/689425780 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f58040acb20 con 0x7f580c147800 2024-03-07T06:53:22.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.145+0000 7f580aa00640 1 --2- 172.21.15.133:0/689425780 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57ec06b4d0 0x7f57ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:22.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.146+0000 7f580aa00640 1 --2- 172.21.15.133:0/689425780 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57ec06b4d0 0x7f57ec06d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f57fc000f60 tx=0x7f57fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:22.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.149+0000 7f5803400640 1 -- 172.21.15.133:0/689425780 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f580c06f360 con 0x7f580c147800 2024-03-07T06:53:22.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:22.345+0000 7f5811e00640 1 -- 172.21.15.133:0/689425780 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f580c062c90 con 0x7f580c147800 2024-03-07T06:53:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.133+0000 7f5803400640 1 -- 172.21.15.133:0/689425780 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5804032e20 con 0x7f580c147800 2024-03-07T06:53:23.133 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2024-03-07T06:53:23.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.137+0000 7f5811e00640 1 -- 172.21.15.133:0/689425780 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57ec06b4d0 msgr2=0x7f57ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:23.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.137+0000 7f5811e00640 1 --2- 172.21.15.133:0/689425780 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57ec06b4d0 0x7f57ec06d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f57fc000f60 tx=0x7f57fc009040 comp rx=0 tx=0).stop 2024-03-07T06:53:23.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.137+0000 7f5811e00640 1 -- 172.21.15.133:0/689425780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c147800 msgr2=0x7f580c1417d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:23.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.137+0000 7f5811e00640 1 --2- 172.21.15.133:0/689425780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c147800 0x7f580c1417d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5804037040 tx=0x7f5804004a40 comp rx=0 tx=0).stop 2024-03-07T06:53:23.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.137+0000 7f5811e00640 1 -- 172.21.15.133:0/689425780 shutdown_connections 2024-03-07T06:53:23.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.137+0000 7f5811e00640 1 --2- 172.21.15.133:0/689425780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f580c147800 0x7f580c1417d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:23.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.137+0000 7f5811e00640 1 --2- 172.21.15.133:0/689425780 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f57ec06b4d0 0x7f57ec06d990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:23.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.137+0000 7f5811e00640 1 --2- 172.21.15.133:0/689425780 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f580c070cb0 0x7f580c1e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:23.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.137+0000 7f5811e00640 1 --2- 172.21.15.133:0/689425780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f580c070820 0x7f580c13d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:23.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.137+0000 7f5811e00640 1 -- 172.21.15.133:0/689425780 >> 172.21.15.133:0/689425780 conn(0x7f580c066eb0 msgr2=0x7f580c0b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:23.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.138+0000 7f5811e00640 1 -- 172.21.15.133:0/689425780 shutdown_connections 2024-03-07T06:53:23.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.138+0000 7f5811e00640 1 -- 172.21.15.133:0/689425780 wait complete. 2024-03-07T06:53:23.164 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-07T06:53:23.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.360+0000 7f4c6f200640 1 Processor -- start 2024-03-07T06:53:23.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.360+0000 7f4c6f200640 1 -- start start 2024-03-07T06:53:23.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.362+0000 7f4c6f200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4c68146870 0x7f4c6806cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:23.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.362+0000 7f4c6f200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4c68147040 0x7f4c68072820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:23.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.362+0000 7f4c6f200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c680ade00 con 0x7f4c680ad840 2024-03-07T06:53:23.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.362+0000 7f4c6f200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c680ae0e0 con 0x7f4c68146870 2024-03-07T06:53:23.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.362+0000 7f4c6f200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c680ae3c0 con 0x7f4c68147040 2024-03-07T06:53:23.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.362+0000 7f4c6de00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4c68146870 0x7f4c6806cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:23.362 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.362+0000 7f4c6de00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4c68146870 0x7f4c6806cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54518/0 (socket says 172.21.15.133:54518) 2024-03-07T06:53:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.362+0000 7f4c6de00640 1 -- 172.21.15.133:0/3871324156 learned_addr learned my addr 172.21.15.133:0/3871324156 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.362+0000 7f4c6d400640 1 --2- 172.21.15.133:0/3871324156 >> v2:172.21.15.161:3300/0 conn(0x7f4c68147040 0x7f4c68072820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.363+0000 7f4c6de00640 1 -- 172.21.15.133:0/3871324156 >> v2:172.21.15.161:3300/0 conn(0x7f4c68147040 msgr2=0x7f4c68072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.363+0000 7f4c6de00640 1 --2- 172.21.15.133:0/3871324156 >> v2:172.21.15.161:3300/0 conn(0x7f4c68147040 0x7f4c68072820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.363+0000 7f4c6de00640 1 -- 172.21.15.133:0/3871324156 >> v1:172.21.15.161:6789/0 conn(0x7f4c680ad840 legacy=0x7f4c680b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.363+0000 7f4c6de00640 1 -- 172.21.15.133:0/3871324156 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c680ae6a0 con 0x7f4c68146870 2024-03-07T06:53:23.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.364+0000 7f4c6de00640 1 --2- 172.21.15.133:0/3871324156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c68146870 0x7f4c6806cf10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4c6806b420 tx=0x7f4c5c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebb07b2970382d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.364+0000 7f4c6ca00640 1 -- 172.21.15.133:0/3871324156 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c035020 con 0x7f4c68146870 2024-03-07T06:53:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.364+0000 7f4c6ca00640 1 -- 172.21.15.133:0/3871324156 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5c031640 con 0x7f4c68146870 2024-03-07T06:53:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.364+0000 7f4c6ca00640 1 -- 172.21.15.133:0/3871324156 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c02d040 con 0x7f4c68146870 2024-03-07T06:53:23.364 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.365+0000 7f4c6f200640 1 -- 172.21.15.133:0/3871324156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c68146870 msgr2=0x7f4c6806cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:23.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.365+0000 7f4c6f200640 1 --2- 172.21.15.133:0/3871324156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c68146870 0x7f4c6806cf10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4c6806b420 tx=0x7f4c5c02b8f0 comp rx=0 tx=0).stop 2024-03-07T06:53:23.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.365+0000 7f4c6f200640 1 -- 172.21.15.133:0/3871324156 shutdown_connections 2024-03-07T06:53:23.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.365+0000 7f4c6f200640 1 --2- 172.21.15.133:0/3871324156 >> v2:172.21.15.161:3300/0 conn(0x7f4c68147040 0x7f4c68072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:23.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.365+0000 7f4c6f200640 1 --2- 172.21.15.133:0/3871324156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c68146870 0x7f4c6806cf10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:23.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.365+0000 7f4c6f200640 1 -- 172.21.15.133:0/3871324156 >> 172.21.15.133:0/3871324156 conn(0x7f4c680b1fe0 msgr2=0x7f4c68070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:23.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.365+0000 7f4c6f200640 1 -- 172.21.15.133:0/3871324156 shutdown_connections 2024-03-07T06:53:23.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.365+0000 7f4c6f200640 1 -- 172.21.15.133:0/3871324156 wait complete. 2024-03-07T06:53:23.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.366+0000 7f4c6f200640 1 Processor -- start 2024-03-07T06:53:23.366 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.367+0000 7f4c6f200640 1 -- start start 2024-03-07T06:53:23.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.367+0000 7f4c6f200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4c680ad840 0x7f4c6815cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:23.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.367+0000 7f4c6f200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c68146870 0x7f4c6815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c68147040 0x7f4c6815b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680b4a30 con 0x7f4c68147040 2024-03-07T06:53:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6f200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680b48b0 con 0x7f4c68146870 2024-03-07T06:53:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6f200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68146c50 con 0x7f4c680ad840 2024-03-07T06:53:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c68147040 0x7f4c6815b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c68147040 0x7f4c6815b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54530/0 (socket says 172.21.15.133:54530) 2024-03-07T06:53:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6e800640 1 -- 172.21.15.133:0/2037917125 learned_addr learned my addr 172.21.15.133:0/2037917125 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6de00640 1 --2- 172.21.15.133:0/2037917125 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4c680ad840 0x7f4c6815cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6d400640 1 --2- 172.21.15.133:0/2037917125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c68146870 0x7f4c6815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6e800640 1 -- 172.21.15.133:0/2037917125 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4c680ad840 msgr2=0x7f4c6815cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6e800640 1 --2- 172.21.15.133:0/2037917125 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4c680ad840 0x7f4c6815cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6e800640 1 -- 172.21.15.133:0/2037917125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c68146870 msgr2=0x7f4c6815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6e800640 1 --2- 172.21.15.133:0/2037917125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c68146870 0x7f4c6815ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:23.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6e800640 1 -- 172.21.15.133:0/2037917125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6815ffe0 con 0x7f4c68147040 2024-03-07T06:53:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.368+0000 7f4c6e800640 1 --2- 172.21.15.133:0/2037917125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c68147040 0x7f4c6815b260 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400c780 tx=0x7f4c6400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.369+0000 7f4c62000640 1 -- 172.21.15.133:0/2037917125 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64004b60 con 0x7f4c68147040 2024-03-07T06:53:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.369+0000 7f4c62000640 1 -- 172.21.15.133:0/2037917125 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c64004d00 con 0x7f4c68147040 2024-03-07T06:53:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.369+0000 7f4c62000640 1 -- 172.21.15.133:0/2037917125 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6400aa90 con 0x7f4c68147040 2024-03-07T06:53:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.369+0000 7f4c6f200640 1 -- 172.21.15.133:0/2037917125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c68142710 con 0x7f4c68147040 2024-03-07T06:53:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.369+0000 7f4c6f200640 1 -- 172.21.15.133:0/2037917125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c68142cd0 con 0x7f4c68147040 2024-03-07T06:53:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.370+0000 7f4c6f200640 1 -- 172.21.15.133:0/2037917125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c68062c90 con 0x7f4c68147040 2024-03-07T06:53:23.370 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.371+0000 7f4c62000640 1 -- 172.21.15.133:0/2037917125 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f4c6400ac30 con 0x7f4c68147040 2024-03-07T06:53:23.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.372+0000 7f4c62000640 1 --2- 172.21.15.133:0/2037917125 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:23.371 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.372+0000 7f4c62000640 1 -- 172.21.15.133:0/2037917125 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4c6408a420 con 0x7f4c68147040 2024-03-07T06:53:23.372 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.373+0000 7f4c6de00640 1 --2- 172.21.15.133:0/2037917125 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:23.373 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.374+0000 7f4c6de00640 1 --2- 172.21.15.133:0/2037917125 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c036000 tx=0x7f4c5c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:23.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.377+0000 7f4c62000640 1 -- 172.21.15.133:0/2037917125 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4c64020070 con 0x7f4c68147040 2024-03-07T06:53:23.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:23.575+0000 7f4c6f200640 1 -- 172.21.15.133:0/2037917125 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4c680aef50 con 0x7f4c68147040 2024-03-07T06:53:24.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.151+0000 7f4c62000640 1 -- 172.21.15.133:0/2037917125 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4c64059350 con 0x7f4c68147040 2024-03-07T06:53:24.151 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2024-03-07T06:53:24.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.154+0000 7f4c6f200640 1 -- 172.21.15.133:0/2037917125 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f4c4c06b4d0 msgr2=0x7f4c4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:24.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.154+0000 7f4c6f200640 1 --2- 172.21.15.133:0/2037917125 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c036000 tx=0x7f4c5c008030 comp rx=0 tx=0).stop 2024-03-07T06:53:24.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.154+0000 7f4c6f200640 1 -- 172.21.15.133:0/2037917125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c68147040 msgr2=0x7f4c6815b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:24.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.154+0000 7f4c6f200640 1 --2- 172.21.15.133:0/2037917125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c68147040 0x7f4c6815b260 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400c780 tx=0x7f4c6400cc40 comp rx=0 tx=0).stop 2024-03-07T06:53:24.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.154+0000 7f4c6f200640 1 -- 172.21.15.133:0/2037917125 shutdown_connections 2024-03-07T06:53:24.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.154+0000 7f4c6f200640 1 --2- 172.21.15.133:0/2037917125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c68147040 0x7f4c6815b260 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:24.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.154+0000 7f4c6f200640 1 --2- 172.21.15.133:0/2037917125 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:24.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.154+0000 7f4c6f200640 1 --2- 172.21.15.133:0/2037917125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c68146870 0x7f4c6815ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:24.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.154+0000 7f4c6f200640 1 --2- 172.21.15.133:0/2037917125 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4c680ad840 0x7f4c6815cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:24.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.154+0000 7f4c6f200640 1 -- 172.21.15.133:0/2037917125 >> 172.21.15.133:0/2037917125 conn(0x7f4c680b1fe0 msgr2=0x7f4c680b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:24.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.155+0000 7f4c6f200640 1 -- 172.21.15.133:0/2037917125 shutdown_connections 2024-03-07T06:53:24.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.155+0000 7f4c6f200640 1 -- 172.21.15.133:0/2037917125 wait complete. 2024-03-07T06:53:24.190 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-07T06:53:24.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.374+0000 7f8b3b200640 1 Processor -- start 2024-03-07T06:53:24.374 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.374+0000 7f8b3b200640 1 -- start start 2024-03-07T06:53:24.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.376+0000 7f8b3b200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8b3c159880 0x7f8b3c0cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:24.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.376+0000 7f8b3b200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b3c0ccff0 0x7f8b3c160010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:24.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.376+0000 7f8b3b200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8b3c160550 0x7f8b3c0b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:24.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.376+0000 7f8b3b200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3c0cde00 con 0x7f8b3c160550 2024-03-07T06:53:24.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.376+0000 7f8b3b200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3c0cd560 con 0x7f8b3c159880 2024-03-07T06:53:24.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.376+0000 7f8b3b200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0cd840 con 0x7f8b3c0ccff0 2024-03-07T06:53:24.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.376+0000 7f8b3a800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8b3c160550 0x7f8b3c0b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:24.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.376+0000 7f8b3a800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8b3c160550 0x7f8b3c0b4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54538/0 (socket says 172.21.15.133:54538) 2024-03-07T06:53:24.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.376+0000 7f8b3a800640 1 -- 172.21.15.133:0/4270010932 learned_addr learned my addr 172.21.15.133:0/4270010932 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:24.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.376+0000 7f8b3a800640 1 -- 172.21.15.133:0/4270010932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b3c0ccff0 msgr2=0x7f8b3c160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:24.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.376+0000 7f8b3a800640 1 --2- 172.21.15.133:0/4270010932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b3c0ccff0 0x7f8b3c160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:24.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.376+0000 7f8b3a800640 1 -- 172.21.15.133:0/4270010932 >> v2:172.21.15.161:3300/0 conn(0x7f8b3c159880 msgr2=0x7f8b3c0cf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:24.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.376+0000 7f8b3a800640 1 --2- 172.21.15.133:0/4270010932 >> v2:172.21.15.161:3300/0 conn(0x7f8b3c159880 0x7f8b3c0cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:24.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.376+0000 7f8b3a800640 1 -- 172.21.15.133:0/4270010932 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c0cdb20 con 0x7f8b3c160550 2024-03-07T06:53:24.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.377+0000 7f8b3a800640 1 --2- 172.21.15.133:0/4270010932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c160550 0x7f8b3c0b4f10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c06b2b0 tx=0x7f8b3402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c3ca11e5035e0c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:24.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.378+0000 7f8b39400640 1 -- 172.21.15.133:0/4270010932 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3402ca80 con 0x7f8b3c160550 2024-03-07T06:53:24.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.378+0000 7f8b39400640 1 -- 172.21.15.133:0/4270010932 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3402cc20 con 0x7f8b3c160550 2024-03-07T06:53:24.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.378+0000 7f8b39400640 1 -- 172.21.15.133:0/4270010932 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b340343f0 con 0x7f8b3c160550 2024-03-07T06:53:24.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.378+0000 7f8b3b200640 1 -- 172.21.15.133:0/4270010932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c160550 msgr2=0x7f8b3c0b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:24.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.378+0000 7f8b3b200640 1 --2- 172.21.15.133:0/4270010932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c160550 0x7f8b3c0b4f10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c06b2b0 tx=0x7f8b3402bee0 comp rx=0 tx=0).stop 2024-03-07T06:53:24.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.378+0000 7f8b3b200640 1 -- 172.21.15.133:0/4270010932 shutdown_connections 2024-03-07T06:53:24.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.378+0000 7f8b3b200640 1 --2- 172.21.15.133:0/4270010932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c160550 0x7f8b3c0b4f10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:24.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.378+0000 7f8b3b200640 1 --2- 172.21.15.133:0/4270010932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b3c0ccff0 0x7f8b3c160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:24.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.378+0000 7f8b3b200640 1 --2- 172.21.15.133:0/4270010932 >> v2:172.21.15.161:3300/0 conn(0x7f8b3c159880 0x7f8b3c0cf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:24.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.378+0000 7f8b3b200640 1 -- 172.21.15.133:0/4270010932 >> 172.21.15.133:0/4270010932 conn(0x7f8b3c071860 msgr2=0x7f8b3c071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:24.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.379+0000 7f8b3b200640 1 -- 172.21.15.133:0/4270010932 shutdown_connections 2024-03-07T06:53:24.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.379+0000 7f8b3b200640 1 -- 172.21.15.133:0/4270010932 wait complete. 2024-03-07T06:53:24.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.380+0000 7f8b3b200640 1 Processor -- start 2024-03-07T06:53:24.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.380+0000 7f8b3b200640 1 -- start start 2024-03-07T06:53:24.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.381+0000 7f8b3b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c0ccff0 0x7f8b3c15bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:24.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.381+0000 7f8b3b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3c159880 0x7f8b3c153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:24.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.381+0000 7f8b3b200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b3c15c6b0 0x7f8b3c15d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:24.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.381+0000 7f8b3b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0cc6f0 con 0x7f8b3c0ccff0 2024-03-07T06:53:24.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.381+0000 7f8b3b200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c159c60 con 0x7f8b3c159880 2024-03-07T06:53:24.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.381+0000 7f8b3b200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c0cc870 con 0x7f8b3c15c6b0 2024-03-07T06:53:24.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.382+0000 7f8b39e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c0ccff0 0x7f8b3c15bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:24.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.382+0000 7f8b39e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c0ccff0 0x7f8b3c15bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54554/0 (socket says 172.21.15.133:54554) 2024-03-07T06:53:24.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.382+0000 7f8b39e00640 1 -- 172.21.15.133:0/11449398 learned_addr learned my addr 172.21.15.133:0/11449398 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:24.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.382+0000 7f8b3a800640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b3c15c6b0 0x7f8b3c15d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:24.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.382+0000 7f8b33e00640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3c159880 0x7f8b3c153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:24.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.382+0000 7f8b39e00640 1 -- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b3c15c6b0 msgr2=0x7f8b3c15d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:24.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.382+0000 7f8b39e00640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b3c15c6b0 0x7f8b3c15d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:24.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.382+0000 7f8b39e00640 1 -- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3c159880 msgr2=0x7f8b3c153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:24.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.382+0000 7f8b39e00640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3c159880 0x7f8b3c153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:24.383 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.382+0000 7f8b39e00640 1 -- 172.21.15.133:0/11449398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3c15aa20 con 0x7f8b3c0ccff0 2024-03-07T06:53:24.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.382+0000 7f8b33e00640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3c159880 0x7f8b3c153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:24.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.383+0000 7f8b39e00640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c0ccff0 0x7f8b3c15bac0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00b440 tx=0x7f8b2c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:24.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.383+0000 7f8b32000640 1 -- 172.21.15.133:0/11449398 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c015020 con 0x7f8b3c0ccff0 2024-03-07T06:53:24.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.383+0000 7f8b32000640 1 -- 172.21.15.133:0/11449398 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c011890 con 0x7f8b3c0ccff0 2024-03-07T06:53:24.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.383+0000 7f8b32000640 1 -- 172.21.15.133:0/11449398 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c011c80 con 0x7f8b3c0ccff0 2024-03-07T06:53:24.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.383+0000 7f8b3b200640 1 -- 172.21.15.133:0/11449398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3c15ad00 con 0x7f8b3c0ccff0 2024-03-07T06:53:24.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.383+0000 7f8b3b200640 1 -- 172.21.15.133:0/11449398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c15b2c0 con 0x7f8b3c0ccff0 2024-03-07T06:53:24.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.384+0000 7f8b3b200640 1 -- 172.21.15.133:0/11449398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3c067c80 con 0x7f8b3c0ccff0 2024-03-07T06:53:24.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.386+0000 7f8b32000640 1 -- 172.21.15.133:0/11449398 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8b2c018020 con 0x7f8b3c0ccff0 2024-03-07T06:53:24.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.387+0000 7f8b32000640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8b1806b4d0 0x7f8b1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:24.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.387+0000 7f8b32000640 1 -- 172.21.15.133:0/11449398 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f8b2c08a820 con 0x7f8b3c0ccff0 2024-03-07T06:53:24.387 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.388+0000 7f8b33e00640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8b1806b4d0 0x7f8b1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:24.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.389+0000 7f8b33e00640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8b1806b4d0 0x7f8b1806d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c153b40 tx=0x7f8b24008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:24.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.391+0000 7f8b32000640 1 -- 172.21.15.133:0/11449398 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f8b2c01f020 con 0x7f8b3c0ccff0 2024-03-07T06:53:24.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:24.625+0000 7f8b3b200640 1 -- 172.21.15.133:0/11449398 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8b3c0680a0 con 0x7f8b3c0ccff0 2024-03-07T06:53:25.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.154+0000 7f8b32000640 1 -- 172.21.15.133:0/11449398 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v23) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8b2c00d040 con 0x7f8b3c0ccff0 2024-03-07T06:53:25.154 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 6 and data pool 7 2024-03-07T06:53:25.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.160+0000 7f8b3b200640 1 -- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8b1806b4d0 msgr2=0x7f8b1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:25.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.160+0000 7f8b3b200640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8b1806b4d0 0x7f8b1806d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c153b40 tx=0x7f8b24008040 comp rx=0 tx=0).stop 2024-03-07T06:53:25.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.160+0000 7f8b3b200640 1 -- 172.21.15.133:0/11449398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c0ccff0 msgr2=0x7f8b3c15bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:25.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.160+0000 7f8b3b200640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c0ccff0 0x7f8b3c15bac0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00b440 tx=0x7f8b2c00b900 comp rx=0 tx=0).stop 2024-03-07T06:53:25.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.163+0000 7f8b3b200640 1 -- 172.21.15.133:0/11449398 shutdown_connections 2024-03-07T06:53:25.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.163+0000 7f8b3b200640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8b3c15c6b0 0x7f8b3c15d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.163+0000 7f8b3b200640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8b1806b4d0 0x7f8b1806d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.163+0000 7f8b3b200640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8b3c159880 0x7f8b3c153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.163+0000 7f8b3b200640 1 --2- 172.21.15.133:0/11449398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b3c0ccff0 0x7f8b3c15bac0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.163+0000 7f8b3b200640 1 -- 172.21.15.133:0/11449398 >> 172.21.15.133:0/11449398 conn(0x7f8b3c071860 msgr2=0x7f8b3c0b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:25.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.164+0000 7f8b3b200640 1 -- 172.21.15.133:0/11449398 shutdown_connections 2024-03-07T06:53:25.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.165+0000 7f8b3b200640 1 -- 172.21.15.133:0/11449398 wait complete. 2024-03-07T06:53:25.202 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:53:25.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.376+0000 7f48d7600640 1 Processor -- start 2024-03-07T06:53:25.376 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.376+0000 7f48d7600640 1 -- start start 2024-03-07T06:53:25.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.378+0000 7f48d7600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f48d015fbb0 0x7f48d00cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:25.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.378+0000 7f48d7600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d0072950 0x7f48d00cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:25.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.378+0000 7f48d7600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d00731b0 con 0x7f48d015ff90 2024-03-07T06:53:25.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.378+0000 7f48d7600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d0072f60 con 0x7f48d015fbb0 2024-03-07T06:53:25.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.378+0000 7f48d7600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d0073490 con 0x7f48d0072950 2024-03-07T06:53:25.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.378+0000 7f48d4c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f48d015fbb0 0x7f48d00cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:25.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.378+0000 7f48d4c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f48d015fbb0 0x7f48d00cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54568/0 (socket says 172.21.15.133:54568) 2024-03-07T06:53:25.378 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.378+0000 7f48d4c00640 1 -- 172.21.15.133:0/3303552780 learned_addr learned my addr 172.21.15.133:0/3303552780 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.378+0000 7f48cfe00640 1 --2- 172.21.15.133:0/3303552780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d0072950 0x7f48d00cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.378+0000 7f48d4c00640 1 -- 172.21.15.133:0/3303552780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d0072950 msgr2=0x7f48d00cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.378+0000 7f48d4c00640 1 --2- 172.21.15.133:0/3303552780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d0072950 0x7f48d00cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.378+0000 7f48d4c00640 1 -- 172.21.15.133:0/3303552780 >> v1:172.21.15.161:6789/0 conn(0x7f48d015ff90 legacy=0x7f48d00acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.378+0000 7f48d4c00640 1 -- 172.21.15.133:0/3303552780 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d0072150 con 0x7f48d015fbb0 2024-03-07T06:53:25.379 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.378+0000 7f48cfe00640 1 --2- 172.21.15.133:0/3303552780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d0072950 0x7f48d00cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:25.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.380+0000 7f48d4c00640 1 --2- 172.21.15.133:0/3303552780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48d015fbb0 0x7f48d00cf7d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f48d00730e0 tx=0x7f48c002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce03ae416f7669b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:25.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.380+0000 7f48cf400640 1 -- 172.21.15.133:0/3303552780 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c002c9c0 con 0x7f48d015fbb0 2024-03-07T06:53:25.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.380+0000 7f48cf400640 1 -- 172.21.15.133:0/3303552780 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c002cb60 con 0x7f48d015fbb0 2024-03-07T06:53:25.380 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.380+0000 7f48cf400640 1 -- 172.21.15.133:0/3303552780 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c00343f0 con 0x7f48d015fbb0 2024-03-07T06:53:25.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.380+0000 7f48d7600640 1 -- 172.21.15.133:0/3303552780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48d015fbb0 msgr2=0x7f48d00cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:25.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.380+0000 7f48d7600640 1 --2- 172.21.15.133:0/3303552780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48d015fbb0 0x7f48d00cf7d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f48d00730e0 tx=0x7f48c002be20 comp rx=0 tx=0).stop 2024-03-07T06:53:25.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.381+0000 7f48d7600640 1 -- 172.21.15.133:0/3303552780 shutdown_connections 2024-03-07T06:53:25.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.381+0000 7f48d7600640 1 --2- 172.21.15.133:0/3303552780 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d0072950 0x7f48d00cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.381+0000 7f48d7600640 1 --2- 172.21.15.133:0/3303552780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48d015fbb0 0x7f48d00cf7d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.381+0000 7f48d7600640 1 -- 172.21.15.133:0/3303552780 >> 172.21.15.133:0/3303552780 conn(0x7f48d00b3120 msgr2=0x7f48d00b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:25.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.381+0000 7f48d7600640 1 -- 172.21.15.133:0/3303552780 shutdown_connections 2024-03-07T06:53:25.381 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.381+0000 7f48d7600640 1 -- 172.21.15.133:0/3303552780 wait complete. 2024-03-07T06:53:25.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.382+0000 7f48d7600640 1 Processor -- start 2024-03-07T06:53:25.382 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.382+0000 7f48d7600640 1 -- start start 2024-03-07T06:53:25.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.383+0000 7f48d7600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d015fbb0 0x7f48d01599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:25.384 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.383+0000 7f48d7600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d015ff90 0x7f48d01f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.383+0000 7f48d7600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48d009ce70 0x7f48d009d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.383+0000 7f48d7600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00b1a00 con 0x7f48d009ce70 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.383+0000 7f48d7600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d0160980 con 0x7f48d015ff90 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.383+0000 7f48d7600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d01603e0 con 0x7f48d015fbb0 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.384+0000 7f48d5600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48d009ce70 0x7f48d009d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.384+0000 7f48d5600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48d009ce70 0x7f48d009d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54582/0 (socket says 172.21.15.133:54582) 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.384+0000 7f48d5600640 1 -- 172.21.15.133:0/3024814853 learned_addr learned my addr 172.21.15.133:0/3024814853 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.384+0000 7f48cfe00640 1 --2- 172.21.15.133:0/3024814853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d015ff90 0x7f48d01f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.384+0000 7f48d4c00640 1 --2- 172.21.15.133:0/3024814853 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d015fbb0 0x7f48d01599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.384+0000 7f48d5600640 1 -- 172.21.15.133:0/3024814853 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d015fbb0 msgr2=0x7f48d01599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.384+0000 7f48d5600640 1 --2- 172.21.15.133:0/3024814853 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d015fbb0 0x7f48d01599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.384+0000 7f48d5600640 1 -- 172.21.15.133:0/3024814853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d015ff90 msgr2=0x7f48d01f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.384+0000 7f48d5600640 1 --2- 172.21.15.133:0/3024814853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d015ff90 0x7f48d01f2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.384+0000 7f48d5600640 1 -- 172.21.15.133:0/3024814853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d015a2c0 con 0x7f48d009ce70 2024-03-07T06:53:25.385 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.384+0000 7f48d5600640 1 --2- 172.21.15.133:0/3024814853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48d009ce70 0x7f48d009d300 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f48c800c390 tx=0x7f48c800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.385+0000 7f48ccc00640 1 -- 172.21.15.133:0/3024814853 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c8017070 con 0x7f48d009ce70 2024-03-07T06:53:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.385+0000 7f48ccc00640 1 -- 172.21.15.133:0/3024814853 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c80035f0 con 0x7f48d009ce70 2024-03-07T06:53:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.385+0000 7f48ccc00640 1 -- 172.21.15.133:0/3024814853 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c80039e0 con 0x7f48d009ce70 2024-03-07T06:53:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.385+0000 7f48d7600640 1 -- 172.21.15.133:0/3024814853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d01f5060 con 0x7f48d009ce70 2024-03-07T06:53:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.385+0000 7f48d7600640 1 -- 172.21.15.133:0/3024814853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d01f5620 con 0x7f48d009ce70 2024-03-07T06:53:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.386+0000 7f48d7600640 1 -- 172.21.15.133:0/3024814853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d0062c90 con 0x7f48d009ce70 2024-03-07T06:53:25.386 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.387+0000 7f48ccc00640 1 -- 172.21.15.133:0/3024814853 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f48c801a070 con 0x7f48d009ce70 2024-03-07T06:53:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.388+0000 7f48ccc00640 1 --2- 172.21.15.133:0/3024814853 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f48ac06b4d0 0x7f48ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.388+0000 7f48ccc00640 1 -- 172.21.15.133:0/3024814853 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48c80895f0 con 0x7f48d009ce70 2024-03-07T06:53:25.388 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.389+0000 7f48d4c00640 1 --2- 172.21.15.133:0/3024814853 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f48ac06b4d0 0x7f48ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:25.389 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.390+0000 7f48d4c00640 1 --2- 172.21.15.133:0/3024814853 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f48ac06b4d0 0x7f48ac06d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f48c0035000 tx=0x7f48c0030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:25.393 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.393+0000 7f48ccc00640 1 -- 172.21.15.133:0/3024814853 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f48c8058470 con 0x7f48d009ce70 2024-03-07T06:53:25.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.574+0000 7f48d7600640 1 -- 172.21.15.133:0/3024814853 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48d00cf970 con 0x7f48d009ce70 2024-03-07T06:53:25.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.577+0000 7f48ccc00640 1 -- 172.21.15.133:0/3024814853 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16674 (secure 0 0 0) 0x7f48c8057d70 con 0x7f48d009ce70 2024-03-07T06:53:25.577 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:25.578 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":45,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:53:24.626983+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:53:22.398527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-07T06:53:23.628483+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:53:25.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.580+0000 7f48d7600640 1 -- 172.21.15.133:0/3024814853 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f48ac06b4d0 msgr2=0x7f48ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:25.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.580+0000 7f48d7600640 1 --2- 172.21.15.133:0/3024814853 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f48ac06b4d0 0x7f48ac06d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f48c0035000 tx=0x7f48c0030000 comp rx=0 tx=0).stop 2024-03-07T06:53:25.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.580+0000 7f48d7600640 1 -- 172.21.15.133:0/3024814853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48d009ce70 msgr2=0x7f48d009d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:25.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.580+0000 7f48d7600640 1 --2- 172.21.15.133:0/3024814853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48d009ce70 0x7f48d009d300 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f48c800c390 tx=0x7f48c800c850 comp rx=0 tx=0).stop 2024-03-07T06:53:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.580+0000 7f48d7600640 1 -- 172.21.15.133:0/3024814853 shutdown_connections 2024-03-07T06:53:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.580+0000 7f48d7600640 1 --2- 172.21.15.133:0/3024814853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48d009ce70 0x7f48d009d300 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.580+0000 7f48d7600640 1 --2- 172.21.15.133:0/3024814853 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f48ac06b4d0 0x7f48ac06d990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.580+0000 7f48d7600640 1 --2- 172.21.15.133:0/3024814853 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d015ff90 0x7f48d01f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.580+0000 7f48d7600640 1 --2- 172.21.15.133:0/3024814853 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d015fbb0 0x7f48d01599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.580+0000 7f48d7600640 1 -- 172.21.15.133:0/3024814853 >> 172.21.15.133:0/3024814853 conn(0x7f48d00b3120 msgr2=0x7f48d00aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.581+0000 7f48d7600640 1 -- 172.21.15.133:0/3024814853 shutdown_connections 2024-03-07T06:53:25.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.581+0000 7f48d7600640 1 -- 172.21.15.133:0/3024814853 wait complete. 2024-03-07T06:53:25.598 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:53:25.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.799+0000 7fbe20e00640 1 Processor -- start 2024-03-07T06:53:25.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.799+0000 7fbe20e00640 1 -- start start 2024-03-07T06:53:25.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.800+0000 7fbe20e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe1c15fbb0 0x7fbe1c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:25.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.800+0000 7fbe20e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbe1c0cdee0 0x7fbe1c15c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:25.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.800+0000 7fbe20e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe1c0731b0 con 0x7fbe1c072950 2024-03-07T06:53:25.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.800+0000 7fbe20e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1c072f60 con 0x7fbe1c0cdee0 2024-03-07T06:53:25.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.800+0000 7fbe20e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c073490 con 0x7fbe1c15fbb0 2024-03-07T06:53:25.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.801+0000 7fbe19e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe1c15fbb0 0x7fbe1c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:25.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.801+0000 7fbe1a800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbe1c0cdee0 0x7fbe1c15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:25.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.801+0000 7fbe19e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe1c15fbb0 0x7fbe1c0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:36240/0 (socket says 172.21.15.133:36240) 2024-03-07T06:53:25.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.801+0000 7fbe19e00640 1 -- 172.21.15.133:0/3429343971 learned_addr learned my addr 172.21.15.133:0/3429343971 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:25.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.802+0000 7fbe1a800640 1 -- 172.21.15.133:0/3429343971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe1c15fbb0 msgr2=0x7fbe1c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:25.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.802+0000 7fbe1a800640 1 --2- 172.21.15.133:0/3429343971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe1c15fbb0 0x7fbe1c0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.802+0000 7fbe1a800640 1 -- 172.21.15.133:0/3429343971 >> v1:172.21.15.161:6789/0 conn(0x7fbe1c072950 legacy=0x7fbe1c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:25.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.802+0000 7fbe1a800640 1 -- 172.21.15.133:0/3429343971 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c072150 con 0x7fbe1c0cdee0 2024-03-07T06:53:25.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.803+0000 7fbe1a800640 1 --2- 172.21.15.133:0/3429343971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe1c0cdee0 0x7fbe1c15c550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0730e0 tx=0x7fbe1402be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=889233e24ad1fcf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:25.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.804+0000 7fbe19400640 1 -- 172.21.15.133:0/3429343971 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14039070 con 0x7fbe1c0cdee0 2024-03-07T06:53:25.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.804+0000 7fbe19400640 1 -- 172.21.15.133:0/3429343971 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1402cbb0 con 0x7fbe1c0cdee0 2024-03-07T06:53:25.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.804+0000 7fbe19400640 1 -- 172.21.15.133:0/3429343971 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14034440 con 0x7fbe1c0cdee0 2024-03-07T06:53:25.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.804+0000 7fbe20e00640 1 -- 172.21.15.133:0/3429343971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe1c0cdee0 msgr2=0x7fbe1c15c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:25.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.804+0000 7fbe20e00640 1 --2- 172.21.15.133:0/3429343971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe1c0cdee0 0x7fbe1c15c550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0730e0 tx=0x7fbe1402be00 comp rx=0 tx=0).stop 2024-03-07T06:53:25.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.804+0000 7fbe20e00640 1 -- 172.21.15.133:0/3429343971 shutdown_connections 2024-03-07T06:53:25.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.804+0000 7fbe20e00640 1 --2- 172.21.15.133:0/3429343971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe1c0cdee0 0x7fbe1c15c550 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.804+0000 7fbe20e00640 1 --2- 172.21.15.133:0/3429343971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe1c15fbb0 0x7fbe1c0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.804+0000 7fbe20e00640 1 -- 172.21.15.133:0/3429343971 >> 172.21.15.133:0/3429343971 conn(0x7fbe1c0b3120 msgr2=0x7fbe1c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:25.804 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.804+0000 7fbe20e00640 1 -- 172.21.15.133:0/3429343971 shutdown_connections 2024-03-07T06:53:25.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.804+0000 7fbe20e00640 1 -- 172.21.15.133:0/3429343971 wait complete. 2024-03-07T06:53:25.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.806+0000 7fbe20e00640 1 Processor -- start 2024-03-07T06:53:25.806 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.806+0000 7fbe20e00640 1 -- start start 2024-03-07T06:53:25.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.807+0000 7fbe20e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe1c0cdee0 0x7fbe1c0b5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:25.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.807+0000 7fbe20e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe1c15fbb0 0x7fbe1c157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:25.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.807+0000 7fbe20e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe1c0afbf0 0x7fbe1c1575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:25.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.807+0000 7fbe20e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c160980 con 0x7fbe1c0afbf0 2024-03-07T06:53:25.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.807+0000 7fbe20e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c0ce2f0 con 0x7fbe1c0cdee0 2024-03-07T06:53:25.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.807+0000 7fbe20e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c0b1a00 con 0x7fbe1c15fbb0 2024-03-07T06:53:25.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.807+0000 7fbe1a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe1c0afbf0 0x7fbe1c1575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:25.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.807+0000 7fbe1a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe1c0afbf0 0x7fbe1c1575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54598/0 (socket says 172.21.15.133:54598) 2024-03-07T06:53:25.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.807+0000 7fbe1a800640 1 -- 172.21.15.133:0/2319727111 learned_addr learned my addr 172.21.15.133:0/2319727111 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:25.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.808+0000 7fbe1a800640 1 -- 172.21.15.133:0/2319727111 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe1c15fbb0 msgr2=0x7fbe1c157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:25.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.808+0000 7fbe1a800640 1 --2- 172.21.15.133:0/2319727111 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe1c15fbb0 0x7fbe1c157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.808+0000 7fbe1a800640 1 -- 172.21.15.133:0/2319727111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe1c0cdee0 msgr2=0x7fbe1c0b5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:25.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.808+0000 7fbe1a800640 1 --2- 172.21.15.133:0/2319727111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe1c0cdee0 0x7fbe1c0b5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:25.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.808+0000 7fbe1a800640 1 -- 172.21.15.133:0/2319727111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c0b5a90 con 0x7fbe1c0afbf0 2024-03-07T06:53:25.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.808+0000 7fbe1a800640 1 --2- 172.21.15.133:0/2319727111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe1c0afbf0 0x7fbe1c1575b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009e80 tx=0x7fbe14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:25.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.808+0000 7fbe12000640 1 -- 172.21.15.133:0/2319727111 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14039050 con 0x7fbe1c0afbf0 2024-03-07T06:53:25.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.808+0000 7fbe12000640 1 -- 172.21.15.133:0/2319727111 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe14045070 con 0x7fbe1c0afbf0 2024-03-07T06:53:25.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.808+0000 7fbe20e00640 1 -- 172.21.15.133:0/2319727111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1c0b5d70 con 0x7fbe1c0afbf0 2024-03-07T06:53:25.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.808+0000 7fbe12000640 1 -- 172.21.15.133:0/2319727111 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14009450 con 0x7fbe1c0afbf0 2024-03-07T06:53:25.809 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.808+0000 7fbe20e00640 1 -- 172.21.15.133:0/2319727111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1c1596e0 con 0x7fbe1c0afbf0 2024-03-07T06:53:25.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.809+0000 7fbe20e00640 1 -- 172.21.15.133:0/2319727111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1c0b6290 con 0x7fbe1c0afbf0 2024-03-07T06:53:25.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.810+0000 7fbe12000640 1 -- 172.21.15.133:0/2319727111 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbe1403e850 con 0x7fbe1c0afbf0 2024-03-07T06:53:25.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.812+0000 7fbe12000640 1 --2- 172.21.15.133:0/2319727111 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbdf806b4d0 0x7fbdf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:25.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.812+0000 7fbe12000640 1 -- 172.21.15.133:0/2319727111 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe140acf40 con 0x7fbe1c0afbf0 2024-03-07T06:53:25.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.812+0000 7fbe19e00640 1 --2- 172.21.15.133:0/2319727111 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbdf806b4d0 0x7fbdf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:25.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.813+0000 7fbe19e00640 1 --2- 172.21.15.133:0/2319727111 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbdf806b4d0 0x7fbdf806d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c006f40 tx=0x7fbe0c006e90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:25.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.816+0000 7fbe12000640 1 -- 172.21.15.133:0/2319727111 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbe1c0b6290 con 0x7fbe1c0afbf0 2024-03-07T06:53:25.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.996+0000 7fbe20e00640 1 -- 172.21.15.133:0/2319727111 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe1c0cf410 con 0x7fbe1c0afbf0 2024-03-07T06:53:25.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:25.999+0000 7fbe12000640 1 -- 172.21.15.133:0/2319727111 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16674 (secure 0 0 0) 0x7fbe1c0cf410 con 0x7fbe1c0afbf0 2024-03-07T06:53:25.999 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:26.000 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":45,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:53:24.626983+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:53:22.398527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-07T06:53:23.628483+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:53:26.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.002+0000 7fbe20e00640 1 -- 172.21.15.133:0/2319727111 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbdf806b4d0 msgr2=0x7fbdf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:26.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.002+0000 7fbe20e00640 1 --2- 172.21.15.133:0/2319727111 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbdf806b4d0 0x7fbdf806d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c006f40 tx=0x7fbe0c006e90 comp rx=0 tx=0).stop 2024-03-07T06:53:26.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.002+0000 7fbe20e00640 1 -- 172.21.15.133:0/2319727111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe1c0afbf0 msgr2=0x7fbe1c1575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:26.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.002+0000 7fbe20e00640 1 --2- 172.21.15.133:0/2319727111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe1c0afbf0 0x7fbe1c1575b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009e80 tx=0x7fbe14004a40 comp rx=0 tx=0).stop 2024-03-07T06:53:26.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.002+0000 7fbe20e00640 1 -- 172.21.15.133:0/2319727111 shutdown_connections 2024-03-07T06:53:26.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.002+0000 7fbe20e00640 1 --2- 172.21.15.133:0/2319727111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe1c0afbf0 0x7fbe1c1575b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:26.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.002+0000 7fbe20e00640 1 --2- 172.21.15.133:0/2319727111 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbe1c15fbb0 0x7fbe1c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:26.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.002+0000 7fbe20e00640 1 --2- 172.21.15.133:0/2319727111 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbdf806b4d0 0x7fbdf806d990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:26.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.002+0000 7fbe20e00640 1 --2- 172.21.15.133:0/2319727111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbe1c0cdee0 0x7fbe1c0b5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:26.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.002+0000 7fbe20e00640 1 -- 172.21.15.133:0/2319727111 >> 172.21.15.133:0/2319727111 conn(0x7fbe1c0b3120 msgr2=0x7fbe1c15ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:26.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.002+0000 7fbe20e00640 1 -- 172.21.15.133:0/2319727111 shutdown_connections 2024-03-07T06:53:26.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.002+0000 7fbe20e00640 1 -- 172.21.15.133:0/2319727111 wait complete. 2024-03-07T06:53:26.018 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-07T06:53:26.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.219+0000 7fd3ad000640 1 Processor -- start 2024-03-07T06:53:26.219 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.219+0000 7fd3ad000640 1 -- start start 2024-03-07T06:53:26.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.221+0000 7fd3ad000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3a80b3ed0 0x7fd3a815b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:26.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.221+0000 7fd3ad000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd3a806d180 0x7fd3a815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:26.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.221+0000 7fd3ad000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a806d740 con 0x7fd3a815b370 2024-03-07T06:53:26.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.221+0000 7fd3ad000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a815c1d0 con 0x7fd3a806d180 2024-03-07T06:53:26.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.221+0000 7fd3ad000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a806da20 con 0x7fd3a80b3ed0 2024-03-07T06:53:26.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.221+0000 7fd3a6800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd3a806d180 0x7fd3a815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:26.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.221+0000 7fd3a6800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fd3a806d180 0x7fd3a815bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54612/0 (socket says 172.21.15.133:54612) 2024-03-07T06:53:26.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.221+0000 7fd3a6800640 1 -- 172.21.15.133:0/3347491828 learned_addr learned my addr 172.21.15.133:0/3347491828 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:26.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.221+0000 7fd3a6800640 1 -- 172.21.15.133:0/3347491828 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3a80b3ed0 msgr2=0x7fd3a815b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:26.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.221+0000 7fd3a6800640 1 --2- 172.21.15.133:0/3347491828 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3a80b3ed0 0x7fd3a815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:26.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.221+0000 7fd3a6800640 1 -- 172.21.15.133:0/3347491828 >> v1:172.21.15.161:6789/0 conn(0x7fd3a815b370 legacy=0x7fd3a80cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:26.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.222+0000 7fd3a6800640 1 -- 172.21.15.133:0/3347491828 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a806dd00 con 0x7fd3a806d180 2024-03-07T06:53:26.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.223+0000 7fd3a6800640 1 --2- 172.21.15.133:0/3347491828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3a806d180 0x7fd3a815bc90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000d150 tx=0x7fd3a002e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d006e9cbaba61d32 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:26.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.223+0000 7fd3a4a00640 1 -- 172.21.15.133:0/3347491828 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a002ea70 con 0x7fd3a806d180 2024-03-07T06:53:26.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.223+0000 7fd3a4a00640 1 -- 172.21.15.133:0/3347491828 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a002ec10 con 0x7fd3a806d180 2024-03-07T06:53:26.222 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.223+0000 7fd3a4a00640 1 -- 172.21.15.133:0/3347491828 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0009600 con 0x7fd3a806d180 2024-03-07T06:53:26.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.223+0000 7fd3ad000640 1 -- 172.21.15.133:0/3347491828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3a806d180 msgr2=0x7fd3a815bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:26.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.223+0000 7fd3ad000640 1 --2- 172.21.15.133:0/3347491828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3a806d180 0x7fd3a815bc90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000d150 tx=0x7fd3a002e0d0 comp rx=0 tx=0).stop 2024-03-07T06:53:26.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.224+0000 7fd3ad000640 1 -- 172.21.15.133:0/3347491828 shutdown_connections 2024-03-07T06:53:26.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.224+0000 7fd3ad000640 1 --2- 172.21.15.133:0/3347491828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3a806d180 0x7fd3a815bc90 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:26.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.224+0000 7fd3ad000640 1 --2- 172.21.15.133:0/3347491828 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3a80b3ed0 0x7fd3a815b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:26.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.224+0000 7fd3ad000640 1 -- 172.21.15.133:0/3347491828 >> 172.21.15.133:0/3347491828 conn(0x7fd3a80b1d30 msgr2=0x7fd3a80b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:26.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.224+0000 7fd3ad000640 1 -- 172.21.15.133:0/3347491828 shutdown_connections 2024-03-07T06:53:26.224 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.224+0000 7fd3ad000640 1 -- 172.21.15.133:0/3347491828 wait complete. 2024-03-07T06:53:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.225+0000 7fd3ad000640 1 Processor -- start 2024-03-07T06:53:26.225 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.225+0000 7fd3ad000640 1 -- start start 2024-03-07T06:53:26.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.226+0000 7fd3ad000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3a806d180 0x7fd3a81f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:26.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.226+0000 7fd3ad000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3a80b3ed0 0x7fd3a81e5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:26.227 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.226+0000 7fd3ad000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3a815b370 0x7fd3a813f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.226+0000 7fd3ad000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a80cf660 con 0x7fd3a815b370 2024-03-07T06:53:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.226+0000 7fd3ad000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a806e2c0 con 0x7fd3a80b3ed0 2024-03-07T06:53:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.226+0000 7fd3ad000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a80cf7e0 con 0x7fd3a806d180 2024-03-07T06:53:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.227+0000 7fd3a6800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3a815b370 0x7fd3a813f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.227+0000 7fd3a6800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3a815b370 0x7fd3a813f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54614/0 (socket says 172.21.15.133:54614) 2024-03-07T06:53:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.227+0000 7fd3a6800640 1 -- 172.21.15.133:0/4095468045 learned_addr learned my addr 172.21.15.133:0/4095468045 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.227+0000 7fd3a5e00640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3a806d180 0x7fd3a81f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.227+0000 7fd3a5400640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3a80b3ed0 0x7fd3a81e5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:26.228 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.227+0000 7fd3a6800640 1 -- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3a806d180 msgr2=0x7fd3a81f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.227+0000 7fd3a6800640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3a806d180 0x7fd3a81f7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.227+0000 7fd3a6800640 1 -- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3a80b3ed0 msgr2=0x7fd3a81e5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.227+0000 7fd3a6800640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3a80b3ed0 0x7fd3a81e5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.227+0000 7fd3a6800640 1 -- 172.21.15.133:0/4095468045 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a81f8220 con 0x7fd3a815b370 2024-03-07T06:53:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.227+0000 7fd3a5e00640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3a806d180 0x7fd3a81f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.227+0000 7fd3a5400640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3a80b3ed0 0x7fd3a81e5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.227+0000 7fd3a6800640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3a815b370 0x7fd3a813f0c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0038040 tx=0x7fd3a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:26.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.228+0000 7fd39e000640 1 -- 172.21.15.133:0/4095468045 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a000b720 con 0x7fd3a815b370 2024-03-07T06:53:26.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.228+0000 7fd39e000640 1 -- 172.21.15.133:0/4095468045 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a0045070 con 0x7fd3a815b370 2024-03-07T06:53:26.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.228+0000 7fd39e000640 1 -- 172.21.15.133:0/4095468045 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0036a60 con 0x7fd3a815b370 2024-03-07T06:53:26.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.228+0000 7fd3ad000640 1 -- 172.21.15.133:0/4095468045 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a8150040 con 0x7fd3a815b370 2024-03-07T06:53:26.230 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.228+0000 7fd3ad000640 1 -- 172.21.15.133:0/4095468045 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a814d7e0 con 0x7fd3a815b370 2024-03-07T06:53:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.229+0000 7fd3ad000640 1 -- 172.21.15.133:0/4095468045 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a8062c90 con 0x7fd3a815b370 2024-03-07T06:53:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.230+0000 7fd39e000640 1 -- 172.21.15.133:0/4095468045 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd3a0036c00 con 0x7fd3a815b370 2024-03-07T06:53:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.231+0000 7fd39e000640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd38806b500 0x7fd38806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.231+0000 7fd39e000640 1 -- 172.21.15.133:0/4095468045 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3a00acc20 con 0x7fd3a815b370 2024-03-07T06:53:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.232+0000 7fd3a5e00640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd38806b500 0x7fd38806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.233+0000 7fd3a5e00640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd38806b500 0x7fd38806d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd39800a810 tx=0x7fd398008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:26.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.236+0000 7fd39e000640 1 -- 172.21.15.133:0/4095468045 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd3a004e020 con 0x7fd3a815b370 2024-03-07T06:53:26.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:26.471+0000 7fd3ad000640 1 -- 172.21.15.133:0/4095468045 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd3a8150e80 con 0x7fd3a815b370 2024-03-07T06:53:27.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.195+0000 7fd39e000640 1 -- 172.21.15.133:0/4095468045 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd3a81f8220 con 0x7fd3a815b370 2024-03-07T06:53:27.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.199+0000 7fd3ad000640 1 -- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd38806b500 msgr2=0x7fd38806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:27.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.199+0000 7fd3ad000640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd38806b500 0x7fd38806d9c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd39800a810 tx=0x7fd398008040 comp rx=0 tx=0).stop 2024-03-07T06:53:27.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.199+0000 7fd3ad000640 1 -- 172.21.15.133:0/4095468045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3a815b370 msgr2=0x7fd3a813f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:27.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.199+0000 7fd3ad000640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3a815b370 0x7fd3a813f0c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0038040 tx=0x7fd3a0004a40 comp rx=0 tx=0).stop 2024-03-07T06:53:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.200+0000 7fd3ad000640 1 -- 172.21.15.133:0/4095468045 shutdown_connections 2024-03-07T06:53:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.200+0000 7fd3ad000640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3a815b370 0x7fd3a813f0c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.200+0000 7fd3ad000640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd38806b500 0x7fd38806d9c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.200+0000 7fd3ad000640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3a80b3ed0 0x7fd3a81e5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.200+0000 7fd3ad000640 1 --2- 172.21.15.133:0/4095468045 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3a806d180 0x7fd3a81f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.200+0000 7fd3ad000640 1 -- 172.21.15.133:0/4095468045 >> 172.21.15.133:0/4095468045 conn(0x7fd3a80b1d30 msgr2=0x7fd3a8152710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.200+0000 7fd3ad000640 1 -- 172.21.15.133:0/4095468045 shutdown_connections 2024-03-07T06:53:27.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.200+0000 7fd3ad000640 1 -- 172.21.15.133:0/4095468045 wait complete. 2024-03-07T06:53:27.224 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-07T06:53:27.224 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-07T06:53:27.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.443+0000 7f958a800640 1 Processor -- start 2024-03-07T06:53:27.443 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.443+0000 7f958a800640 1 -- start start 2024-03-07T06:53:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.445+0000 7f958a800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f95840ccff0 0x7f95840cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.445+0000 7f958a800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f95840b4f10 0x7f95840b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.445+0000 7f958a800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9584160720 con 0x7f9584159880 2024-03-07T06:53:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.445+0000 7f958a800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95840b5880 con 0x7f95840ccff0 2024-03-07T06:53:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.445+0000 7f958a800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95840cde00 con 0x7f95840b4f10 2024-03-07T06:53:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.445+0000 7f9588a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f95840ccff0 0x7f95840cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:27.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.445+0000 7f9588a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f95840ccff0 0x7f95840cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54622/0 (socket says 172.21.15.133:54622) 2024-03-07T06:53:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.445+0000 7f9588a00640 1 -- 172.21.15.133:0/3993005723 learned_addr learned my addr 172.21.15.133:0/3993005723 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.446+0000 7f9588a00640 1 -- 172.21.15.133:0/3993005723 >> v2:172.21.15.161:3300/0 conn(0x7f95840b4f10 msgr2=0x7f95840b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.446+0000 7f9588a00640 1 --2- 172.21.15.133:0/3993005723 >> v2:172.21.15.161:3300/0 conn(0x7f95840b4f10 0x7f95840b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.446+0000 7f9588a00640 1 -- 172.21.15.133:0/3993005723 >> v1:172.21.15.161:6789/0 conn(0x7f9584159880 legacy=0x7f9584160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.446+0000 7f9588a00640 1 -- 172.21.15.133:0/3993005723 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95840cd560 con 0x7f95840ccff0 2024-03-07T06:53:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.447+0000 7f9588a00640 1 --2- 172.21.15.133:0/3993005723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95840ccff0 0x7f95840cf2b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f95840cfb00 tx=0x7f957002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9eeec4ce218da9 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:27.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.447+0000 7f9583e00640 1 -- 172.21.15.133:0/3993005723 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570035020 con 0x7f95840ccff0 2024-03-07T06:53:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.447+0000 7f9583e00640 1 -- 172.21.15.133:0/3993005723 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9570031870 con 0x7f95840ccff0 2024-03-07T06:53:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.447+0000 7f9583e00640 1 -- 172.21.15.133:0/3993005723 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570031c40 con 0x7f95840ccff0 2024-03-07T06:53:27.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.448+0000 7f958a800640 1 -- 172.21.15.133:0/3993005723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95840ccff0 msgr2=0x7f95840cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.448+0000 7f958a800640 1 --2- 172.21.15.133:0/3993005723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95840ccff0 0x7f95840cf2b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f95840cfb00 tx=0x7f957002b910 comp rx=0 tx=0).stop 2024-03-07T06:53:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.448+0000 7f958a800640 1 -- 172.21.15.133:0/3993005723 shutdown_connections 2024-03-07T06:53:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.448+0000 7f958a800640 1 --2- 172.21.15.133:0/3993005723 >> v2:172.21.15.161:3300/0 conn(0x7f95840b4f10 0x7f95840b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.448+0000 7f958a800640 1 --2- 172.21.15.133:0/3993005723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95840ccff0 0x7f95840cf2b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.448+0000 7f958a800640 1 -- 172.21.15.133:0/3993005723 >> 172.21.15.133:0/3993005723 conn(0x7f9584071860 msgr2=0x7f9584071c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.448+0000 7f958a800640 1 -- 172.21.15.133:0/3993005723 shutdown_connections 2024-03-07T06:53:27.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.448+0000 7f958a800640 1 -- 172.21.15.133:0/3993005723 wait complete. 2024-03-07T06:53:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.449+0000 7f958a800640 1 Processor -- start 2024-03-07T06:53:27.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.450+0000 7f958a800640 1 -- start start 2024-03-07T06:53:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f958a800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95840b4f10 0x7f958415e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f958a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95840ccff0 0x7f95840b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f958a800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9584159880 0x7f95841e5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f958a800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958415faf0 con 0x7f95840ccff0 2024-03-07T06:53:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f958a800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95840cdb20 con 0x7f95840b4f10 2024-03-07T06:53:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f958a800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9584159c60 con 0x7f9584159880 2024-03-07T06:53:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f9588a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95840ccff0 0x7f95840b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:27.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f9588a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95840ccff0 0x7f95840b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54630/0 (socket says 172.21.15.133:54630) 2024-03-07T06:53:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f9588a00640 1 -- 172.21.15.133:0/3264060830 learned_addr learned my addr 172.21.15.133:0/3264060830 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f9589e00640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9584159880 0x7f95841e5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f9589400640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95840b4f10 0x7f958415e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f9588a00640 1 -- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9584159880 msgr2=0x7f95841e5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f9588a00640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9584159880 0x7f95841e5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f9588a00640 1 -- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95840b4f10 msgr2=0x7f958415e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f9588a00640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95840b4f10 0x7f958415e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f9588a00640 1 -- 172.21.15.133:0/3264060830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958415eb80 con 0x7f95840ccff0 2024-03-07T06:53:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.451+0000 7f9589e00640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9584159880 0x7f95841e5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:27.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.452+0000 7f9588a00640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95840ccff0 0x7f95840b0970 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f957002b950 tx=0x7f95700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:27.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.452+0000 7f9581600640 1 -- 172.21.15.133:0/3264060830 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95700443f0 con 0x7f95840ccff0 2024-03-07T06:53:27.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.452+0000 7f9581600640 1 -- 172.21.15.133:0/3264060830 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9570044590 con 0x7f95840ccff0 2024-03-07T06:53:27.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.452+0000 7f958a800640 1 -- 172.21.15.133:0/3264060830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958415ee60 con 0x7f95840ccff0 2024-03-07T06:53:27.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.452+0000 7f9581600640 1 -- 172.21.15.133:0/3264060830 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570044980 con 0x7f95840ccff0 2024-03-07T06:53:27.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.452+0000 7f958a800640 1 -- 172.21.15.133:0/3264060830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958415f420 con 0x7f95840ccff0 2024-03-07T06:53:27.453 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.453+0000 7f958a800640 1 -- 172.21.15.133:0/3264060830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958415e8d0 con 0x7f95840ccff0 2024-03-07T06:53:27.454 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.454+0000 7f9581600640 1 -- 172.21.15.133:0/3264060830 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9570038020 con 0x7f95840ccff0 2024-03-07T06:53:27.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.455+0000 7f9581600640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f956406b4d0 0x7f956406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:27.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.455+0000 7f9581600640 1 -- 172.21.15.133:0/3264060830 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95700ac4f0 con 0x7f95840ccff0 2024-03-07T06:53:27.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.456+0000 7f9589400640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f956406b4d0 0x7f956406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:27.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.457+0000 7f9589400640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f956406b4d0 0x7f956406d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f95780015d0 tx=0x7f9578009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:27.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.460+0000 7f9581600640 1 -- 172.21.15.133:0/3264060830 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f957003f070 con 0x7f95840ccff0 2024-03-07T06:53:27.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:27.696+0000 7f958a800640 1 -- 172.21.15.133:0/3264060830 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f95840680a0 con 0x7f95840ccff0 2024-03-07T06:53:28.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.206+0000 7f9581600640 1 -- 172.21.15.133:0/3264060830 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) v1 ==== 122+0+0 (secure 0 0 0) 0x7f957007b360 con 0x7f95840ccff0 2024-03-07T06:53:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.210+0000 7f958a800640 1 -- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f956406b4d0 msgr2=0x7f956406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.210+0000 7f958a800640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f956406b4d0 0x7f956406d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f95780015d0 tx=0x7f9578009040 comp rx=0 tx=0).stop 2024-03-07T06:53:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.210+0000 7f958a800640 1 -- 172.21.15.133:0/3264060830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95840ccff0 msgr2=0x7f95840b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.210+0000 7f958a800640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95840ccff0 0x7f95840b0970 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f957002b950 tx=0x7f95700046c0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.210+0000 7f958a800640 1 -- 172.21.15.133:0/3264060830 shutdown_connections 2024-03-07T06:53:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.210+0000 7f958a800640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9584159880 0x7f95841e5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.210+0000 7f958a800640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95840ccff0 0x7f95840b0970 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.210+0000 7f958a800640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f956406b4d0 0x7f956406d990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.210+0000 7f958a800640 1 --2- 172.21.15.133:0/3264060830 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95840b4f10 0x7f958415e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.210+0000 7f958a800640 1 -- 172.21.15.133:0/3264060830 >> 172.21.15.133:0/3264060830 conn(0x7f9584071860 msgr2=0x7f9584160010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:28.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.210+0000 7f958a800640 1 -- 172.21.15.133:0/3264060830 shutdown_connections 2024-03-07T06:53:28.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.210+0000 7f958a800640 1 -- 172.21.15.133:0/3264060830 wait complete. 2024-03-07T06:53:28.232 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-07T06:53:28.232 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:53:28.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.433+0000 7f9d3cc00640 1 Processor -- start 2024-03-07T06:53:28.433 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.433+0000 7f9d3cc00640 1 -- start start 2024-03-07T06:53:28.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.435+0000 7f9d3cc00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9d3806e1c0 0x7f9d38066eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:28.434 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.435+0000 7f9d3cc00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d38067650 con 0x7f9d3815bf40 2024-03-07T06:53:28.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.435+0000 7f9d3cc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d380ac460 con 0x7f9d380ac000 2024-03-07T06:53:28.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.435+0000 7f9d3cc00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d38070e60 con 0x7f9d3806e1c0 2024-03-07T06:53:28.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.435+0000 7f9d35e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9d3815bf40 0x7f9d380712f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55178/0 (socket says 172.21.15.133:55178) 2024-03-07T06:53:28.435 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.435+0000 7f9d35e00640 1 -- 172.21.15.133:0/3331445763 learned_addr learned my addr 172.21.15.133:0/3331445763 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:28.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.435+0000 7f9d36800640 1 --2- 172.21.15.133:0/3331445763 >> v2:172.21.15.161:3300/0 conn(0x7f9d3806e1c0 0x7f9d38066eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:28.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 -- 172.21.15.133:0/3331445763 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 909085285 0 0) 0x7f9d38067650 con 0x7f9d3815bf40 2024-03-07T06:53:28.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 -- 172.21.15.133:0/3331445763 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d1c003520 con 0x7f9d3815bf40 2024-03-07T06:53:28.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 -- 172.21.15.133:0/3331445763 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2434958606 0 0) 0x7f9d1c003520 con 0x7f9d3815bf40 2024-03-07T06:53:28.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 -- 172.21.15.133:0/3331445763 >> v2:172.21.15.161:3300/0 conn(0x7f9d3806e1c0 msgr2=0x7f9d38066eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:28.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 --2- 172.21.15.133:0/3331445763 >> v2:172.21.15.161:3300/0 conn(0x7f9d3806e1c0 0x7f9d38066eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 -- 172.21.15.133:0/3331445763 >> v1:172.21.15.161:6789/0 conn(0x7f9d380ac000 legacy=0x7f9d3806dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:28.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 -- 172.21.15.133:0/3331445763 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d38070530 con 0x7f9d3815bf40 2024-03-07T06:53:28.437 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 -- 172.21.15.133:0/3331445763 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f9d280019f0 con 0x7f9d3815bf40 2024-03-07T06:53:28.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 -- 172.21.15.133:0/3331445763 >> v1:172.21.15.133:6789/0 conn(0x7f9d3815bf40 legacy=0x7f9d380712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:28.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 --2- 172.21.15.133:0/3331445763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d1c0039b0 0x7f9d1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:28.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 --2- 172.21.15.133:0/3331445763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d1c004800 0x7f9d1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:28.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 --2- 172.21.15.133:0/3331445763 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1c0051c0 0x7f9d1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:28.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 -- 172.21.15.133:0/3331445763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38070e60 con 0x7f9d1c004800 2024-03-07T06:53:28.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 -- 172.21.15.133:0/3331445763 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c00c160 con 0x7f9d1c0039b0 2024-03-07T06:53:28.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.436+0000 7f9d35400640 1 -- 172.21.15.133:0/3331445763 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c00c360 con 0x7f9d1c0051c0 2024-03-07T06:53:28.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.437+0000 7f9d2d400640 1 --2- 172.21.15.133:0/3331445763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d1c004800 0x7f9d1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:28.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.437+0000 7f9d2d400640 1 -- 172.21.15.133:0/3331445763 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1c0051c0 msgr2=0x7f9d1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:28.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.437+0000 7f9d2d400640 1 --2- 172.21.15.133:0/3331445763 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d1c0051c0 0x7f9d1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.437+0000 7f9d35e00640 1 --2- 172.21.15.133:0/3331445763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d1c0039b0 0x7f9d1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:28.438 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.437+0000 7f9d2d400640 1 -- 172.21.15.133:0/3331445763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d1c0039b0 msgr2=0x7f9d1c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.437+0000 7f9d2d400640 1 --2- 172.21.15.133:0/3331445763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d1c0039b0 0x7f9d1c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.437+0000 7f9d2d400640 1 -- 172.21.15.133:0/3331445763 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c003520 con 0x7f9d1c004800 2024-03-07T06:53:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.437+0000 7f9d35e00640 1 --2- 172.21.15.133:0/3331445763 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d1c0039b0 0x7f9d1c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:28.439 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.437+0000 7f9d36800640 1 -- 172.21.15.133:0/3331445763 reap_dead start 2024-03-07T06:53:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.438+0000 7f9d2d400640 1 --2- 172.21.15.133:0/3331445763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d1c004800 0x7f9d1c004c80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9d20001750 tx=0x7f9d1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af992bb223bd107 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.438+0000 7f9d35400640 1 -- 172.21.15.133:0/3331445763 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d20001370 con 0x7f9d1c004800 2024-03-07T06:53:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.438+0000 7f9d35400640 1 -- 172.21.15.133:0/3331445763 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d200052d0 con 0x7f9d1c004800 2024-03-07T06:53:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.439+0000 7f9d35400640 1 -- 172.21.15.133:0/3331445763 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d20003830 con 0x7f9d1c004800 2024-03-07T06:53:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.439+0000 7f9d3cc00640 1 -- 172.21.15.133:0/3331445763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d1c004800 msgr2=0x7f9d1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:28.440 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.439+0000 7f9d3cc00640 1 --2- 172.21.15.133:0/3331445763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d1c004800 0x7f9d1c004c80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9d20001750 tx=0x7f9d1c005830 comp rx=0 tx=0).stop 2024-03-07T06:53:28.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.439+0000 7f9d3cc00640 1 -- 172.21.15.133:0/3331445763 shutdown_connections 2024-03-07T06:53:28.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.439+0000 7f9d3cc00640 1 --2- 172.21.15.133:0/3331445763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d1c004800 0x7f9d1c004c80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.439+0000 7f9d3cc00640 1 -- 172.21.15.133:0/3331445763 >> 172.21.15.133:0/3331445763 conn(0x7f9d380cd160 msgr2=0x7f9d380cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:28.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.439+0000 7f9d3cc00640 1 -- 172.21.15.133:0/3331445763 shutdown_connections 2024-03-07T06:53:28.441 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.439+0000 7f9d3cc00640 1 -- 172.21.15.133:0/3331445763 wait complete. 2024-03-07T06:53:28.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.442+0000 7f9d3cc00640 1 Processor -- start 2024-03-07T06:53:28.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.443+0000 7f9d3cc00640 1 -- start start 2024-03-07T06:53:28.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.444+0000 7f9d3cc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d1c004800 0x7f9d3814c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:28.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.444+0000 7f9d3cc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d3814f9c0 0x7f9d3814be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:28.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.444+0000 7f9d3cc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d3814d0b0 0x7f9d3814d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:28.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.444+0000 7f9d3cc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38066eb0 con 0x7f9d1c004800 2024-03-07T06:53:28.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.444+0000 7f9d3cc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38070af0 con 0x7f9d3814d0b0 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.444+0000 7f9d3cc00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38067030 con 0x7f9d3814f9c0 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.444+0000 7f9d35e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d1c004800 0x7f9d3814c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.444+0000 7f9d35e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d1c004800 0x7f9d3814c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54648/0 (socket says 172.21.15.133:54648) 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.444+0000 7f9d35e00640 1 -- 172.21.15.133:0/2371835598 learned_addr learned my addr 172.21.15.133:0/2371835598 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.445+0000 7f9d35e00640 1 -- 172.21.15.133:0/2371835598 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d3814f9c0 msgr2=0x7f9d3814be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.445+0000 7f9d2d400640 1 --2- 172.21.15.133:0/2371835598 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d3814f9c0 0x7f9d3814be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.445+0000 7f9d35e00640 1 --2- 172.21.15.133:0/2371835598 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d3814f9c0 0x7f9d3814be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.445+0000 7f9d35e00640 1 -- 172.21.15.133:0/2371835598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d3814d0b0 msgr2=0x7f9d3814d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.445+0000 7f9d35e00640 1 --2- 172.21.15.133:0/2371835598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d3814d0b0 0x7f9d3814d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.445+0000 7f9d35e00640 1 -- 172.21.15.133:0/2371835598 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3814fe40 con 0x7f9d1c004800 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.445+0000 7f9d35e00640 1 --2- 172.21.15.133:0/2371835598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d1c004800 0x7f9d3814c600 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9d280020d0 tx=0x7f9d2800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.445+0000 7f9d2ea00640 1 -- 172.21.15.133:0/2371835598 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28019070 con 0x7f9d1c004800 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.446+0000 7f9d2ea00640 1 -- 172.21.15.133:0/2371835598 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d28004b60 con 0x7f9d1c004800 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.446+0000 7f9d2ea00640 1 -- 172.21.15.133:0/2371835598 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d280070d0 con 0x7f9d1c004800 2024-03-07T06:53:28.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.446+0000 7f9d3cc00640 1 -- 172.21.15.133:0/2371835598 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3815c570 con 0x7f9d1c004800 2024-03-07T06:53:28.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.446+0000 7f9d3cc00640 1 -- 172.21.15.133:0/2371835598 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3815cb30 con 0x7f9d1c004800 2024-03-07T06:53:28.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.447+0000 7f9d3cc00640 1 -- 172.21.15.133:0/2371835598 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d38062c90 con 0x7f9d1c004800 2024-03-07T06:53:28.447 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.448+0000 7f9d2ea00640 1 -- 172.21.15.133:0/2371835598 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9d2801c070 con 0x7f9d1c004800 2024-03-07T06:53:28.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.448+0000 7f9d2ea00640 1 --2- 172.21.15.133:0/2371835598 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9d1406b500 0x7f9d1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:28.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.449+0000 7f9d2ea00640 1 -- 172.21.15.133:0/2371835598 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d2808a270 con 0x7f9d1c004800 2024-03-07T06:53:28.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.449+0000 7f9d2d400640 1 --2- 172.21.15.133:0/2371835598 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9d1406b500 0x7f9d1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:28.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.450+0000 7f9d2d400640 1 --2- 172.21.15.133:0/2371835598 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9d1406b500 0x7f9d1406d9c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9d20033000 tx=0x7f9d20009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:28.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.457+0000 7f9d2ea00640 1 -- 172.21.15.133:0/2371835598 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9d280590e0 con 0x7f9d1c004800 2024-03-07T06:53:28.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.907+0000 7f9d3cc00640 1 -- 172.21.15.133:0/2371835598 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d380ce1c0 con 0x7f9d1c004800 2024-03-07T06:53:28.907 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.908+0000 7f9d2ea00640 1 -- 172.21.15.133:0/2371835598 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4980 (secure 0 0 0) 0x7f9d280589e0 con 0x7f9d1c004800 2024-03-07T06:53:28.908 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:28.908 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/44975892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":44975892},{"type":"v1","addr":"172.21.15.161:6835","nonce":44975892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4972,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/906115542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":906115542},{"type":"v1","addr":"172.21.15.133:6833","nonce":906115542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1998424684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1998424684},{"type":"v1","addr":"172.21.15.161:6837","nonce":1998424684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:53:24.627534+0000","modified":"2024-03-07T06:53:27.698471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/2951618124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2951618124},{"type":"v1","addr":"172.21.15.133:6835","nonce":2951618124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":3}]} 2024-03-07T06:53:28.908 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 27 2024-03-07T06:53:28.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.911+0000 7f9d3cc00640 1 -- 172.21.15.133:0/2371835598 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9d1406b500 msgr2=0x7f9d1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:28.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.911+0000 7f9d3cc00640 1 --2- 172.21.15.133:0/2371835598 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9d1406b500 0x7f9d1406d9c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9d20033000 tx=0x7f9d20009030 comp rx=0 tx=0).stop 2024-03-07T06:53:28.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.911+0000 7f9d3cc00640 1 -- 172.21.15.133:0/2371835598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d1c004800 msgr2=0x7f9d3814c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:28.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.911+0000 7f9d3cc00640 1 --2- 172.21.15.133:0/2371835598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d1c004800 0x7f9d3814c600 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9d280020d0 tx=0x7f9d2800b4a0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.911+0000 7f9d3cc00640 1 -- 172.21.15.133:0/2371835598 shutdown_connections 2024-03-07T06:53:28.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.912+0000 7f9d3cc00640 1 --2- 172.21.15.133:0/2371835598 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9d1406b500 0x7f9d1406d9c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.912+0000 7f9d3cc00640 1 --2- 172.21.15.133:0/2371835598 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9d3814d0b0 0x7f9d3814d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.912+0000 7f9d3cc00640 1 --2- 172.21.15.133:0/2371835598 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9d3814f9c0 0x7f9d3814be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.912+0000 7f9d3cc00640 1 --2- 172.21.15.133:0/2371835598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d1c004800 0x7f9d3814c600 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:28.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.912+0000 7f9d3cc00640 1 -- 172.21.15.133:0/2371835598 >> 172.21.15.133:0/2371835598 conn(0x7f9d380cd160 msgr2=0x7f9d380ab640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:28.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.912+0000 7f9d3cc00640 1 -- 172.21.15.133:0/2371835598 shutdown_connections 2024-03-07T06:53:28.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:28.912+0000 7f9d3cc00640 1 -- 172.21.15.133:0/2371835598 wait complete. 2024-03-07T06:53:28.937 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:53:29.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.131+0000 7f7ea9000640 1 Processor -- start 2024-03-07T06:53:29.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.131+0000 7f7ea9000640 1 -- start start 2024-03-07T06:53:29.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.133+0000 7f7ea9000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7ea415be00 0x7f7ea40d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:29.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.133+0000 7f7ea9000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7ea4160240 0x7f7ea40cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:29.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.133+0000 7f7ea9000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea40b7910 con 0x7f7ea40d0800 2024-03-07T06:53:29.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.133+0000 7f7ea9000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea40b7bf0 con 0x7f7ea415be00 2024-03-07T06:53:29.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.133+0000 7f7ea9000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea40ceb20 con 0x7f7ea4160240 2024-03-07T06:53:29.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.133+0000 7f7ea1e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7ea415be00 0x7f7ea40d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:29.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.133+0000 7f7ea1e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7ea415be00 0x7f7ea40d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54656/0 (socket says 172.21.15.133:54656) 2024-03-07T06:53:29.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.133+0000 7f7ea1e00640 1 -- 172.21.15.133:0/3406597902 learned_addr learned my addr 172.21.15.133:0/3406597902 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:29.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.133+0000 7f7ea2800640 1 --2- 172.21.15.133:0/3406597902 >> v2:172.21.15.161:3300/0 conn(0x7f7ea4160240 0x7f7ea40cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:29.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.134+0000 7f7ea1e00640 1 -- 172.21.15.133:0/3406597902 >> v2:172.21.15.161:3300/0 conn(0x7f7ea4160240 msgr2=0x7f7ea40cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:29.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.134+0000 7f7ea1e00640 1 --2- 172.21.15.133:0/3406597902 >> v2:172.21.15.161:3300/0 conn(0x7f7ea4160240 0x7f7ea40cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.134+0000 7f7ea1e00640 1 -- 172.21.15.133:0/3406597902 >> v1:172.21.15.161:6789/0 conn(0x7f7ea40d0800 legacy=0x7f7ea415fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:29.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.134+0000 7f7ea1e00640 1 -- 172.21.15.133:0/3406597902 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea40cee00 con 0x7f7ea415be00 2024-03-07T06:53:29.134 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.134+0000 7f7ea2800640 1 --2- 172.21.15.133:0/3406597902 >> v2:172.21.15.161:3300/0 conn(0x7f7ea4160240 0x7f7ea40cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:29.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.135+0000 7f7ea1e00640 1 --2- 172.21.15.133:0/3406597902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ea415be00 0x7f7ea40d02c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4160b00 tx=0x7f7e9402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f3f5513d0ef7b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:29.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.135+0000 7f7ea1400640 1 -- 172.21.15.133:0/3406597902 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9402e070 con 0x7f7ea415be00 2024-03-07T06:53:29.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.135+0000 7f7ea1400640 1 -- 172.21.15.133:0/3406597902 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e94037070 con 0x7f7ea415be00 2024-03-07T06:53:29.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.135+0000 7f7ea1400640 1 -- 172.21.15.133:0/3406597902 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94031a00 con 0x7f7ea415be00 2024-03-07T06:53:29.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.136+0000 7f7ea9000640 1 -- 172.21.15.133:0/3406597902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ea415be00 msgr2=0x7f7ea40d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:29.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.136+0000 7f7ea9000640 1 --2- 172.21.15.133:0/3406597902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ea415be00 0x7f7ea40d02c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4160b00 tx=0x7f7e9402b880 comp rx=0 tx=0).stop 2024-03-07T06:53:29.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.136+0000 7f7ea9000640 1 -- 172.21.15.133:0/3406597902 shutdown_connections 2024-03-07T06:53:29.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.136+0000 7f7ea9000640 1 --2- 172.21.15.133:0/3406597902 >> v2:172.21.15.161:3300/0 conn(0x7f7ea4160240 0x7f7ea40cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.136+0000 7f7ea9000640 1 --2- 172.21.15.133:0/3406597902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ea415be00 0x7f7ea40d02c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.136+0000 7f7ea9000640 1 -- 172.21.15.133:0/3406597902 >> 172.21.15.133:0/3406597902 conn(0x7f7ea4066e50 msgr2=0x7f7ea4067260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:29.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.136+0000 7f7ea9000640 1 -- 172.21.15.133:0/3406597902 shutdown_connections 2024-03-07T06:53:29.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.136+0000 7f7ea9000640 1 -- 172.21.15.133:0/3406597902 wait complete. 2024-03-07T06:53:29.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.138+0000 7f7ea9000640 1 Processor -- start 2024-03-07T06:53:29.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.138+0000 7f7ea9000640 1 -- start start 2024-03-07T06:53:29.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.139+0000 7f7ea9000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ea40d0800 0x7f7ea4144ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:29.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.139+0000 7f7ea9000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea415be00 0x7f7ea4071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:29.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.139+0000 7f7ea9000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ea4160240 0x7f7ea4153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:29.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.139+0000 7f7ea9000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40cf9e0 con 0x7f7ea4160240 2024-03-07T06:53:29.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.139+0000 7f7ea9000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40cf860 con 0x7f7ea415be00 2024-03-07T06:53:29.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.139+0000 7f7ea9000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40cfb60 con 0x7f7ea40d0800 2024-03-07T06:53:29.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.140+0000 7f7ea2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ea4160240 0x7f7ea4153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:29.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.140+0000 7f7ea2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ea4160240 0x7f7ea4153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54658/0 (socket says 172.21.15.133:54658) 2024-03-07T06:53:29.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.140+0000 7f7ea2800640 1 -- 172.21.15.133:0/611585415 learned_addr learned my addr 172.21.15.133:0/611585415 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:29.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.140+0000 7f7ea1e00640 1 --2- 172.21.15.133:0/611585415 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ea40d0800 0x7f7ea4144ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:29.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.140+0000 7f7ea2800640 1 -- 172.21.15.133:0/611585415 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ea40d0800 msgr2=0x7f7ea4144ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:29.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.140+0000 7f7e9be00640 1 --2- 172.21.15.133:0/611585415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea415be00 0x7f7ea4071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:29.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.140+0000 7f7ea2800640 1 --2- 172.21.15.133:0/611585415 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ea40d0800 0x7f7ea4144ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.140+0000 7f7ea2800640 1 -- 172.21.15.133:0/611585415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea415be00 msgr2=0x7f7ea4071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:29.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.140+0000 7f7ea2800640 1 --2- 172.21.15.133:0/611585415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea415be00 0x7f7ea4071f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.140+0000 7f7ea2800640 1 -- 172.21.15.133:0/611585415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea4072a20 con 0x7f7ea4160240 2024-03-07T06:53:29.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.140+0000 7f7ea2800640 1 --2- 172.21.15.133:0/611585415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ea4160240 0x7f7ea4153d30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00cd50 tx=0x7f7e9c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:29.141 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.141+0000 7f7e9a000640 1 -- 172.21.15.133:0/611585415 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00f680 con 0x7f7ea4160240 2024-03-07T06:53:29.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.141+0000 7f7e9a000640 1 -- 172.21.15.133:0/611585415 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c00f840 con 0x7f7ea4160240 2024-03-07T06:53:29.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.141+0000 7f7ea9000640 1 -- 172.21.15.133:0/611585415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea4072d00 con 0x7f7ea4160240 2024-03-07T06:53:29.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.141+0000 7f7e9a000640 1 -- 172.21.15.133:0/611585415 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00fc30 con 0x7f7ea4160240 2024-03-07T06:53:29.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.141+0000 7f7ea9000640 1 -- 172.21.15.133:0/611585415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea41502a0 con 0x7f7ea4160240 2024-03-07T06:53:29.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.142+0000 7f7ea9000640 1 -- 172.21.15.133:0/611585415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea40cbae0 con 0x7f7ea4160240 2024-03-07T06:53:29.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.143+0000 7f7e9a000640 1 -- 172.21.15.133:0/611585415 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7e9c0213f0 con 0x7f7ea4160240 2024-03-07T06:53:29.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.144+0000 7f7e9a000640 1 --2- 172.21.15.133:0/611585415 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7e8006b4d0 0x7f7e8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:29.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.144+0000 7f7e9a000640 1 -- 172.21.15.133:0/611585415 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e9c08afe0 con 0x7f7ea4160240 2024-03-07T06:53:29.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.145+0000 7f7ea1e00640 1 --2- 172.21.15.133:0/611585415 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7e8006b4d0 0x7f7e8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:29.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.146+0000 7f7ea1e00640 1 --2- 172.21.15.133:0/611585415 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7e8006b4d0 0x7f7e8006d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e94036000 tx=0x7f7e94008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:29.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.149+0000 7f7e9a000640 1 -- 172.21.15.133:0/611585415 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7e9c014030 con 0x7f7ea4160240 2024-03-07T06:53:29.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.347+0000 7f7ea9000640 1 -- 172.21.15.133:0/611585415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ea4062c90 con 0x7f7ea4160240 2024-03-07T06:53:29.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.349+0000 7f7e9a000640 1 -- 172.21.15.133:0/611585415 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16661 (secure 0 0 0) 0x7f7e9c0597b0 con 0x7f7ea4160240 2024-03-07T06:53:29.349 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:29.349 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":46,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:53:26.170797+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:53:22.398527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-07T06:53:23.628483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:53:29.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.351+0000 7f7ea9000640 1 -- 172.21.15.133:0/611585415 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7e8006b4d0 msgr2=0x7f7e8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:29.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.351+0000 7f7ea9000640 1 --2- 172.21.15.133:0/611585415 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7e8006b4d0 0x7f7e8006d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7e94036000 tx=0x7f7e94008030 comp rx=0 tx=0).stop 2024-03-07T06:53:29.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.351+0000 7f7ea9000640 1 -- 172.21.15.133:0/611585415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ea4160240 msgr2=0x7f7ea4153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:29.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.351+0000 7f7ea9000640 1 --2- 172.21.15.133:0/611585415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ea4160240 0x7f7ea4153d30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00cd50 tx=0x7f7e9c00a650 comp rx=0 tx=0).stop 2024-03-07T06:53:29.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.352+0000 7f7ea9000640 1 -- 172.21.15.133:0/611585415 shutdown_connections 2024-03-07T06:53:29.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.352+0000 7f7ea9000640 1 --2- 172.21.15.133:0/611585415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ea4160240 0x7f7ea4153d30 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.352+0000 7f7ea9000640 1 --2- 172.21.15.133:0/611585415 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7e8006b4d0 0x7f7e8006d990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.352+0000 7f7ea9000640 1 --2- 172.21.15.133:0/611585415 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea415be00 0x7f7ea4071f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.352+0000 7f7ea9000640 1 --2- 172.21.15.133:0/611585415 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ea40d0800 0x7f7ea4144ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.352+0000 7f7ea9000640 1 -- 172.21.15.133:0/611585415 >> 172.21.15.133:0/611585415 conn(0x7f7ea4066e50 msgr2=0x7f7ea4159d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:29.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.352+0000 7f7ea9000640 1 -- 172.21.15.133:0/611585415 shutdown_connections 2024-03-07T06:53:29.352 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.352+0000 7f7ea9000640 1 -- 172.21.15.133:0/611585415 wait complete. 2024-03-07T06:53:29.370 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-07T06:53:29.370 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:53:29.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.633+0000 7f5ce7e00640 1 Processor -- start 2024-03-07T06:53:29.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.633+0000 7f5ce7e00640 1 -- start start 2024-03-07T06:53:29.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.634+0000 7f5ce7e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5ce815bd30 0x7f5ce806e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:29.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.635+0000 7f5ce7e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5ce80cf5d0 0x7f5ce8072830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:29.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.635+0000 7f5ce7e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce8160760 con 0x7f5ce8072d70 2024-03-07T06:53:29.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.635+0000 7f5ce7e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce80731e0 con 0x7f5ce815bd30 2024-03-07T06:53:29.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.635+0000 7f5ce7e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce815f9a0 con 0x7f5ce80cf5d0 2024-03-07T06:53:29.635 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.635+0000 7f5ce6a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5ce815bd30 0x7f5ce806e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:29.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.635+0000 7f5ce6a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5ce815bd30 0x7f5ce806e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54664/0 (socket says 172.21.15.133:54664) 2024-03-07T06:53:29.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.635+0000 7f5ce6a00640 1 -- 172.21.15.133:0/3501639035 learned_addr learned my addr 172.21.15.133:0/3501639035 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:29.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.635+0000 7f5ce6000640 1 --2- 172.21.15.133:0/3501639035 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5ce80cf5d0 0x7f5ce8072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:29.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.635+0000 7f5ce6a00640 1 -- 172.21.15.133:0/3501639035 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5ce80cf5d0 msgr2=0x7f5ce8072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:29.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.635+0000 7f5ce6a00640 1 --2- 172.21.15.133:0/3501639035 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5ce80cf5d0 0x7f5ce8072830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.635+0000 7f5ce6a00640 1 -- 172.21.15.133:0/3501639035 >> v1:172.21.15.161:6789/0 conn(0x7f5ce8072d70 legacy=0x7f5ce806cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:29.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.636+0000 7f5ce6a00640 1 -- 172.21.15.133:0/3501639035 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce815fc80 con 0x7f5ce815bd30 2024-03-07T06:53:29.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.636+0000 7f5ce6000640 1 --2- 172.21.15.133:0/3501639035 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5ce80cf5d0 0x7f5ce8072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:29.636 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.637+0000 7f5ce6a00640 1 --2- 172.21.15.133:0/3501639035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ce815bd30 0x7f5ce806e8e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009870 tx=0x7f5cd802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0448ef4c2fb server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:29.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.638+0000 7f5ce5600640 1 -- 172.21.15.133:0/3501639035 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8035020 con 0x7f5ce815bd30 2024-03-07T06:53:29.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.638+0000 7f5ce5600640 1 -- 172.21.15.133:0/3501639035 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd8031640 con 0x7f5ce815bd30 2024-03-07T06:53:29.637 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.638+0000 7f5ce5600640 1 -- 172.21.15.133:0/3501639035 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd802d040 con 0x7f5ce815bd30 2024-03-07T06:53:29.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.638+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3501639035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ce815bd30 msgr2=0x7f5ce806e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:29.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.638+0000 7f5ce7e00640 1 --2- 172.21.15.133:0/3501639035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ce815bd30 0x7f5ce806e8e0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009870 tx=0x7f5cd802b8f0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.638+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3501639035 shutdown_connections 2024-03-07T06:53:29.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.638+0000 7f5ce7e00640 1 --2- 172.21.15.133:0/3501639035 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5ce80cf5d0 0x7f5ce8072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.638+0000 7f5ce7e00640 1 --2- 172.21.15.133:0/3501639035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ce815bd30 0x7f5ce806e8e0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.638+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3501639035 >> 172.21.15.133:0/3501639035 conn(0x7f5ce80b2e90 msgr2=0x7f5ce80b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:29.638 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.638+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3501639035 shutdown_connections 2024-03-07T06:53:29.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.638+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3501639035 wait complete. 2024-03-07T06:53:29.639 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.640+0000 7f5ce7e00640 1 Processor -- start 2024-03-07T06:53:29.640 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.640+0000 7f5ce7e00640 1 -- start start 2024-03-07T06:53:29.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.641+0000 7f5ce7e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ce8072d70 0x7f5ce815af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:29.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.641+0000 7f5ce7e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5ce80cf5d0 0x7f5ce8153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:29.641 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.641+0000 7f5ce7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ce815bd30 0x7f5ce815a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:29.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.641+0000 7f5ce7e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce806ee20 con 0x7f5ce815bd30 2024-03-07T06:53:29.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.641+0000 7f5ce7e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce8160240 con 0x7f5ce8072d70 2024-03-07T06:53:29.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.641+0000 7f5ce7e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce806f130 con 0x7f5ce80cf5d0 2024-03-07T06:53:29.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.641+0000 7f5ce7400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ce815bd30 0x7f5ce815a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:29.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.641+0000 7f5ce7400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ce815bd30 0x7f5ce815a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54668/0 (socket says 172.21.15.133:54668) 2024-03-07T06:53:29.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.641+0000 7f5ce7400640 1 -- 172.21.15.133:0/3689910137 learned_addr learned my addr 172.21.15.133:0/3689910137 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:29.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.642+0000 7f5ce6000640 1 --2- 172.21.15.133:0/3689910137 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5ce80cf5d0 0x7f5ce8153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:29.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.642+0000 7f5ce6a00640 1 --2- 172.21.15.133:0/3689910137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ce8072d70 0x7f5ce815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:29.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.642+0000 7f5ce7400640 1 -- 172.21.15.133:0/3689910137 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5ce80cf5d0 msgr2=0x7f5ce8153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:29.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.642+0000 7f5ce7400640 1 --2- 172.21.15.133:0/3689910137 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5ce80cf5d0 0x7f5ce8153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.642+0000 7f5ce7400640 1 -- 172.21.15.133:0/3689910137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ce8072d70 msgr2=0x7f5ce815af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:29.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.642+0000 7f5ce7400640 1 --2- 172.21.15.133:0/3689910137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ce8072d70 0x7f5ce815af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.642 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.642+0000 7f5ce7400640 1 -- 172.21.15.133:0/3689910137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce8153b70 con 0x7f5ce815bd30 2024-03-07T06:53:29.643 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.642+0000 7f5ce7400640 1 --2- 172.21.15.133:0/3689910137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ce815bd30 0x7f5ce815a180 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000b970 tx=0x7f5ce000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:29.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.643+0000 7f5cdea00640 1 -- 172.21.15.133:0/3689910137 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0019070 con 0x7f5ce815bd30 2024-03-07T06:53:29.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.643+0000 7f5cdea00640 1 -- 172.21.15.133:0/3689910137 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce000cb40 con 0x7f5ce815bd30 2024-03-07T06:53:29.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.643+0000 7f5cdea00640 1 -- 172.21.15.133:0/3689910137 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0014440 con 0x7f5ce815bd30 2024-03-07T06:53:29.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.643+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3689910137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce80ce220 con 0x7f5ce815bd30 2024-03-07T06:53:29.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.643+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3689910137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce80cd740 con 0x7f5ce815bd30 2024-03-07T06:53:29.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.644+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3689910137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce80acd10 con 0x7f5ce815bd30 2024-03-07T06:53:29.644 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.645+0000 7f5cdea00640 1 -- 172.21.15.133:0/3689910137 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5ce001c070 con 0x7f5ce815bd30 2024-03-07T06:53:29.645 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.646+0000 7f5cdea00640 1 --2- 172.21.15.133:0/3689910137 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5cc806b4d0 0x7f5cc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:29.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.646+0000 7f5cdea00640 1 -- 172.21.15.133:0/3689910137 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ce0089e20 con 0x7f5ce815bd30 2024-03-07T06:53:29.646 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.647+0000 7f5ce6a00640 1 --2- 172.21.15.133:0/3689910137 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5cc806b4d0 0x7f5cc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:29.647 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.648+0000 7f5ce6a00640 1 --2- 172.21.15.133:0/3689910137 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5cc806b4d0 0x7f5cc806d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8036000 tx=0x7f5cd803c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:29.650 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.651+0000 7f5cdea00640 1 -- 172.21.15.133:0/3689910137 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f5ce80acd10 con 0x7f5ce815bd30 2024-03-07T06:53:29.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.862+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3689910137 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ce8062c90 con 0x7f5cc806b4d0 2024-03-07T06:53:29.879 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.880+0000 7f5cdea00640 1 -- 172.21.15.133:0/3689910137 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239859 (secure 0 0 0) 0x7f5ce8062c90 con 0x7f5cc806b4d0 2024-03-07T06:53:29.883 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:29.884 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:53:29.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.885+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3689910137 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5cc806b4d0 msgr2=0x7f5cc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:29.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.885+0000 7f5ce7e00640 1 --2- 172.21.15.133:0/3689910137 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5cc806b4d0 0x7f5cc806d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8036000 tx=0x7f5cd803c040 comp rx=0 tx=0).stop 2024-03-07T06:53:29.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.885+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3689910137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ce815bd30 msgr2=0x7f5ce815a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:29.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.885+0000 7f5ce7e00640 1 --2- 172.21.15.133:0/3689910137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ce815bd30 0x7f5ce815a180 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000b970 tx=0x7f5ce000be30 comp rx=0 tx=0).stop 2024-03-07T06:53:29.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.885+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3689910137 shutdown_connections 2024-03-07T06:53:29.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.885+0000 7f5ce7e00640 1 --2- 172.21.15.133:0/3689910137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ce815bd30 0x7f5ce815a180 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.885+0000 7f5ce7e00640 1 --2- 172.21.15.133:0/3689910137 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5ce80cf5d0 0x7f5ce8153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.886 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.885+0000 7f5ce7e00640 1 --2- 172.21.15.133:0/3689910137 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f5cc806b4d0 0x7f5cc806d990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.885+0000 7f5ce7e00640 1 --2- 172.21.15.133:0/3689910137 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5ce8072d70 0x7f5ce815af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:29.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.885+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3689910137 >> 172.21.15.133:0/3689910137 conn(0x7f5ce80b2e90 msgr2=0x7f5ce80af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:29.887 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.885+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3689910137 shutdown_connections 2024-03-07T06:53:29.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:29.885+0000 7f5ce7e00640 1 -- 172.21.15.133:0/3689910137 wait complete. 2024-03-07T06:53:29.902 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-03-07T06:53:28.518027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":246196,"kb_used_data":5036,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754728524,"statfs":{"total":773094113280,"available":772842008576,"internally_reserved":0,"allocated":5156864,"data_stored":2755267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.381368"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.144907+0000","last_change":"2024-03-07T06:53:23.144907+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.144907+0000","last_undegraded":"2024-03-07T06:53:23.144907+0000","last_fullsized":"2024-03-07T06:53:23.144907+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.143029+0000","last_change":"2024-03-07T06:53:23.143029+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.143029+0000","last_undegraded":"2024-03-07T06:53:23.143029+0000","last_fullsized":"2024-03-07T06:53:23.143029+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.146433+0000","last_change":"2024-03-07T06:53:23.146433+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.146433+0000","last_undegraded":"2024-03-07T06:53:23.146433+0000","last_fullsized":"2024-03-07T06:53:23.146433+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.153163+0000","last_change":"2024-03-07T06:53:23.153163+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.153163+0000","last_undegraded":"2024-03-07T06:53:23.153163+0000","last_fullsized":"2024-03-07T06:53:23.153163+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.144751+0000","last_change":"2024-03-07T06:53:23.144751+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.144751+0000","last_undegraded":"2024-03-07T06:53:23.144751+0000","last_fullsized":"2024-03-07T06:53:23.144751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.142881+0000","last_change":"2024-03-07T06:53:23.142881+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.142881+0000","last_undegraded":"2024-03-07T06:53:23.142881+0000","last_fullsized":"2024-03-07T06:53:23.142881+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.148958+0000","last_change":"2024-03-07T06:53:23.148958+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.148958+0000","last_undegraded":"2024-03-07T06:53:23.148958+0000","last_fullsized":"2024-03-07T06:53:23.148958+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.150145+0000","last_change":"2024-03-07T06:53:23.150145+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.150145+0000","last_undegraded":"2024-03-07T06:53:23.150145+0000","last_fullsized":"2024-03-07T06:53:23.150145+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.141811+0000","last_change":"2024-03-07T06:53:23.141811+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.141811+0000","last_undegraded":"2024-03-07T06:53:23.141811+0000","last_fullsized":"2024-03-07T06:53:23.141811+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.150591+0000","last_change":"2024-03-07T06:53:23.150591+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.150591+0000","last_undegraded":"2024-03-07T06:53:23.150591+0000","last_fullsized":"2024-03-07T06:53:23.150591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.154774+0000","last_change":"2024-03-07T06:53:23.154774+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.154774+0000","last_undegraded":"2024-03-07T06:53:23.154774+0000","last_fullsized":"2024-03-07T06:53:23.154774+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.139418+0000","last_change":"2024-03-07T06:53:23.139418+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.139418+0000","last_undegraded":"2024-03-07T06:53:23.139418+0000","last_fullsized":"2024-03-07T06:53:23.139418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.143170+0000","last_change":"2024-03-07T06:53:23.143170+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.143170+0000","last_undegraded":"2024-03-07T06:53:23.143170+0000","last_fullsized":"2024-03-07T06:53:23.143170+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.151055+0000","last_change":"2024-03-07T06:53:23.151055+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.151055+0000","last_undegraded":"2024-03-07T06:53:23.151055+0000","last_fullsized":"2024-03-07T06:53:23.151055+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.142082+0000","last_change":"2024-03-07T06:53:23.142082+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.142082+0000","last_undegraded":"2024-03-07T06:53:23.142082+0000","last_fullsized":"2024-03-07T06:53:23.142082+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.149633+0000","last_change":"2024-03-07T06:53:23.149633+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.149633+0000","last_undegraded":"2024-03-07T06:53:23.149633+0000","last_fullsized":"2024-03-07T06:53:23.149633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.139542+0000","last_change":"2024-03-07T06:53:23.139542+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.139542+0000","last_undegraded":"2024-03-07T06:53:23.139542+0000","last_fullsized":"2024-03-07T06:53:23.139542+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.141712+0000","last_change":"2024-03-07T06:53:23.141712+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.141712+0000","last_undegraded":"2024-03-07T06:53:23.141712+0000","last_fullsized":"2024-03-07T06:53:23.141712+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.149277+0000","last_change":"2024-03-07T06:53:23.149277+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.149277+0000","last_undegraded":"2024-03-07T06:53:23.149277+0000","last_fullsized":"2024-03-07T06:53:23.149277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.142353+0000","last_change":"2024-03-07T06:53:23.142353+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.142353+0000","last_undegraded":"2024-03-07T06:53:23.142353+0000","last_fullsized":"2024-03-07T06:53:23.142353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.142083+0000","last_change":"2024-03-07T06:53:23.142083+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.142083+0000","last_undegraded":"2024-03-07T06:53:23.142083+0000","last_fullsized":"2024-03-07T06:53:23.142083+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.148919+0000","last_change":"2024-03-07T06:53:23.148919+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.148919+0000","last_undegraded":"2024-03-07T06:53:23.148919+0000","last_fullsized":"2024-03-07T06:53:23.148919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.148664+0000","last_change":"2024-03-07T06:53:23.148664+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.148664+0000","last_undegraded":"2024-03-07T06:53:23.148664+0000","last_fullsized":"2024-03-07T06:53:23.148664+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.151736+0000","last_change":"2024-03-07T06:53:23.151736+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.151736+0000","last_undegraded":"2024-03-07T06:53:23.151736+0000","last_fullsized":"2024-03-07T06:53:23.151736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.142707+0000","last_change":"2024-03-07T06:53:23.142707+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.142707+0000","last_undegraded":"2024-03-07T06:53:23.142707+0000","last_fullsized":"2024-03-07T06:53:23.142707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.153734+0000","last_change":"2024-03-07T06:53:23.153734+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.153734+0000","last_undegraded":"2024-03-07T06:53:23.153734+0000","last_fullsized":"2024-03-07T06:53:23.153734+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.146416+0000","last_change":"2024-03-07T06:53:23.146416+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.146416+0000","last_undegraded":"2024-03-07T06:53:23.146416+0000","last_fullsized":"2024-03-07T06:53:23.146416+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.151557+0000","last_change":"2024-03-07T06:53:23.151557+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.151557+0000","last_undegraded":"2024-03-07T06:53:23.151557+0000","last_fullsized":"2024-03-07T06:53:23.151557+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.141664+0000","last_change":"2024-03-07T06:53:23.141664+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.141664+0000","last_undegraded":"2024-03-07T06:53:23.141664+0000","last_fullsized":"2024-03-07T06:53:23.141664+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.146435+0000","last_change":"2024-03-07T06:53:23.146435+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.146435+0000","last_undegraded":"2024-03-07T06:53:23.146435+0000","last_fullsized":"2024-03-07T06:53:23.146435+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.148648+0000","last_change":"2024-03-07T06:53:23.148648+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.148648+0000","last_undegraded":"2024-03-07T06:53:23.148648+0000","last_fullsized":"2024-03-07T06:53:23.148648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.149824+0000","last_change":"2024-03-07T06:53:23.149824+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.149824+0000","last_undegraded":"2024-03-07T06:53:23.149824+0000","last_fullsized":"2024-03-07T06:53:23.149824+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-07T06:53:23.156773+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:53:23.156773+0000","last_peered":"2024-03-07T06:53:23.156773+0000","last_clean":"2024-03-07T06:53:23.156773+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:53:23.156773+0000","last_undegraded":"2024-03-07T06:53:23.156773+0000","last_fullsized":"2024-03-07T06:53:23.156773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":790820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27508,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":491520,"data_stored":194635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":503808,"data_stored":200452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":491520,"data_stored":194635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31668,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":491520,"data_stored":194635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":194635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":503808,"data_stored":200452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1085440,"data_stored":785003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:53:29.907 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:53:30.022 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:53:30.029 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:53:30.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.173+0000 7f7b16000640 1 Processor -- start 2024-03-07T06:53:30.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.173+0000 7f7b16000640 1 -- start start 2024-03-07T06:53:30.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.174+0000 7f7b16000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b1015aaf0 0x7f7b1006e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.174+0000 7f7b16000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7b100cd4c0 0x7f7b1015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.174+0000 7f7b16000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7b10072830 0x7f7b10072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.174+0000 7f7b16000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b100731e0 con 0x7f7b10072830 2024-03-07T06:53:30.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.174+0000 7f7b16000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1006e850 con 0x7f7b100cd4c0 2024-03-07T06:53:30.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.174+0000 7f7b16000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1006eb30 con 0x7f7b1015aaf0 2024-03-07T06:53:30.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.175+0000 7f7b15600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7b10072830 0x7f7b10072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:30.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.175+0000 7f7b15600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7b10072830 0x7f7b10072ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54676/0 (socket says 172.21.15.133:54676) 2024-03-07T06:53:30.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.175+0000 7f7b15600640 1 -- 172.21.15.133:0/1147248367 learned_addr learned my addr 172.21.15.133:0/1147248367 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:30.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.175+0000 7f7b0fe00640 1 --2- 172.21.15.133:0/1147248367 >> v2:172.21.15.161:3300/0 conn(0x7f7b100cd4c0 0x7f7b1015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:30.175 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.175+0000 7f7b14c00640 1 --2- 172.21.15.133:0/1147248367 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b1015aaf0 0x7f7b1006e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:30.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.175+0000 7f7b15600640 1 -- 172.21.15.133:0/1147248367 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b1015aaf0 msgr2=0x7f7b1006e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:30.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.175+0000 7f7b15600640 1 --2- 172.21.15.133:0/1147248367 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b1015aaf0 0x7f7b1006e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.175+0000 7f7b15600640 1 -- 172.21.15.133:0/1147248367 >> v2:172.21.15.161:3300/0 conn(0x7f7b100cd4c0 msgr2=0x7f7b1015aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:30.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.175+0000 7f7b15600640 1 --2- 172.21.15.133:0/1147248367 >> v2:172.21.15.161:3300/0 conn(0x7f7b100cd4c0 0x7f7b1015aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.175+0000 7f7b15600640 1 -- 172.21.15.133:0/1147248367 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1006ee10 con 0x7f7b10072830 2024-03-07T06:53:30.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.177+0000 7f7b15600640 1 --2- 172.21.15.133:0/1147248367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b10072830 0x7f7b10072ca0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7b10069140 tx=0x7f7b0802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba073ea77b4dc0d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.177+0000 7f7b0f400640 1 -- 172.21.15.133:0/1147248367 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0802ca70 con 0x7f7b10072830 2024-03-07T06:53:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.178+0000 7f7b0f400640 1 -- 172.21.15.133:0/1147248367 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0802cc10 con 0x7f7b10072830 2024-03-07T06:53:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.178+0000 7f7b0f400640 1 -- 172.21.15.133:0/1147248367 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b080343f0 con 0x7f7b10072830 2024-03-07T06:53:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.178+0000 7f7b16000640 1 -- 172.21.15.133:0/1147248367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b10072830 msgr2=0x7f7b10072ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.178+0000 7f7b16000640 1 --2- 172.21.15.133:0/1147248367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b10072830 0x7f7b10072ca0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7b10069140 tx=0x7f7b0802bed0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.178+0000 7f7b16000640 1 -- 172.21.15.133:0/1147248367 shutdown_connections 2024-03-07T06:53:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.178+0000 7f7b16000640 1 --2- 172.21.15.133:0/1147248367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b10072830 0x7f7b10072ca0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.178+0000 7f7b16000640 1 --2- 172.21.15.133:0/1147248367 >> v2:172.21.15.161:3300/0 conn(0x7f7b100cd4c0 0x7f7b1015aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.178+0000 7f7b16000640 1 --2- 172.21.15.133:0/1147248367 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b1015aaf0 0x7f7b1006e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.178+0000 7f7b16000640 1 -- 172.21.15.133:0/1147248367 >> 172.21.15.133:0/1147248367 conn(0x7f7b100b2e90 msgr2=0x7f7b100b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:30.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.178+0000 7f7b16000640 1 -- 172.21.15.133:0/1147248367 shutdown_connections 2024-03-07T06:53:30.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.178+0000 7f7b16000640 1 -- 172.21.15.133:0/1147248367 wait complete. 2024-03-07T06:53:30.179 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.180+0000 7f7b16000640 1 Processor -- start 2024-03-07T06:53:30.180 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.180+0000 7f7b16000640 1 -- start start 2024-03-07T06:53:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.181+0000 7f7b16000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b10072830 0x7f7b100c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.181+0000 7f7b16000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b100cd4c0 0x7f7b101f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.181+0000 7f7b16000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b1015aaf0 0x7f7b100cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.181+0000 7f7b16000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1006f0f0 con 0x7f7b100cd4c0 2024-03-07T06:53:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.181+0000 7f7b16000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1006e560 con 0x7f7b10072830 2024-03-07T06:53:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.181+0000 7f7b16000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1015b410 con 0x7f7b1015aaf0 2024-03-07T06:53:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.181+0000 7f7b0fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b100cd4c0 0x7f7b101f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.181+0000 7f7b0fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b100cd4c0 0x7f7b101f72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54680/0 (socket says 172.21.15.133:54680) 2024-03-07T06:53:30.181 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.181+0000 7f7b0fe00640 1 -- 172.21.15.133:0/3689258287 learned_addr learned my addr 172.21.15.133:0/3689258287 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:30.182 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.182+0000 7f7b0fe00640 1 -- 172.21.15.133:0/3689258287 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b1015aaf0 msgr2=0x7f7b100cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.182+0000 7f7b0fe00640 1 --2- 172.21.15.133:0/3689258287 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b1015aaf0 0x7f7b100cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.182+0000 7f7b0fe00640 1 -- 172.21.15.133:0/3689258287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b10072830 msgr2=0x7f7b100c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.182+0000 7f7b0fe00640 1 --2- 172.21.15.133:0/3689258287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b10072830 0x7f7b100c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.182+0000 7f7b0fe00640 1 -- 172.21.15.133:0/3689258287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b101f7d70 con 0x7f7b100cd4c0 2024-03-07T06:53:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.182+0000 7f7b0fe00640 1 --2- 172.21.15.133:0/3689258287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b100cd4c0 0x7f7b101f72b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00d5f0 tx=0x7f7afc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.182+0000 7f7b0cc00640 1 -- 172.21.15.133:0/3689258287 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0125f0 con 0x7f7b100cd4c0 2024-03-07T06:53:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.182+0000 7f7b0cc00640 1 -- 172.21.15.133:0/3689258287 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc019070 con 0x7f7b100cd4c0 2024-03-07T06:53:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.182+0000 7f7b0cc00640 1 -- 172.21.15.133:0/3689258287 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0129d0 con 0x7f7b100cd4c0 2024-03-07T06:53:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.182+0000 7f7b16000640 1 -- 172.21.15.133:0/3689258287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b101f8050 con 0x7f7b100cd4c0 2024-03-07T06:53:30.183 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.183+0000 7f7b16000640 1 -- 172.21.15.133:0/3689258287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1014ef30 con 0x7f7b100cd4c0 2024-03-07T06:53:30.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.184+0000 7f7b16000640 1 -- 172.21.15.133:0/3689258287 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b10062c90 con 0x7f7b100cd4c0 2024-03-07T06:53:30.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.185+0000 7f7b0cc00640 1 -- 172.21.15.133:0/3689258287 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7afc012b70 con 0x7f7b100cd4c0 2024-03-07T06:53:30.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.185+0000 7f7b0cc00640 1 --2- 172.21.15.133:0/3689258287 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7af006b4d0 0x7f7af006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.186+0000 7f7b0cc00640 1 -- 172.21.15.133:0/3689258287 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7afc08a620 con 0x7f7b100cd4c0 2024-03-07T06:53:30.185 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.186+0000 7f7b14c00640 1 --2- 172.21.15.133:0/3689258287 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7af006b4d0 0x7f7af006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:30.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.187+0000 7f7b14c00640 1 --2- 172.21.15.133:0/3689258287 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7af006b4d0 0x7f7af006d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400b7f0 tx=0x7f7b04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:30.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.191+0000 7f7b0cc00640 1 -- 172.21.15.133:0/3689258287 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7afc059490 con 0x7f7b100cd4c0 2024-03-07T06:53:30.398 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.399+0000 7f7b16000640 1 -- 172.21.15.133:0/3689258287 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b1015c0d0 con 0x7f7af006b4d0 2024-03-07T06:53:30.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.416+0000 7f7b0cc00640 1 -- 172.21.15.133:0/3689258287 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239859 (secure 0 0 0) 0x7f7b1015c0d0 con 0x7f7af006b4d0 2024-03-07T06:53:30.419 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:30.420 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:53:30.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.420+0000 7f7b16000640 1 -- 172.21.15.133:0/3689258287 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7af006b4d0 msgr2=0x7f7af006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:30.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.421+0000 7f7b16000640 1 --2- 172.21.15.133:0/3689258287 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7af006b4d0 0x7f7af006d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7b0400b7f0 tx=0x7f7b04008040 comp rx=0 tx=0).stop 2024-03-07T06:53:30.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.421+0000 7f7b16000640 1 -- 172.21.15.133:0/3689258287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b100cd4c0 msgr2=0x7f7b101f72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:30.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.421+0000 7f7b16000640 1 --2- 172.21.15.133:0/3689258287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b100cd4c0 0x7f7b101f72b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00d5f0 tx=0x7f7afc00dab0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.421+0000 7f7b16000640 1 -- 172.21.15.133:0/3689258287 shutdown_connections 2024-03-07T06:53:30.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.421+0000 7f7b16000640 1 --2- 172.21.15.133:0/3689258287 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b1015aaf0 0x7f7b100cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.421+0000 7f7b16000640 1 --2- 172.21.15.133:0/3689258287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b100cd4c0 0x7f7b101f72b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.421+0000 7f7b16000640 1 --2- 172.21.15.133:0/3689258287 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7af006b4d0 0x7f7af006d990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.421+0000 7f7b16000640 1 --2- 172.21.15.133:0/3689258287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b10072830 0x7f7b100c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.421+0000 7f7b16000640 1 -- 172.21.15.133:0/3689258287 >> 172.21.15.133:0/3689258287 conn(0x7f7b100b2e90 msgr2=0x7f7b1015d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:30.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.421+0000 7f7b16000640 1 -- 172.21.15.133:0/3689258287 shutdown_connections 2024-03-07T06:53:30.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.421+0000 7f7b16000640 1 -- 172.21.15.133:0/3689258287 wait complete. 2024-03-07T06:53:30.441 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-03-07T06:53:28.518027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":246196,"kb_used_data":5036,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754728524,"statfs":{"total":773094113280,"available":772842008576,"internally_reserved":0,"allocated":5156864,"data_stored":2755267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.381368"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.144907+0000","last_change":"2024-03-07T06:53:23.144907+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.144907+0000","last_undegraded":"2024-03-07T06:53:23.144907+0000","last_fullsized":"2024-03-07T06:53:23.144907+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.143029+0000","last_change":"2024-03-07T06:53:23.143029+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.143029+0000","last_undegraded":"2024-03-07T06:53:23.143029+0000","last_fullsized":"2024-03-07T06:53:23.143029+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.146433+0000","last_change":"2024-03-07T06:53:23.146433+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.146433+0000","last_undegraded":"2024-03-07T06:53:23.146433+0000","last_fullsized":"2024-03-07T06:53:23.146433+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.153163+0000","last_change":"2024-03-07T06:53:23.153163+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.153163+0000","last_undegraded":"2024-03-07T06:53:23.153163+0000","last_fullsized":"2024-03-07T06:53:23.153163+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.144751+0000","last_change":"2024-03-07T06:53:23.144751+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.144751+0000","last_undegraded":"2024-03-07T06:53:23.144751+0000","last_fullsized":"2024-03-07T06:53:23.144751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.142881+0000","last_change":"2024-03-07T06:53:23.142881+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.142881+0000","last_undegraded":"2024-03-07T06:53:23.142881+0000","last_fullsized":"2024-03-07T06:53:23.142881+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.148958+0000","last_change":"2024-03-07T06:53:23.148958+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.148958+0000","last_undegraded":"2024-03-07T06:53:23.148958+0000","last_fullsized":"2024-03-07T06:53:23.148958+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.150145+0000","last_change":"2024-03-07T06:53:23.150145+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.150145+0000","last_undegraded":"2024-03-07T06:53:23.150145+0000","last_fullsized":"2024-03-07T06:53:23.150145+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.141811+0000","last_change":"2024-03-07T06:53:23.141811+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.141811+0000","last_undegraded":"2024-03-07T06:53:23.141811+0000","last_fullsized":"2024-03-07T06:53:23.141811+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.150591+0000","last_change":"2024-03-07T06:53:23.150591+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.150591+0000","last_undegraded":"2024-03-07T06:53:23.150591+0000","last_fullsized":"2024-03-07T06:53:23.150591+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.154774+0000","last_change":"2024-03-07T06:53:23.154774+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.154774+0000","last_undegraded":"2024-03-07T06:53:23.154774+0000","last_fullsized":"2024-03-07T06:53:23.154774+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.139418+0000","last_change":"2024-03-07T06:53:23.139418+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.139418+0000","last_undegraded":"2024-03-07T06:53:23.139418+0000","last_fullsized":"2024-03-07T06:53:23.139418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.143170+0000","last_change":"2024-03-07T06:53:23.143170+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.143170+0000","last_undegraded":"2024-03-07T06:53:23.143170+0000","last_fullsized":"2024-03-07T06:53:23.143170+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.151055+0000","last_change":"2024-03-07T06:53:23.151055+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.151055+0000","last_undegraded":"2024-03-07T06:53:23.151055+0000","last_fullsized":"2024-03-07T06:53:23.151055+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.142082+0000","last_change":"2024-03-07T06:53:23.142082+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.142082+0000","last_undegraded":"2024-03-07T06:53:23.142082+0000","last_fullsized":"2024-03-07T06:53:23.142082+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.149633+0000","last_change":"2024-03-07T06:53:23.149633+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.149633+0000","last_undegraded":"2024-03-07T06:53:23.149633+0000","last_fullsized":"2024-03-07T06:53:23.149633+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.139542+0000","last_change":"2024-03-07T06:53:23.139542+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.139542+0000","last_undegraded":"2024-03-07T06:53:23.139542+0000","last_fullsized":"2024-03-07T06:53:23.139542+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.141712+0000","last_change":"2024-03-07T06:53:23.141712+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.141712+0000","last_undegraded":"2024-03-07T06:53:23.141712+0000","last_fullsized":"2024-03-07T06:53:23.141712+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.149277+0000","last_change":"2024-03-07T06:53:23.149277+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.149277+0000","last_undegraded":"2024-03-07T06:53:23.149277+0000","last_fullsized":"2024-03-07T06:53:23.149277+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.142353+0000","last_change":"2024-03-07T06:53:23.142353+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.142353+0000","last_undegraded":"2024-03-07T06:53:23.142353+0000","last_fullsized":"2024-03-07T06:53:23.142353+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.142083+0000","last_change":"2024-03-07T06:53:23.142083+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.142083+0000","last_undegraded":"2024-03-07T06:53:23.142083+0000","last_fullsized":"2024-03-07T06:53:23.142083+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.148919+0000","last_change":"2024-03-07T06:53:23.148919+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.148919+0000","last_undegraded":"2024-03-07T06:53:23.148919+0000","last_fullsized":"2024-03-07T06:53:23.148919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.148664+0000","last_change":"2024-03-07T06:53:23.148664+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.148664+0000","last_undegraded":"2024-03-07T06:53:23.148664+0000","last_fullsized":"2024-03-07T06:53:23.148664+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.151736+0000","last_change":"2024-03-07T06:53:23.151736+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.151736+0000","last_undegraded":"2024-03-07T06:53:23.151736+0000","last_fullsized":"2024-03-07T06:53:23.151736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.142707+0000","last_change":"2024-03-07T06:53:23.142707+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.142707+0000","last_undegraded":"2024-03-07T06:53:23.142707+0000","last_fullsized":"2024-03-07T06:53:23.142707+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.153734+0000","last_change":"2024-03-07T06:53:23.153734+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.153734+0000","last_undegraded":"2024-03-07T06:53:23.153734+0000","last_fullsized":"2024-03-07T06:53:23.153734+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.146416+0000","last_change":"2024-03-07T06:53:23.146416+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.146416+0000","last_undegraded":"2024-03-07T06:53:23.146416+0000","last_fullsized":"2024-03-07T06:53:23.146416+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.151557+0000","last_change":"2024-03-07T06:53:23.151557+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.151557+0000","last_undegraded":"2024-03-07T06:53:23.151557+0000","last_fullsized":"2024-03-07T06:53:23.151557+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.141664+0000","last_change":"2024-03-07T06:53:23.141664+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.141664+0000","last_undegraded":"2024-03-07T06:53:23.141664+0000","last_fullsized":"2024-03-07T06:53:23.141664+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.146435+0000","last_change":"2024-03-07T06:53:23.146435+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.146435+0000","last_undegraded":"2024-03-07T06:53:23.146435+0000","last_fullsized":"2024-03-07T06:53:23.146435+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.148648+0000","last_change":"2024-03-07T06:53:23.148648+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.148648+0000","last_undegraded":"2024-03-07T06:53:23.148648+0000","last_fullsized":"2024-03-07T06:53:23.148648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-07T06:53:24.135186+0000","last_change":"2024-03-07T06:53:24.135186+0000","last_active":"2024-03-07T06:53:24.135186+0000","last_peered":"2024-03-07T06:53:24.135186+0000","last_clean":"2024-03-07T06:53:24.135186+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:24.135186+0000","last_undegraded":"2024-03-07T06:53:24.135186+0000","last_fullsized":"2024-03-07T06:53:24.135186+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-03-07T06:53:23.149824+0000","last_change":"2024-03-07T06:53:23.149824+0000","last_active":"2024-03-07T06:53:23.120207+0000","last_peered":"2024-03-07T06:53:23.120207+0000","last_clean":"2024-03-07T06:53:23.120207+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-07T06:53:23.149824+0000","last_undegraded":"2024-03-07T06:53:23.149824+0000","last_fullsized":"2024-03-07T06:53:23.149824+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-03-07T06:53:23.156773+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:53:23.156773+0000","last_peered":"2024-03-07T06:53:23.156773+0000","last_clean":"2024-03-07T06:53:23.156773+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:53:23.156773+0000","last_undegraded":"2024-03-07T06:53:23.156773+0000","last_fullsized":"2024-03-07T06:53:23.156773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":790820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27508,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":491520,"data_stored":194635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":503808,"data_stored":200452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":491520,"data_stored":194635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31668,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":491520,"data_stored":194635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27504,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":491520,"data_stored":194635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":503808,"data_stored":200452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1085440,"data_stored":785003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:53:30.445 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:53:30.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.682+0000 7f45e6000640 1 Processor -- start 2024-03-07T06:53:30.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.682+0000 7f45e6000640 1 -- start start 2024-03-07T06:53:30.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.684+0000 7f45e6000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45e0071d20 0x7f45e00724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.684+0000 7f45e6000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e00cdf80 con 0x7f45e0073050 2024-03-07T06:53:30.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.684+0000 7f45e6000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e015e770 con 0x7f45e015e390 2024-03-07T06:53:30.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.684+0000 7f45e6000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00ce4d0 con 0x7f45e0071d20 2024-03-07T06:53:30.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.684+0000 7f45dfe00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f45e0073050 0x7f45e006d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:55188/0 (socket says 172.21.15.133:55188) 2024-03-07T06:53:30.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.684+0000 7f45dfe00640 1 -- 172.21.15.133:0/874989866 learned_addr learned my addr 172.21.15.133:0/874989866 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:30.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.684+0000 7f45d7e00640 1 --2- 172.21.15.133:0/874989866 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45e0071d20 0x7f45e00724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:30.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 -- 172.21.15.133:0/874989866 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098333585 0 0) 0x7f45e00cdf80 con 0x7f45e0073050 2024-03-07T06:53:30.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 -- 172.21.15.133:0/874989866 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45c4003520 con 0x7f45e0073050 2024-03-07T06:53:30.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 -- 172.21.15.133:0/874989866 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1404386845 0 0) 0x7f45c4003520 con 0x7f45e0073050 2024-03-07T06:53:30.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 -- 172.21.15.133:0/874989866 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45e0071d20 msgr2=0x7f45e00724b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:30.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 --2- 172.21.15.133:0/874989866 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45e0071d20 0x7f45e00724b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 -- 172.21.15.133:0/874989866 >> v1:172.21.15.161:6789/0 conn(0x7f45e015e390 legacy=0x7f45e0071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:30.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 -- 172.21.15.133:0/874989866 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e00b3d90 con 0x7f45e0073050 2024-03-07T06:53:30.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 -- 172.21.15.133:0/874989866 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f45d80019f0 con 0x7f45e0073050 2024-03-07T06:53:30.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 -- 172.21.15.133:0/874989866 >> v1:172.21.15.133:6789/0 conn(0x7f45e0073050 legacy=0x7f45e006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:30.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 --2- 172.21.15.133:0/874989866 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45e0071d20 0x7f45c40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 --2- 172.21.15.133:0/874989866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c4003f60 0x7f45c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 --2- 172.21.15.133:0/874989866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45c4004d40 0x7f45c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 -- 172.21.15.133:0/874989866 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00ce4d0 con 0x7f45c4003f60 2024-03-07T06:53:30.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 -- 172.21.15.133:0/874989866 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c400baa0 con 0x7f45c4004d40 2024-03-07T06:53:30.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.685+0000 7f45dea00640 1 -- 172.21.15.133:0/874989866 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c400bca0 con 0x7f45e0071d20 2024-03-07T06:53:30.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.686+0000 7f45d7e00640 1 --2- 172.21.15.133:0/874989866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c4003f60 0x7f45c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:30.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.686+0000 7f45dfe00640 1 --2- 172.21.15.133:0/874989866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45c4004d40 0x7f45c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:30.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.686+0000 7f45df400640 1 --2- 172.21.15.133:0/874989866 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45e0071d20 0x7f45c40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:30.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.686+0000 7f45d7e00640 1 -- 172.21.15.133:0/874989866 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45e0071d20 msgr2=0x7f45c40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:30.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.686+0000 7f45d7e00640 1 --2- 172.21.15.133:0/874989866 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45e0071d20 0x7f45c40039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.686+0000 7f45d7e00640 1 -- 172.21.15.133:0/874989866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45c4004d40 msgr2=0x7f45c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:30.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.686+0000 7f45d7e00640 1 --2- 172.21.15.133:0/874989866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45c4004d40 0x7f45c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.686+0000 7f45d7e00640 1 -- 172.21.15.133:0/874989866 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c4003520 con 0x7f45c4003f60 2024-03-07T06:53:30.687 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.686+0000 7f45dfe00640 1 --2- 172.21.15.133:0/874989866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45c4004d40 0x7f45c40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:30.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.687+0000 7f45d7e00640 1 --2- 172.21.15.133:0/874989866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c4003f60 0x7f45c4004800 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009ac0 tx=0x7f45c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1c536f3ce4fdf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:30.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.687+0000 7f45dea00640 1 -- 172.21.15.133:0/874989866 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8039070 con 0x7f45c4003f60 2024-03-07T06:53:30.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.687+0000 7f45dea00640 1 -- 172.21.15.133:0/874989866 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c802c6b0 con 0x7f45c4003f60 2024-03-07T06:53:30.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.687+0000 7f45dea00640 1 -- 172.21.15.133:0/874989866 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8009690 con 0x7f45c4003f60 2024-03-07T06:53:30.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.688+0000 7f45e6000640 1 -- 172.21.15.133:0/874989866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c4003f60 msgr2=0x7f45c4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:30.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.688+0000 7f45e6000640 1 --2- 172.21.15.133:0/874989866 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c4003f60 0x7f45c4004800 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009ac0 tx=0x7f45c4005830 comp rx=0 tx=0).stop 2024-03-07T06:53:30.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.688+0000 7f45dfe00640 1 -- 172.21.15.133:0/874989866 reap_dead start 2024-03-07T06:53:30.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.688+0000 7f45e6000640 1 -- 172.21.15.133:0/874989866 shutdown_connections 2024-03-07T06:53:30.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.688+0000 7f45e6000640 1 -- 172.21.15.133:0/874989866 >> 172.21.15.133:0/874989866 conn(0x7f45e00b3120 msgr2=0x7f45e00b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:30.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.688+0000 7f45e6000640 1 -- 172.21.15.133:0/874989866 shutdown_connections 2024-03-07T06:53:30.689 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.688+0000 7f45e6000640 1 -- 172.21.15.133:0/874989866 wait complete. 2024-03-07T06:53:30.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.690+0000 7f45e6000640 1 Processor -- start 2024-03-07T06:53:30.690 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.690+0000 7f45e6000640 1 -- start start 2024-03-07T06:53:30.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.691+0000 7f45e6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c4003f60 0x7f45e0151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.691+0000 7f45e6000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45e009cbf0 0x7f45e00c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.691+0000 7f45e6000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45e009d070 0x7f45e00c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.691+0000 7f45e6000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00724b0 con 0x7f45c4003f60 2024-03-07T06:53:30.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.691+0000 7f45e6000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00b4070 con 0x7f45e009cbf0 2024-03-07T06:53:30.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.691+0000 7f45e6000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e0072630 con 0x7f45e009d070 2024-03-07T06:53:30.691 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.691+0000 7f45df400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c4003f60 0x7f45e0151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:30.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.691+0000 7f45df400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c4003f60 0x7f45e0151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54698/0 (socket says 172.21.15.133:54698) 2024-03-07T06:53:30.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.691+0000 7f45df400640 1 -- 172.21.15.133:0/1752069744 learned_addr learned my addr 172.21.15.133:0/1752069744 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:30.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.692+0000 7f45dfe00640 1 --2- 172.21.15.133:0/1752069744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45e009d070 0x7f45e00c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:30.692 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.692+0000 7f45d7e00640 1 --2- 172.21.15.133:0/1752069744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45e009cbf0 0x7f45e00c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:30.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.692+0000 7f45df400640 1 -- 172.21.15.133:0/1752069744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45e009d070 msgr2=0x7f45e00c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:30.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.692+0000 7f45df400640 1 --2- 172.21.15.133:0/1752069744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45e009d070 0x7f45e00c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.692+0000 7f45df400640 1 -- 172.21.15.133:0/1752069744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45e009cbf0 msgr2=0x7f45e00c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:30.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.692+0000 7f45df400640 1 --2- 172.21.15.133:0/1752069744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45e009cbf0 0x7f45e00c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.692+0000 7f45df400640 1 -- 172.21.15.133:0/1752069744 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e00cc690 con 0x7f45c4003f60 2024-03-07T06:53:30.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.692+0000 7f45df400640 1 --2- 172.21.15.133:0/1752069744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c4003f60 0x7f45e0151ad0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f45d000d570 tx=0x7f45d000da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:30.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.693+0000 7f45d7400640 1 -- 172.21.15.133:0/1752069744 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0005480 con 0x7f45c4003f60 2024-03-07T06:53:30.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.693+0000 7f45d7400640 1 -- 172.21.15.133:0/1752069744 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d0017070 con 0x7f45c4003f60 2024-03-07T06:53:30.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.693+0000 7f45e6000640 1 -- 172.21.15.133:0/1752069744 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e00cc970 con 0x7f45c4003f60 2024-03-07T06:53:30.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.693+0000 7f45d7400640 1 -- 172.21.15.133:0/1752069744 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0003810 con 0x7f45c4003f60 2024-03-07T06:53:30.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.693+0000 7f45e6000640 1 -- 172.21.15.133:0/1752069744 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e00ccf30 con 0x7f45c4003f60 2024-03-07T06:53:30.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.695+0000 7f45e6000640 1 -- 172.21.15.133:0/1752069744 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e0062c90 con 0x7f45c4003f60 2024-03-07T06:53:30.694 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.695+0000 7f45d7400640 1 -- 172.21.15.133:0/1752069744 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f45d00039b0 con 0x7f45c4003f60 2024-03-07T06:53:30.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.696+0000 7f45d7400640 1 --2- 172.21.15.133:0/1752069744 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f45bc06b4d0 0x7f45bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:30.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.697+0000 7f45d7400640 1 -- 172.21.15.133:0/1752069744 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45d008a0c0 con 0x7f45c4003f60 2024-03-07T06:53:30.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.697+0000 7f45d7e00640 1 --2- 172.21.15.133:0/1752069744 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f45bc06b4d0 0x7f45bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:30.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.698+0000 7f45d7e00640 1 --2- 172.21.15.133:0/1752069744 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f45bc06b4d0 0x7f45bc06d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f45c8036000 tx=0x7f45c8030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:30.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.701+0000 7f45d7400640 1 -- 172.21.15.133:0/1752069744 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f45d0058ea0 con 0x7f45c4003f60 2024-03-07T06:53:30.947 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.948+0000 7f45e6000640 1 -- 172.21.15.133:0/1752069744 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f45e00b0470 con 0x7f45bc06b4d0 2024-03-07T06:53:30.966 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.967+0000 7f45d7400640 1 -- 172.21.15.133:0/1752069744 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251960 (secure 0 0 0) 0x7f45c8082850 con 0x7f45bc06b4d0 2024-03-07T06:53:30.969 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:30.971 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:53:30.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.972+0000 7f45e6000640 1 -- 172.21.15.133:0/1752069744 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f45bc06b4d0 msgr2=0x7f45bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:30.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.972+0000 7f45e6000640 1 --2- 172.21.15.133:0/1752069744 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f45bc06b4d0 0x7f45bc06d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f45c8036000 tx=0x7f45c8030000 comp rx=0 tx=0).stop 2024-03-07T06:53:30.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.972+0000 7f45e6000640 1 -- 172.21.15.133:0/1752069744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c4003f60 msgr2=0x7f45e0151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:30.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.972+0000 7f45e6000640 1 --2- 172.21.15.133:0/1752069744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c4003f60 0x7f45e0151ad0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f45d000d570 tx=0x7f45d000da30 comp rx=0 tx=0).stop 2024-03-07T06:53:30.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.972+0000 7f45e6000640 1 -- 172.21.15.133:0/1752069744 shutdown_connections 2024-03-07T06:53:30.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.972+0000 7f45e6000640 1 --2- 172.21.15.133:0/1752069744 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f45e009d070 0x7f45e00c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.972+0000 7f45e6000640 1 --2- 172.21.15.133:0/1752069744 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f45bc06b4d0 0x7f45bc06d990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.972+0000 7f45e6000640 1 --2- 172.21.15.133:0/1752069744 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f45e009cbf0 0x7f45e00c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.972+0000 7f45e6000640 1 --2- 172.21.15.133:0/1752069744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c4003f60 0x7f45e0151ad0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:30.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.972+0000 7f45e6000640 1 -- 172.21.15.133:0/1752069744 >> 172.21.15.133:0/1752069744 conn(0x7f45e00b3120 msgr2=0x7f45e00ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:30.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.972+0000 7f45e6000640 1 -- 172.21.15.133:0/1752069744 shutdown_connections 2024-03-07T06:53:30.973 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:30.972+0000 7f45e6000640 1 -- 172.21.15.133:0/1752069744 wait complete. 2024-03-07T06:53:30.990 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-03-07T06:53:30.518666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250760,"kb_used_data":5448,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754723960,"statfs":{"total":773094113280,"available":772837335040,"internally_reserved":0,"allocated":5578752,"data_stored":2943908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.363674"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194303+0000","last_change":"2024-03-07T06:53:25.202958+0000","last_active":"2024-03-07T06:53:26.194303+0000","last_peered":"2024-03-07T06:53:26.194303+0000","last_clean":"2024-03-07T06:53:26.194303+0000","last_became_active":"2024-03-07T06:53:25.202689+0000","last_became_peered":"2024-03-07T06:53:25.202689+0000","last_unstale":"2024-03-07T06:53:26.194303+0000","last_undegraded":"2024-03-07T06:53:26.194303+0000","last_fullsized":"2024-03-07T06:53:26.194303+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:51:54.139595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192936+0000","last_change":"2024-03-07T06:53:25.205060+0000","last_active":"2024-03-07T06:53:26.192936+0000","last_peered":"2024-03-07T06:53:26.192936+0000","last_clean":"2024-03-07T06:53:26.192936+0000","last_became_active":"2024-03-07T06:53:25.204801+0000","last_became_peered":"2024-03-07T06:53:25.204801+0000","last_unstale":"2024-03-07T06:53:26.192936+0000","last_undegraded":"2024-03-07T06:53:26.192936+0000","last_fullsized":"2024-03-07T06:53:26.192936+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:14.520156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796955+0000","last_change":"2024-03-07T06:53:25.189468+0000","last_active":"2024-03-07T06:53:26.796955+0000","last_peered":"2024-03-07T06:53:26.796955+0000","last_clean":"2024-03-07T06:53:26.796955+0000","last_became_active":"2024-03-07T06:53:25.189178+0000","last_became_peered":"2024-03-07T06:53:25.189178+0000","last_unstale":"2024-03-07T06:53:26.796955+0000","last_undegraded":"2024-03-07T06:53:26.796955+0000","last_fullsized":"2024-03-07T06:53:26.796955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:04.729008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556013+0000","last_change":"2024-03-07T06:53:25.193381+0000","last_active":"2024-03-07T06:53:26.556013+0000","last_peered":"2024-03-07T06:53:26.556013+0000","last_clean":"2024-03-07T06:53:26.556013+0000","last_became_active":"2024-03-07T06:53:25.193079+0000","last_became_peered":"2024-03-07T06:53:25.193079+0000","last_unstale":"2024-03-07T06:53:26.556013+0000","last_undegraded":"2024-03-07T06:53:26.556013+0000","last_fullsized":"2024-03-07T06:53:26.556013+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:14.520156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193500+0000","last_change":"2024-03-07T06:53:25.202836+0000","last_active":"2024-03-07T06:53:26.193500+0000","last_peered":"2024-03-07T06:53:26.193500+0000","last_clean":"2024-03-07T06:53:26.193500+0000","last_became_active":"2024-03-07T06:53:25.202463+0000","last_became_peered":"2024-03-07T06:53:25.202463+0000","last_unstale":"2024-03-07T06:53:26.193500+0000","last_undegraded":"2024-03-07T06:53:26.193500+0000","last_fullsized":"2024-03-07T06:53:26.193500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:44.090672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568638+0000","last_change":"2024-03-07T06:53:25.203678+0000","last_active":"2024-03-07T06:53:26.568638+0000","last_peered":"2024-03-07T06:53:26.568638+0000","last_clean":"2024-03-07T06:53:26.568638+0000","last_became_active":"2024-03-07T06:53:25.198295+0000","last_became_peered":"2024-03-07T06:53:25.198295+0000","last_unstale":"2024-03-07T06:53:26.568638+0000","last_undegraded":"2024-03-07T06:53:26.568638+0000","last_fullsized":"2024-03-07T06:53:26.568638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:49:12.380843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.190260+0000","last_change":"2024-03-07T06:53:25.193112+0000","last_active":"2024-03-07T06:53:26.190260+0000","last_peered":"2024-03-07T06:53:26.190260+0000","last_clean":"2024-03-07T06:53:26.190260+0000","last_became_active":"2024-03-07T06:53:25.192861+0000","last_became_peered":"2024-03-07T06:53:25.192861+0000","last_unstale":"2024-03-07T06:53:26.190260+0000","last_undegraded":"2024-03-07T06:53:26.190260+0000","last_fullsized":"2024-03-07T06:53:26.190260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:08:50.099907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555336+0000","last_change":"2024-03-07T06:53:25.200927+0000","last_active":"2024-03-07T06:53:26.555336+0000","last_peered":"2024-03-07T06:53:26.555336+0000","last_clean":"2024-03-07T06:53:26.555336+0000","last_became_active":"2024-03-07T06:53:25.200535+0000","last_became_peered":"2024-03-07T06:53:25.200535+0000","last_unstale":"2024-03-07T06:53:26.555336+0000","last_undegraded":"2024-03-07T06:53:26.555336+0000","last_fullsized":"2024-03-07T06:53:26.555336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:55.274941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556694+0000","last_change":"2024-03-07T06:53:25.179342+0000","last_active":"2024-03-07T06:53:26.556694+0000","last_peered":"2024-03-07T06:53:26.556694+0000","last_clean":"2024-03-07T06:53:26.556694+0000","last_became_active":"2024-03-07T06:53:25.179031+0000","last_became_peered":"2024-03-07T06:53:25.179031+0000","last_unstale":"2024-03-07T06:53:26.556694+0000","last_undegraded":"2024-03-07T06:53:26.556694+0000","last_fullsized":"2024-03-07T06:53:26.556694+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:54:32.067212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556561+0000","last_change":"2024-03-07T06:53:25.199475+0000","last_active":"2024-03-07T06:53:26.556561+0000","last_peered":"2024-03-07T06:53:26.556561+0000","last_clean":"2024-03-07T06:53:26.556561+0000","last_became_active":"2024-03-07T06:53:25.193080+0000","last_became_peered":"2024-03-07T06:53:25.193080+0000","last_unstale":"2024-03-07T06:53:26.556561+0000","last_undegraded":"2024-03-07T06:53:26.556561+0000","last_fullsized":"2024-03-07T06:53:26.556561+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:08:50.099907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555759+0000","last_change":"2024-03-07T06:53:25.177187+0000","last_active":"2024-03-07T06:53:26.555759+0000","last_peered":"2024-03-07T06:53:26.555759+0000","last_clean":"2024-03-07T06:53:26.555759+0000","last_became_active":"2024-03-07T06:53:25.176924+0000","last_became_peered":"2024-03-07T06:53:25.176924+0000","last_unstale":"2024-03-07T06:53:26.555759+0000","last_undegraded":"2024-03-07T06:53:26.555759+0000","last_fullsized":"2024-03-07T06:53:26.555759+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:04.729008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193384+0000","last_change":"2024-03-07T06:53:25.197797+0000","last_active":"2024-03-07T06:53:26.193384+0000","last_peered":"2024-03-07T06:53:26.193384+0000","last_clean":"2024-03-07T06:53:26.193384+0000","last_became_active":"2024-03-07T06:53:25.197396+0000","last_became_peered":"2024-03-07T06:53:25.197396+0000","last_unstale":"2024-03-07T06:53:26.193384+0000","last_undegraded":"2024-03-07T06:53:26.193384+0000","last_fullsized":"2024-03-07T06:53:26.193384+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:49:12.962971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797908+0000","last_change":"2024-03-07T06:53:25.198602+0000","last_active":"2024-03-07T06:53:26.797908+0000","last_peered":"2024-03-07T06:53:26.797908+0000","last_clean":"2024-03-07T06:53:26.797908+0000","last_became_active":"2024-03-07T06:53:25.197653+0000","last_became_peered":"2024-03-07T06:53:25.197653+0000","last_unstale":"2024-03-07T06:53:26.797908+0000","last_undegraded":"2024-03-07T06:53:26.797908+0000","last_fullsized":"2024-03-07T06:53:26.797908+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:03:09.371531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555920+0000","last_change":"2024-03-07T06:53:25.180030+0000","last_active":"2024-03-07T06:53:26.555920+0000","last_peered":"2024-03-07T06:53:26.555920+0000","last_clean":"2024-03-07T06:53:26.555920+0000","last_became_active":"2024-03-07T06:53:25.178772+0000","last_became_peered":"2024-03-07T06:53:25.178772+0000","last_unstale":"2024-03-07T06:53:26.555920+0000","last_undegraded":"2024-03-07T06:53:26.555920+0000","last_fullsized":"2024-03-07T06:53:26.555920+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:46:45.216443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568373+0000","last_change":"2024-03-07T06:53:25.197311+0000","last_active":"2024-03-07T06:53:26.568373+0000","last_peered":"2024-03-07T06:53:26.568373+0000","last_clean":"2024-03-07T06:53:26.568373+0000","last_became_active":"2024-03-07T06:53:25.196985+0000","last_became_peered":"2024-03-07T06:53:25.196985+0000","last_unstale":"2024-03-07T06:53:26.568373+0000","last_undegraded":"2024-03-07T06:53:26.568373+0000","last_fullsized":"2024-03-07T06:53:26.568373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:05:54.805828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192024+0000","last_change":"2024-03-07T06:53:25.204127+0000","last_active":"2024-03-07T06:53:26.192024+0000","last_peered":"2024-03-07T06:53:26.192024+0000","last_clean":"2024-03-07T06:53:26.192024+0000","last_became_active":"2024-03-07T06:53:25.203266+0000","last_became_peered":"2024-03-07T06:53:25.203266+0000","last_unstale":"2024-03-07T06:53:26.192024+0000","last_undegraded":"2024-03-07T06:53:26.192024+0000","last_fullsized":"2024-03-07T06:53:26.192024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:13:15.845709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797626+0000","last_change":"2024-03-07T06:53:25.194438+0000","last_active":"2024-03-07T06:53:26.797626+0000","last_peered":"2024-03-07T06:53:26.797626+0000","last_clean":"2024-03-07T06:53:26.797626+0000","last_became_active":"2024-03-07T06:53:25.194130+0000","last_became_peered":"2024-03-07T06:53:25.194130+0000","last_unstale":"2024-03-07T06:53:26.797626+0000","last_undegraded":"2024-03-07T06:53:26.797626+0000","last_fullsized":"2024-03-07T06:53:26.797626+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:19:59.756501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797261+0000","last_change":"2024-03-07T06:53:25.206131+0000","last_active":"2024-03-07T06:53:26.797261+0000","last_peered":"2024-03-07T06:53:26.797261+0000","last_clean":"2024-03-07T06:53:26.797261+0000","last_became_active":"2024-03-07T06:53:25.205578+0000","last_became_peered":"2024-03-07T06:53:25.205578+0000","last_unstale":"2024-03-07T06:53:26.797261+0000","last_undegraded":"2024-03-07T06:53:26.797261+0000","last_fullsized":"2024-03-07T06:53:26.797261+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:44.090672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195955+0000","last_change":"2024-03-07T06:53:25.188986+0000","last_active":"2024-03-07T06:53:26.195955+0000","last_peered":"2024-03-07T06:53:26.195955+0000","last_clean":"2024-03-07T06:53:26.195955+0000","last_became_active":"2024-03-07T06:53:25.188678+0000","last_became_peered":"2024-03-07T06:53:25.188678+0000","last_unstale":"2024-03-07T06:53:26.195955+0000","last_undegraded":"2024-03-07T06:53:26.195955+0000","last_fullsized":"2024-03-07T06:53:26.195955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:32.048437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798581+0000","last_change":"2024-03-07T06:53:25.204686+0000","last_active":"2024-03-07T06:53:26.798581+0000","last_peered":"2024-03-07T06:53:26.798581+0000","last_clean":"2024-03-07T06:53:26.798581+0000","last_became_active":"2024-03-07T06:53:25.203607+0000","last_became_peered":"2024-03-07T06:53:25.203607+0000","last_unstale":"2024-03-07T06:53:26.798581+0000","last_undegraded":"2024-03-07T06:53:26.798581+0000","last_fullsized":"2024-03-07T06:53:26.798581+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:27:56.445897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568794+0000","last_change":"2024-03-07T06:53:25.190607+0000","last_active":"2024-03-07T06:53:26.568794+0000","last_peered":"2024-03-07T06:53:26.568794+0000","last_clean":"2024-03-07T06:53:26.568794+0000","last_became_active":"2024-03-07T06:53:25.190083+0000","last_became_peered":"2024-03-07T06:53:25.190083+0000","last_unstale":"2024-03-07T06:53:26.568794+0000","last_undegraded":"2024-03-07T06:53:26.568794+0000","last_fullsized":"2024-03-07T06:53:26.568794+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:47:02.991917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555545+0000","last_change":"2024-03-07T06:53:25.200334+0000","last_active":"2024-03-07T06:53:26.555545+0000","last_peered":"2024-03-07T06:53:26.555545+0000","last_clean":"2024-03-07T06:53:26.555545+0000","last_became_active":"2024-03-07T06:53:25.200096+0000","last_became_peered":"2024-03-07T06:53:25.200096+0000","last_unstale":"2024-03-07T06:53:26.555545+0000","last_undegraded":"2024-03-07T06:53:26.555545+0000","last_fullsized":"2024-03-07T06:53:26.555545+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:33:47.684117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193801+0000","last_change":"2024-03-07T06:53:25.202494+0000","last_active":"2024-03-07T06:53:26.193801+0000","last_peered":"2024-03-07T06:53:26.193801+0000","last_clean":"2024-03-07T06:53:26.193801+0000","last_became_active":"2024-03-07T06:53:25.202185+0000","last_became_peered":"2024-03-07T06:53:25.202185+0000","last_unstale":"2024-03-07T06:53:26.193801+0000","last_undegraded":"2024-03-07T06:53:26.193801+0000","last_fullsized":"2024-03-07T06:53:26.193801+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:36.565865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192462+0000","last_change":"2024-03-07T06:53:25.203558+0000","last_active":"2024-03-07T06:53:26.192462+0000","last_peered":"2024-03-07T06:53:26.192462+0000","last_clean":"2024-03-07T06:53:26.192462+0000","last_became_active":"2024-03-07T06:53:25.202854+0000","last_became_peered":"2024-03-07T06:53:25.202854+0000","last_unstale":"2024-03-07T06:53:26.192462+0000","last_undegraded":"2024-03-07T06:53:26.192462+0000","last_fullsized":"2024-03-07T06:53:26.192462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:34:02.638312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195196+0000","last_change":"2024-03-07T06:53:25.193176+0000","last_active":"2024-03-07T06:53:26.195196+0000","last_peered":"2024-03-07T06:53:26.195196+0000","last_clean":"2024-03-07T06:53:26.195196+0000","last_became_active":"2024-03-07T06:53:25.192881+0000","last_became_peered":"2024-03-07T06:53:25.192881+0000","last_unstale":"2024-03-07T06:53:26.195196+0000","last_undegraded":"2024-03-07T06:53:26.195196+0000","last_fullsized":"2024-03-07T06:53:26.195196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:07:11.833897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798298+0000","last_change":"2024-03-07T06:53:25.205870+0000","last_active":"2024-03-07T06:53:26.798298+0000","last_peered":"2024-03-07T06:53:26.798298+0000","last_clean":"2024-03-07T06:53:26.798298+0000","last_became_active":"2024-03-07T06:53:25.205595+0000","last_became_peered":"2024-03-07T06:53:25.205595+0000","last_unstale":"2024-03-07T06:53:26.798298+0000","last_undegraded":"2024-03-07T06:53:26.798298+0000","last_fullsized":"2024-03-07T06:53:26.798298+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:13:01.272121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797104+0000","last_change":"2024-03-07T06:53:25.187957+0000","last_active":"2024-03-07T06:53:26.797104+0000","last_peered":"2024-03-07T06:53:26.797104+0000","last_clean":"2024-03-07T06:53:26.797104+0000","last_became_active":"2024-03-07T06:53:25.184716+0000","last_became_peered":"2024-03-07T06:53:25.184716+0000","last_unstale":"2024-03-07T06:53:26.797104+0000","last_undegraded":"2024-03-07T06:53:26.797104+0000","last_fullsized":"2024-03-07T06:53:26.797104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:14.520156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194874+0000","last_change":"2024-03-07T06:53:25.194725+0000","last_active":"2024-03-07T06:53:26.194874+0000","last_peered":"2024-03-07T06:53:26.194874+0000","last_clean":"2024-03-07T06:53:26.194874+0000","last_became_active":"2024-03-07T06:53:25.194434+0000","last_became_peered":"2024-03-07T06:53:25.194434+0000","last_unstale":"2024-03-07T06:53:26.194874+0000","last_undegraded":"2024-03-07T06:53:26.194874+0000","last_fullsized":"2024-03-07T06:53:26.194874+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:08:50.099907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195182+0000","last_change":"2024-03-07T06:53:25.188058+0000","last_active":"2024-03-07T06:53:26.195182+0000","last_peered":"2024-03-07T06:53:26.195182+0000","last_clean":"2024-03-07T06:53:26.195182+0000","last_became_active":"2024-03-07T06:53:25.187778+0000","last_became_peered":"2024-03-07T06:53:25.187778+0000","last_unstale":"2024-03-07T06:53:26.195182+0000","last_undegraded":"2024-03-07T06:53:26.195182+0000","last_fullsized":"2024-03-07T06:53:26.195182+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:41:07.796024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796976+0000","last_change":"2024-03-07T06:53:25.203241+0000","last_active":"2024-03-07T06:53:26.796976+0000","last_peered":"2024-03-07T06:53:26.796976+0000","last_clean":"2024-03-07T06:53:26.796976+0000","last_became_active":"2024-03-07T06:53:25.202804+0000","last_became_peered":"2024-03-07T06:53:25.202804+0000","last_unstale":"2024-03-07T06:53:26.796976+0000","last_undegraded":"2024-03-07T06:53:26.796976+0000","last_fullsized":"2024-03-07T06:53:26.796976+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:46:45.216443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568377+0000","last_change":"2024-03-07T06:53:25.193410+0000","last_active":"2024-03-07T06:53:26.568377+0000","last_peered":"2024-03-07T06:53:26.568377+0000","last_clean":"2024-03-07T06:53:26.568377+0000","last_became_active":"2024-03-07T06:53:25.193130+0000","last_became_peered":"2024-03-07T06:53:25.193130+0000","last_unstale":"2024-03-07T06:53:26.568377+0000","last_undegraded":"2024-03-07T06:53:26.568377+0000","last_fullsized":"2024-03-07T06:53:26.568377+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:17:12.387823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797108+0000","last_change":"2024-03-07T06:53:25.181460+0000","last_active":"2024-03-07T06:53:26.797108+0000","last_peered":"2024-03-07T06:53:26.797108+0000","last_clean":"2024-03-07T06:53:26.797108+0000","last_became_active":"2024-03-07T06:53:25.181176+0000","last_became_peered":"2024-03-07T06:53:25.181176+0000","last_unstale":"2024-03-07T06:53:26.797108+0000","last_undegraded":"2024-03-07T06:53:26.797108+0000","last_fullsized":"2024-03-07T06:53:26.797108+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:31:32.369129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797278+0000","last_change":"2024-03-07T06:53:25.185869+0000","last_active":"2024-03-07T06:53:26.797278+0000","last_peered":"2024-03-07T06:53:26.797278+0000","last_clean":"2024-03-07T06:53:26.797278+0000","last_became_active":"2024-03-07T06:53:25.185574+0000","last_became_peered":"2024-03-07T06:53:25.185574+0000","last_unstale":"2024-03-07T06:53:26.797278+0000","last_undegraded":"2024-03-07T06:53:26.797278+0000","last_fullsized":"2024-03-07T06:53:26.797278+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:32.048437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569830+0000","last_change":"2024-03-07T06:53:24.190584+0000","last_active":"2024-03-07T06:53:26.569830+0000","last_peered":"2024-03-07T06:53:26.569830+0000","last_clean":"2024-03-07T06:53:26.569830+0000","last_became_active":"2024-03-07T06:53:24.190262+0000","last_became_peered":"2024-03-07T06:53:24.190262+0000","last_unstale":"2024-03-07T06:53:26.569830+0000","last_undegraded":"2024-03-07T06:53:26.569830+0000","last_fullsized":"2024-03-07T06:53:26.569830+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:53:36.898081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.196313+0000","last_change":"2024-03-07T06:53:25.195388+0000","last_active":"2024-03-07T06:53:26.196313+0000","last_peered":"2024-03-07T06:53:26.196313+0000","last_clean":"2024-03-07T06:53:26.196313+0000","last_became_active":"2024-03-07T06:53:25.192645+0000","last_became_peered":"2024-03-07T06:53:25.192645+0000","last_unstale":"2024-03-07T06:53:26.196313+0000","last_undegraded":"2024-03-07T06:53:26.196313+0000","last_fullsized":"2024-03-07T06:53:26.196313+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:12:49.521040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797418+0000","last_change":"2024-03-07T06:53:24.190898+0000","last_active":"2024-03-07T06:53:26.797418+0000","last_peered":"2024-03-07T06:53:26.797418+0000","last_clean":"2024-03-07T06:53:26.797418+0000","last_became_active":"2024-03-07T06:53:24.190216+0000","last_became_peered":"2024-03-07T06:53:24.190216+0000","last_unstale":"2024-03-07T06:53:26.797418+0000","last_undegraded":"2024-03-07T06:53:26.797418+0000","last_fullsized":"2024-03-07T06:53:26.797418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:14:52.563314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193786+0000","last_change":"2024-03-07T06:53:25.206962+0000","last_active":"2024-03-07T06:53:26.193786+0000","last_peered":"2024-03-07T06:53:26.193786+0000","last_clean":"2024-03-07T06:53:26.193786+0000","last_became_active":"2024-03-07T06:53:25.206428+0000","last_became_peered":"2024-03-07T06:53:25.206428+0000","last_unstale":"2024-03-07T06:53:26.193786+0000","last_undegraded":"2024-03-07T06:53:26.193786+0000","last_fullsized":"2024-03-07T06:53:26.193786+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:19:59.756501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798392+0000","last_change":"2024-03-07T06:53:24.189931+0000","last_active":"2024-03-07T06:53:26.798392+0000","last_peered":"2024-03-07T06:53:26.798392+0000","last_clean":"2024-03-07T06:53:26.798392+0000","last_became_active":"2024-03-07T06:53:24.189610+0000","last_became_peered":"2024-03-07T06:53:24.189610+0000","last_unstale":"2024-03-07T06:53:26.798392+0000","last_undegraded":"2024-03-07T06:53:26.798392+0000","last_fullsized":"2024-03-07T06:53:26.798392+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:36:40.695222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798250+0000","last_change":"2024-03-07T06:53:25.190984+0000","last_active":"2024-03-07T06:53:26.798250+0000","last_peered":"2024-03-07T06:53:26.798250+0000","last_clean":"2024-03-07T06:53:26.798250+0000","last_became_active":"2024-03-07T06:53:25.190624+0000","last_became_peered":"2024-03-07T06:53:25.190624+0000","last_unstale":"2024-03-07T06:53:26.798250+0000","last_undegraded":"2024-03-07T06:53:26.798250+0000","last_fullsized":"2024-03-07T06:53:26.798250+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:36:14.943170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.189900+0000","last_change":"2024-03-07T06:53:24.192196+0000","last_active":"2024-03-07T06:53:26.189900+0000","last_peered":"2024-03-07T06:53:26.189900+0000","last_clean":"2024-03-07T06:53:26.189900+0000","last_became_active":"2024-03-07T06:53:24.191698+0000","last_became_peered":"2024-03-07T06:53:24.191698+0000","last_unstale":"2024-03-07T06:53:26.189900+0000","last_undegraded":"2024-03-07T06:53:26.189900+0000","last_fullsized":"2024-03-07T06:53:26.189900+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:45:09.141933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796861+0000","last_change":"2024-03-07T06:53:25.193044+0000","last_active":"2024-03-07T06:53:26.796861+0000","last_peered":"2024-03-07T06:53:26.796861+0000","last_clean":"2024-03-07T06:53:26.796861+0000","last_became_active":"2024-03-07T06:53:25.192696+0000","last_became_peered":"2024-03-07T06:53:25.192696+0000","last_unstale":"2024-03-07T06:53:26.796861+0000","last_undegraded":"2024-03-07T06:53:26.796861+0000","last_fullsized":"2024-03-07T06:53:26.796861+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:45:10.156912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569421+0000","last_change":"2024-03-07T06:53:24.180332+0000","last_active":"2024-03-07T06:53:26.569421+0000","last_peered":"2024-03-07T06:53:26.569421+0000","last_clean":"2024-03-07T06:53:26.569421+0000","last_became_active":"2024-03-07T06:53:24.179783+0000","last_became_peered":"2024-03-07T06:53:24.179783+0000","last_unstale":"2024-03-07T06:53:26.569421+0000","last_undegraded":"2024-03-07T06:53:26.569421+0000","last_fullsized":"2024-03-07T06:53:26.569421+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:03.714029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556363+0000","last_change":"2024-03-07T06:53:25.193219+0000","last_active":"2024-03-07T06:53:26.556363+0000","last_peered":"2024-03-07T06:53:26.556363+0000","last_clean":"2024-03-07T06:53:26.556363+0000","last_became_active":"2024-03-07T06:53:25.192945+0000","last_became_peered":"2024-03-07T06:53:25.192945+0000","last_unstale":"2024-03-07T06:53:26.556363+0000","last_undegraded":"2024-03-07T06:53:26.556363+0000","last_fullsized":"2024-03-07T06:53:26.556363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:44.090672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797025+0000","last_change":"2024-03-07T06:53:24.186564+0000","last_active":"2024-03-07T06:53:26.797025+0000","last_peered":"2024-03-07T06:53:26.797025+0000","last_clean":"2024-03-07T06:53:26.797025+0000","last_became_active":"2024-03-07T06:53:24.184717+0000","last_became_peered":"2024-03-07T06:53:24.184717+0000","last_unstale":"2024-03-07T06:53:26.797025+0000","last_undegraded":"2024-03-07T06:53:26.797025+0000","last_fullsized":"2024-03-07T06:53:26.797025+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:07:10.818918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555572+0000","last_change":"2024-03-07T06:53:25.196813+0000","last_active":"2024-03-07T06:53:26.555572+0000","last_peered":"2024-03-07T06:53:26.555572+0000","last_clean":"2024-03-07T06:53:26.555572+0000","last_became_active":"2024-03-07T06:53:25.195979+0000","last_became_peered":"2024-03-07T06:53:25.195979+0000","last_unstale":"2024-03-07T06:53:26.555572+0000","last_undegraded":"2024-03-07T06:53:26.555572+0000","last_fullsized":"2024-03-07T06:53:26.555572+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:53:37.913060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193612+0000","last_change":"2024-03-07T06:53:24.187243+0000","last_active":"2024-03-07T06:53:26.193612+0000","last_peered":"2024-03-07T06:53:26.193612+0000","last_clean":"2024-03-07T06:53:26.193612+0000","last_became_active":"2024-03-07T06:53:24.186379+0000","last_became_peered":"2024-03-07T06:53:24.186379+0000","last_unstale":"2024-03-07T06:53:26.193612+0000","last_undegraded":"2024-03-07T06:53:26.193612+0000","last_fullsized":"2024-03-07T06:53:26.193612+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:12:48.506061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195332+0000","last_change":"2024-03-07T06:53:25.203867+0000","last_active":"2024-03-07T06:53:26.195332+0000","last_peered":"2024-03-07T06:53:26.195332+0000","last_clean":"2024-03-07T06:53:26.195332+0000","last_became_active":"2024-03-07T06:53:25.203567+0000","last_became_peered":"2024-03-07T06:53:25.203567+0000","last_unstale":"2024-03-07T06:53:26.195332+0000","last_undegraded":"2024-03-07T06:53:26.195332+0000","last_fullsized":"2024-03-07T06:53:26.195332+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:43:40.476283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555610+0000","last_change":"2024-03-07T06:53:24.176450+0000","last_active":"2024-03-07T06:53:26.555610+0000","last_peered":"2024-03-07T06:53:26.555610+0000","last_clean":"2024-03-07T06:53:26.555610+0000","last_became_active":"2024-03-07T06:53:24.175876+0000","last_became_peered":"2024-03-07T06:53:24.175876+0000","last_unstale":"2024-03-07T06:53:26.555610+0000","last_undegraded":"2024-03-07T06:53:26.555610+0000","last_fullsized":"2024-03-07T06:53:26.555610+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:20:37.347168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797243+0000","last_change":"2024-03-07T06:53:25.205883+0000","last_active":"2024-03-07T06:53:26.797243+0000","last_peered":"2024-03-07T06:53:26.797243+0000","last_clean":"2024-03-07T06:53:26.797243+0000","last_became_active":"2024-03-07T06:53:25.205193+0000","last_became_peered":"2024-03-07T06:53:25.205193+0000","last_unstale":"2024-03-07T06:53:26.797243+0000","last_undegraded":"2024-03-07T06:53:26.797243+0000","last_fullsized":"2024-03-07T06:53:26.797243+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:49:12.962971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797587+0000","last_change":"2024-03-07T06:53:24.184752+0000","last_active":"2024-03-07T06:53:26.797587+0000","last_peered":"2024-03-07T06:53:26.797587+0000","last_clean":"2024-03-07T06:53:26.797587+0000","last_became_active":"2024-03-07T06:53:24.184338+0000","last_became_peered":"2024-03-07T06:53:24.184338+0000","last_unstale":"2024-03-07T06:53:26.797587+0000","last_undegraded":"2024-03-07T06:53:26.797587+0000","last_fullsized":"2024-03-07T06:53:26.797587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:41:06.781045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194726+0000","last_change":"2024-03-07T06:53:25.203839+0000","last_active":"2024-03-07T06:53:26.194726+0000","last_peered":"2024-03-07T06:53:26.194726+0000","last_clean":"2024-03-07T06:53:26.194726+0000","last_became_active":"2024-03-07T06:53:25.203570+0000","last_became_peered":"2024-03-07T06:53:25.203570+0000","last_unstale":"2024-03-07T06:53:26.194726+0000","last_undegraded":"2024-03-07T06:53:26.194726+0000","last_fullsized":"2024-03-07T06:53:26.194726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:25:16.302702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556704+0000","last_change":"2024-03-07T06:53:24.180862+0000","last_active":"2024-03-07T06:53:26.556704+0000","last_peered":"2024-03-07T06:53:26.556704+0000","last_clean":"2024-03-07T06:53:26.556704+0000","last_became_active":"2024-03-07T06:53:24.180558+0000","last_became_peered":"2024-03-07T06:53:24.180558+0000","last_unstale":"2024-03-07T06:53:26.556704+0000","last_undegraded":"2024-03-07T06:53:26.556704+0000","last_fullsized":"2024-03-07T06:53:26.556704+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:23:18.942189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569862+0000","last_change":"2024-03-07T06:53:25.193872+0000","last_active":"2024-03-07T06:53:26.569862+0000","last_peered":"2024-03-07T06:53:26.569862+0000","last_clean":"2024-03-07T06:53:26.569862+0000","last_became_active":"2024-03-07T06:53:25.193677+0000","last_became_peered":"2024-03-07T06:53:25.193677+0000","last_unstale":"2024-03-07T06:53:26.569862+0000","last_undegraded":"2024-03-07T06:53:26.569862+0000","last_fullsized":"2024-03-07T06:53:26.569862+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:40:13.617392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192612+0000","last_change":"2024-03-07T06:53:24.193883+0000","last_active":"2024-03-07T06:53:26.192612+0000","last_peered":"2024-03-07T06:53:26.192612+0000","last_clean":"2024-03-07T06:53:26.192612+0000","last_became_active":"2024-03-07T06:53:24.193619+0000","last_became_peered":"2024-03-07T06:53:24.193619+0000","last_unstale":"2024-03-07T06:53:26.192612+0000","last_undegraded":"2024-03-07T06:53:26.192612+0000","last_fullsized":"2024-03-07T06:53:26.192612+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:33:46.669137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.191713+0000","last_change":"2024-03-07T06:53:25.208536+0000","last_active":"2024-03-07T06:53:26.191713+0000","last_peered":"2024-03-07T06:53:26.191713+0000","last_clean":"2024-03-07T06:53:26.191713+0000","last_became_active":"2024-03-07T06:53:25.208127+0000","last_became_peered":"2024-03-07T06:53:25.208127+0000","last_unstale":"2024-03-07T06:53:26.191713+0000","last_undegraded":"2024-03-07T06:53:26.191713+0000","last_fullsized":"2024-03-07T06:53:26.191713+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:54:32.067212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798314+0000","last_change":"2024-03-07T06:53:24.182955+0000","last_active":"2024-03-07T06:53:26.798314+0000","last_peered":"2024-03-07T06:53:26.798314+0000","last_clean":"2024-03-07T06:53:26.798314+0000","last_became_active":"2024-03-07T06:53:24.182685+0000","last_became_peered":"2024-03-07T06:53:24.182685+0000","last_unstale":"2024-03-07T06:53:26.798314+0000","last_undegraded":"2024-03-07T06:53:26.798314+0000","last_fullsized":"2024-03-07T06:53:26.798314+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:35:03.451977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194621+0000","last_change":"2024-03-07T06:53:25.190435+0000","last_active":"2024-03-07T06:53:26.194621+0000","last_peered":"2024-03-07T06:53:26.194621+0000","last_clean":"2024-03-07T06:53:26.194621+0000","last_became_active":"2024-03-07T06:53:25.189951+0000","last_became_peered":"2024-03-07T06:53:25.189951+0000","last_unstale":"2024-03-07T06:53:26.194621+0000","last_undegraded":"2024-03-07T06:53:26.194621+0000","last_fullsized":"2024-03-07T06:53:26.194621+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:14:53.578293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569710+0000","last_change":"2024-03-07T06:53:24.177161+0000","last_active":"2024-03-07T06:53:26.569710+0000","last_peered":"2024-03-07T06:53:26.569710+0000","last_clean":"2024-03-07T06:53:26.569710+0000","last_became_active":"2024-03-07T06:53:24.176716+0000","last_became_peered":"2024-03-07T06:53:24.176716+0000","last_unstale":"2024-03-07T06:53:26.569710+0000","last_undegraded":"2024-03-07T06:53:26.569710+0000","last_fullsized":"2024-03-07T06:53:26.569710+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:46:44.201464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556482+0000","last_change":"2024-03-07T06:53:25.203128+0000","last_active":"2024-03-07T06:53:26.556482+0000","last_peered":"2024-03-07T06:53:26.556482+0000","last_clean":"2024-03-07T06:53:26.556482+0000","last_became_active":"2024-03-07T06:53:25.202924+0000","last_became_peered":"2024-03-07T06:53:25.202924+0000","last_unstale":"2024-03-07T06:53:26.556482+0000","last_undegraded":"2024-03-07T06:53:26.556482+0000","last_fullsized":"2024-03-07T06:53:26.556482+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:19:59.756501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.190349+0000","last_change":"2024-03-07T06:53:24.188380+0000","last_active":"2024-03-07T06:53:26.190349+0000","last_peered":"2024-03-07T06:53:26.190349+0000","last_clean":"2024-03-07T06:53:26.190349+0000","last_became_active":"2024-03-07T06:53:24.187005+0000","last_became_peered":"2024-03-07T06:53:24.187005+0000","last_unstale":"2024-03-07T06:53:26.190349+0000","last_undegraded":"2024-03-07T06:53:26.190349+0000","last_fullsized":"2024-03-07T06:53:26.190349+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:43:50.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568329+0000","last_change":"2024-03-07T06:53:25.185049+0000","last_active":"2024-03-07T06:53:26.568329+0000","last_peered":"2024-03-07T06:53:26.568329+0000","last_clean":"2024-03-07T06:53:26.568329+0000","last_became_active":"2024-03-07T06:53:25.184192+0000","last_became_peered":"2024-03-07T06:53:25.184192+0000","last_unstale":"2024-03-07T06:53:26.568329+0000","last_undegraded":"2024-03-07T06:53:26.568329+0000","last_fullsized":"2024-03-07T06:53:26.568329+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:45.850119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568464+0000","last_change":"2024-03-07T06:53:24.177157+0000","last_active":"2024-03-07T06:53:26.568464+0000","last_peered":"2024-03-07T06:53:26.568464+0000","last_clean":"2024-03-07T06:53:26.568464+0000","last_became_active":"2024-03-07T06:53:24.176724+0000","last_became_peered":"2024-03-07T06:53:24.176724+0000","last_unstale":"2024-03-07T06:53:26.568464+0000","last_undegraded":"2024-03-07T06:53:26.568464+0000","last_fullsized":"2024-03-07T06:53:26.568464+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:54.259961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.190647+0000","last_change":"2024-03-07T06:53:25.194921+0000","last_active":"2024-03-07T06:53:26.190647+0000","last_peered":"2024-03-07T06:53:26.190647+0000","last_clean":"2024-03-07T06:53:26.190647+0000","last_became_active":"2024-03-07T06:53:25.193907+0000","last_became_peered":"2024-03-07T06:53:25.193907+0000","last_unstale":"2024-03-07T06:53:26.190647+0000","last_undegraded":"2024-03-07T06:53:26.190647+0000","last_fullsized":"2024-03-07T06:53:26.190647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:21:58.317633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":22,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555716+0000","last_change":"2024-03-07T06:53:24.181056+0000","last_active":"2024-03-07T06:53:26.555716+0000","last_peered":"2024-03-07T06:53:26.555716+0000","last_clean":"2024-03-07T06:53:26.555716+0000","last_became_active":"2024-03-07T06:53:24.180668+0000","last_became_peered":"2024-03-07T06:53:24.180668+0000","last_unstale":"2024-03-07T06:53:26.555716+0000","last_undegraded":"2024-03-07T06:53:26.555716+0000","last_fullsized":"2024-03-07T06:53:26.555716+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:27:55.430918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798458+0000","last_change":"2024-03-07T06:53:25.196095+0000","last_active":"2024-03-07T06:53:26.798458+0000","last_peered":"2024-03-07T06:53:26.798458+0000","last_clean":"2024-03-07T06:53:26.798458+0000","last_became_active":"2024-03-07T06:53:25.195557+0000","last_became_peered":"2024-03-07T06:53:25.195557+0000","last_unstale":"2024-03-07T06:53:26.798458+0000","last_undegraded":"2024-03-07T06:53:26.798458+0000","last_fullsized":"2024-03-07T06:53:26.798458+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:20:38.362147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798600+0000","last_change":"2024-03-07T06:53:24.179843+0000","last_active":"2024-03-07T06:53:26.798600+0000","last_peered":"2024-03-07T06:53:26.798600+0000","last_clean":"2024-03-07T06:53:26.798600+0000","last_became_active":"2024-03-07T06:53:24.179393+0000","last_became_peered":"2024-03-07T06:53:24.179393+0000","last_unstale":"2024-03-07T06:53:26.798600+0000","last_undegraded":"2024-03-07T06:53:26.798600+0000","last_fullsized":"2024-03-07T06:53:26.798600+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:51:11.895335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556574+0000","last_change":"2024-03-07T06:53:25.205258+0000","last_active":"2024-03-07T06:53:26.556574+0000","last_peered":"2024-03-07T06:53:26.556574+0000","last_clean":"2024-03-07T06:53:26.556574+0000","last_became_active":"2024-03-07T06:53:25.204991+0000","last_became_peered":"2024-03-07T06:53:25.204991+0000","last_unstale":"2024-03-07T06:53:26.556574+0000","last_undegraded":"2024-03-07T06:53:26.556574+0000","last_fullsized":"2024-03-07T06:53:26.556574+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:21:58.317633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797419+0000","last_change":"2024-03-07T06:53:24.185672+0000","last_active":"2024-03-07T06:53:26.797419+0000","last_peered":"2024-03-07T06:53:26.797419+0000","last_clean":"2024-03-07T06:53:26.797419+0000","last_became_active":"2024-03-07T06:53:24.185368+0000","last_became_peered":"2024-03-07T06:53:24.185368+0000","last_unstale":"2024-03-07T06:53:26.797419+0000","last_undegraded":"2024-03-07T06:53:26.797419+0000","last_fullsized":"2024-03-07T06:53:26.797419+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:36:40.695222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195109+0000","last_change":"2024-03-07T06:53:25.186795+0000","last_active":"2024-03-07T06:53:26.195109+0000","last_peered":"2024-03-07T06:53:26.195109+0000","last_clean":"2024-03-07T06:53:26.195109+0000","last_became_active":"2024-03-07T06:53:25.186502+0000","last_became_peered":"2024-03-07T06:53:25.186502+0000","last_unstale":"2024-03-07T06:53:26.195109+0000","last_undegraded":"2024-03-07T06:53:26.195109+0000","last_fullsized":"2024-03-07T06:53:26.195109+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:36:41.710201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193159+0000","last_change":"2024-03-07T06:53:24.186577+0000","last_active":"2024-03-07T06:53:26.193159+0000","last_peered":"2024-03-07T06:53:26.193159+0000","last_clean":"2024-03-07T06:53:26.193159+0000","last_became_active":"2024-03-07T06:53:24.186258+0000","last_became_peered":"2024-03-07T06:53:24.186258+0000","last_unstale":"2024-03-07T06:53:26.193159+0000","last_undegraded":"2024-03-07T06:53:26.193159+0000","last_fullsized":"2024-03-07T06:53:26.193159+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:31.033458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195414+0000","last_change":"2024-03-07T06:53:25.189444+0000","last_active":"2024-03-07T06:53:26.195414+0000","last_peered":"2024-03-07T06:53:26.195414+0000","last_clean":"2024-03-07T06:53:26.195414+0000","last_became_active":"2024-03-07T06:53:25.187460+0000","last_became_peered":"2024-03-07T06:53:25.187460+0000","last_unstale":"2024-03-07T06:53:26.195414+0000","last_undegraded":"2024-03-07T06:53:26.195414+0000","last_fullsized":"2024-03-07T06:53:26.195414+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:28:41.703936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569599+0000","last_change":"2024-03-07T06:53:24.189372+0000","last_active":"2024-03-07T06:53:26.569599+0000","last_peered":"2024-03-07T06:53:26.569599+0000","last_clean":"2024-03-07T06:53:26.569599+0000","last_became_active":"2024-03-07T06:53:24.189061+0000","last_became_peered":"2024-03-07T06:53:24.189061+0000","last_unstale":"2024-03-07T06:53:26.569599+0000","last_undegraded":"2024-03-07T06:53:26.569599+0000","last_fullsized":"2024-03-07T06:53:26.569599+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:45:09.141933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568500+0000","last_change":"2024-03-07T06:53:25.193713+0000","last_active":"2024-03-07T06:53:26.568500+0000","last_peered":"2024-03-07T06:53:26.568500+0000","last_clean":"2024-03-07T06:53:26.568500+0000","last_became_active":"2024-03-07T06:53:25.186210+0000","last_became_peered":"2024-03-07T06:53:25.186210+0000","last_unstale":"2024-03-07T06:53:26.568500+0000","last_undegraded":"2024-03-07T06:53:26.568500+0000","last_fullsized":"2024-03-07T06:53:26.568500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:33:15.293206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.570025+0000","last_change":"2024-03-07T06:53:24.180089+0000","last_active":"2024-03-07T06:53:26.570025+0000","last_peered":"2024-03-07T06:53:26.570025+0000","last_clean":"2024-03-07T06:53:26.570025+0000","last_became_active":"2024-03-07T06:53:24.179418+0000","last_became_peered":"2024-03-07T06:53:24.179418+0000","last_unstale":"2024-03-07T06:53:26.570025+0000","last_undegraded":"2024-03-07T06:53:26.570025+0000","last_fullsized":"2024-03-07T06:53:26.570025+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:13.505176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797763+0000","last_change":"2024-03-07T06:53:25.196193+0000","last_active":"2024-03-07T06:53:26.797763+0000","last_peered":"2024-03-07T06:53:26.797763+0000","last_clean":"2024-03-07T06:53:26.797763+0000","last_became_active":"2024-03-07T06:53:25.195898+0000","last_became_peered":"2024-03-07T06:53:25.195898+0000","last_unstale":"2024-03-07T06:53:26.797763+0000","last_undegraded":"2024-03-07T06:53:26.797763+0000","last_fullsized":"2024-03-07T06:53:26.797763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:12:49.521040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193010+0000","last_change":"2024-03-07T06:53:24.191253+0000","last_active":"2024-03-07T06:53:26.193010+0000","last_peered":"2024-03-07T06:53:26.193010+0000","last_clean":"2024-03-07T06:53:26.193010+0000","last_became_active":"2024-03-07T06:53:24.190919+0000","last_became_peered":"2024-03-07T06:53:24.190919+0000","last_unstale":"2024-03-07T06:53:26.193010+0000","last_undegraded":"2024-03-07T06:53:26.193010+0000","last_fullsized":"2024-03-07T06:53:26.193010+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:43:50.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797624+0000","last_change":"2024-03-07T06:53:25.202509+0000","last_active":"2024-03-07T06:53:26.797624+0000","last_peered":"2024-03-07T06:53:26.797624+0000","last_clean":"2024-03-07T06:53:26.797624+0000","last_became_active":"2024-03-07T06:53:25.202242+0000","last_became_peered":"2024-03-07T06:53:25.202242+0000","last_unstale":"2024-03-07T06:53:26.797624+0000","last_undegraded":"2024-03-07T06:53:26.797624+0000","last_fullsized":"2024-03-07T06:53:26.797624+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:01:23.009414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555290+0000","last_change":"2024-03-07T06:53:24.181121+0000","last_active":"2024-03-07T06:53:26.555290+0000","last_peered":"2024-03-07T06:53:26.555290+0000","last_clean":"2024-03-07T06:53:26.555290+0000","last_became_active":"2024-03-07T06:53:24.180669+0000","last_became_peered":"2024-03-07T06:53:24.180669+0000","last_unstale":"2024-03-07T06:53:26.555290+0000","last_undegraded":"2024-03-07T06:53:26.555290+0000","last_fullsized":"2024-03-07T06:53:26.555290+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:24:07.355696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555633+0000","last_change":"2024-03-07T06:53:25.177613+0000","last_active":"2024-03-07T06:53:26.555633+0000","last_peered":"2024-03-07T06:53:26.555633+0000","last_clean":"2024-03-07T06:53:26.555633+0000","last_became_active":"2024-03-07T06:53:25.177320+0000","last_became_peered":"2024-03-07T06:53:25.177320+0000","last_unstale":"2024-03-07T06:53:26.555633+0000","last_undegraded":"2024-03-07T06:53:26.555633+0000","last_fullsized":"2024-03-07T06:53:26.555633+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:45:10.156912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.189619+0000","last_change":"2024-03-07T06:53:24.184705+0000","last_active":"2024-03-07T06:53:26.189619+0000","last_peered":"2024-03-07T06:53:26.189619+0000","last_clean":"2024-03-07T06:53:26.189619+0000","last_became_active":"2024-03-07T06:53:24.184354+0000","last_became_peered":"2024-03-07T06:53:24.184354+0000","last_unstale":"2024-03-07T06:53:26.189619+0000","last_undegraded":"2024-03-07T06:53:26.189619+0000","last_fullsized":"2024-03-07T06:53:26.189619+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:37.281189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.196424+0000","last_change":"2024-03-07T06:53:25.204962+0000","last_active":"2024-03-07T06:53:26.196424+0000","last_peered":"2024-03-07T06:53:26.196424+0000","last_clean":"2024-03-07T06:53:26.196424+0000","last_became_active":"2024-03-07T06:53:25.203898+0000","last_became_peered":"2024-03-07T06:53:25.203898+0000","last_unstale":"2024-03-07T06:53:26.196424+0000","last_undegraded":"2024-03-07T06:53:26.196424+0000","last_fullsized":"2024-03-07T06:53:26.196424+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:36:14.943170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569903+0000","last_change":"2024-03-07T06:53:24.182347+0000","last_active":"2024-03-07T06:53:26.569903+0000","last_peered":"2024-03-07T06:53:26.569903+0000","last_clean":"2024-03-07T06:53:26.569903+0000","last_became_active":"2024-03-07T06:53:24.182031+0000","last_became_peered":"2024-03-07T06:53:24.182031+0000","last_unstale":"2024-03-07T06:53:26.569903+0000","last_undegraded":"2024-03-07T06:53:26.569903+0000","last_fullsized":"2024-03-07T06:53:26.569903+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:33:46.669137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556121+0000","last_change":"2024-03-07T06:53:25.184985+0000","last_active":"2024-03-07T06:53:26.556121+0000","last_peered":"2024-03-07T06:53:26.556121+0000","last_clean":"2024-03-07T06:53:26.556121+0000","last_became_active":"2024-03-07T06:53:25.184052+0000","last_became_peered":"2024-03-07T06:53:25.184052+0000","last_unstale":"2024-03-07T06:53:26.556121+0000","last_undegraded":"2024-03-07T06:53:26.556121+0000","last_fullsized":"2024-03-07T06:53:26.556121+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:49:12.962971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192906+0000","last_change":"2024-03-07T06:53:24.194031+0000","last_active":"2024-03-07T06:53:26.192906+0000","last_peered":"2024-03-07T06:53:26.192906+0000","last_clean":"2024-03-07T06:53:26.192906+0000","last_became_active":"2024-03-07T06:53:24.193718+0000","last_became_peered":"2024-03-07T06:53:24.193718+0000","last_unstale":"2024-03-07T06:53:26.192906+0000","last_undegraded":"2024-03-07T06:53:26.192906+0000","last_fullsized":"2024-03-07T06:53:26.192906+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:53:36.898081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193172+0000","last_change":"2024-03-07T06:53:25.202526+0000","last_active":"2024-03-07T06:53:26.193172+0000","last_peered":"2024-03-07T06:53:26.193172+0000","last_clean":"2024-03-07T06:53:26.193172+0000","last_became_active":"2024-03-07T06:53:25.202243+0000","last_became_peered":"2024-03-07T06:53:25.202243+0000","last_unstale":"2024-03-07T06:53:26.193172+0000","last_undegraded":"2024-03-07T06:53:26.193172+0000","last_fullsized":"2024-03-07T06:53:26.193172+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:55.587418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796596+0000","last_change":"2024-03-07T06:53:24.183011+0000","last_active":"2024-03-07T06:53:26.796596+0000","last_peered":"2024-03-07T06:53:26.796596+0000","last_clean":"2024-03-07T06:53:26.796596+0000","last_became_active":"2024-03-07T06:53:24.182772+0000","last_became_peered":"2024-03-07T06:53:24.182772+0000","last_unstale":"2024-03-07T06:53:26.796596+0000","last_undegraded":"2024-03-07T06:53:26.796596+0000","last_fullsized":"2024-03-07T06:53:26.796596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:14:52.563314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568643+0000","last_change":"2024-03-07T06:53:25.173467+0000","last_active":"2024-03-07T06:53:26.568643+0000","last_peered":"2024-03-07T06:53:26.568643+0000","last_clean":"2024-03-07T06:53:26.568643+0000","last_became_active":"2024-03-07T06:53:25.173157+0000","last_became_peered":"2024-03-07T06:53:25.173157+0000","last_unstale":"2024-03-07T06:53:26.568643+0000","last_undegraded":"2024-03-07T06:53:26.568643+0000","last_fullsized":"2024-03-07T06:53:26.568643+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:04:56.048641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.191151+0000","last_change":"2024-03-07T06:53:24.196702+0000","last_active":"2024-03-07T06:53:26.191151+0000","last_peered":"2024-03-07T06:53:26.191151+0000","last_clean":"2024-03-07T06:53:26.191151+0000","last_became_active":"2024-03-07T06:53:24.196356+0000","last_became_peered":"2024-03-07T06:53:24.196356+0000","last_unstale":"2024-03-07T06:53:26.191151+0000","last_undegraded":"2024-03-07T06:53:26.191151+0000","last_fullsized":"2024-03-07T06:53:26.191151+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:54.259961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569797+0000","last_change":"2024-03-07T06:53:25.190375+0000","last_active":"2024-03-07T06:53:26.569797+0000","last_peered":"2024-03-07T06:53:26.569797+0000","last_clean":"2024-03-07T06:53:26.569797+0000","last_became_active":"2024-03-07T06:53:25.189627+0000","last_became_peered":"2024-03-07T06:53:25.189627+0000","last_unstale":"2024-03-07T06:53:26.569797+0000","last_undegraded":"2024-03-07T06:53:26.569797+0000","last_fullsized":"2024-03-07T06:53:26.569797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:39:28.721920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797059+0000","last_change":"2024-03-07T06:53:24.180383+0000","last_active":"2024-03-07T06:53:26.797059+0000","last_peered":"2024-03-07T06:53:26.797059+0000","last_clean":"2024-03-07T06:53:26.797059+0000","last_became_active":"2024-03-07T06:53:24.180057+0000","last_became_peered":"2024-03-07T06:53:24.180057+0000","last_unstale":"2024-03-07T06:53:26.797059+0000","last_undegraded":"2024-03-07T06:53:26.797059+0000","last_fullsized":"2024-03-07T06:53:26.797059+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:43:39.461304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192907+0000","last_change":"2024-03-07T06:53:25.195366+0000","last_active":"2024-03-07T06:53:26.192907+0000","last_peered":"2024-03-07T06:53:26.192907+0000","last_clean":"2024-03-07T06:53:26.192907+0000","last_became_active":"2024-03-07T06:53:25.194848+0000","last_became_peered":"2024-03-07T06:53:25.194848+0000","last_unstale":"2024-03-07T06:53:26.192907+0000","last_undegraded":"2024-03-07T06:53:26.192907+0000","last_fullsized":"2024-03-07T06:53:26.192907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:46:45.216443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796937+0000","last_change":"2024-03-07T06:53:24.185042+0000","last_active":"2024-03-07T06:53:26.796937+0000","last_peered":"2024-03-07T06:53:26.796937+0000","last_clean":"2024-03-07T06:53:26.796937+0000","last_became_active":"2024-03-07T06:53:24.184633+0000","last_became_peered":"2024-03-07T06:53:24.184633+0000","last_unstale":"2024-03-07T06:53:26.796937+0000","last_undegraded":"2024-03-07T06:53:26.796937+0000","last_fullsized":"2024-03-07T06:53:26.796937+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:35.550886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796752+0000","last_change":"2024-03-07T06:53:25.186793+0000","last_active":"2024-03-07T06:53:26.796752+0000","last_peered":"2024-03-07T06:53:26.796752+0000","last_clean":"2024-03-07T06:53:26.796752+0000","last_became_active":"2024-03-07T06:53:25.182526+0000","last_became_peered":"2024-03-07T06:53:25.182526+0000","last_unstale":"2024-03-07T06:53:26.796752+0000","last_undegraded":"2024-03-07T06:53:26.796752+0000","last_fullsized":"2024-03-07T06:53:26.796752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:53:37.913060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555749+0000","last_change":"2024-03-07T06:53:24.176113+0000","last_active":"2024-03-07T06:53:26.555749+0000","last_peered":"2024-03-07T06:53:26.555749+0000","last_clean":"2024-03-07T06:53:26.555749+0000","last_became_active":"2024-03-07T06:53:24.175506+0000","last_became_peered":"2024-03-07T06:53:24.175506+0000","last_unstale":"2024-03-07T06:53:26.555749+0000","last_undegraded":"2024-03-07T06:53:26.555749+0000","last_fullsized":"2024-03-07T06:53:26.555749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:20:28.607718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556710+0000","last_change":"2024-03-07T06:53:25.203466+0000","last_active":"2024-03-07T06:53:26.556710+0000","last_peered":"2024-03-07T06:53:26.556710+0000","last_clean":"2024-03-07T06:53:26.556710+0000","last_became_active":"2024-03-07T06:53:25.203113+0000","last_became_peered":"2024-03-07T06:53:25.203113+0000","last_unstale":"2024-03-07T06:53:26.556710+0000","last_undegraded":"2024-03-07T06:53:26.556710+0000","last_fullsized":"2024-03-07T06:53:26.556710+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:55.587418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194351+0000","last_change":"2024-03-07T06:53:24.186782+0000","last_active":"2024-03-07T06:53:26.194351+0000","last_peered":"2024-03-07T06:53:26.194351+0000","last_clean":"2024-03-07T06:53:26.194351+0000","last_became_active":"2024-03-07T06:53:24.186396+0000","last_became_peered":"2024-03-07T06:53:24.186396+0000","last_unstale":"2024-03-07T06:53:26.194351+0000","last_undegraded":"2024-03-07T06:53:26.194351+0000","last_fullsized":"2024-03-07T06:53:26.194351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:36:13.928191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.189635+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:53:26.189635+0000","last_peered":"2024-03-07T06:53:26.189635+0000","last_clean":"2024-03-07T06:53:26.189635+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:53:26.189635+0000","last_undegraded":"2024-03-07T06:53:26.189635+0000","last_fullsized":"2024-03-07T06:53:26.189635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":810252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":548864,"data_stored":220466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":219864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":552960,"data_stored":221406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":552960,"data_stored":220432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":221428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":219864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1138688,"data_stored":810196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:53:30.994 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-07T06:53:31.123 INFO:tasks.ceph.mds.a.smithi133.stderr:2024-03-07T06:53:31.123+0000 7f99fb000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-07T06:53:31.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.231+0000 7f6fa0e00640 1 Processor -- start 2024-03-07T06:53:31.231 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.231+0000 7f6fa0e00640 1 -- start start 2024-03-07T06:53:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.233+0000 7f6fa0e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b3c10 0x7f6f9c071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.232 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.233+0000 7f6fa0e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6f9c071b50 0x7f6f9c162f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.233+0000 7f6fa0e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9c071fe0 con 0x7f6f9c165ed0 2024-03-07T06:53:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.233+0000 7f6fa0e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9c0734a0 con 0x7f6f9c071b50 2024-03-07T06:53:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.233+0000 7f6fa0e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0ce540 con 0x7f6f9c0b3c10 2024-03-07T06:53:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.233+0000 7f6f99e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6f9c165ed0 0x7f6f9c06ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47450/0 (socket says 172.21.15.133:47450) 2024-03-07T06:53:31.233 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.233+0000 7f6f99e00640 1 -- 172.21.15.133:0/884724755 learned_addr learned my addr 172.21.15.133:0/884724755 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.233+0000 7f6f9a800640 1 --2- 172.21.15.133:0/884724755 >> v2:172.21.15.161:3300/0 conn(0x7f6f9c071b50 0x7f6f9c162f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.233+0000 7f6f99400640 1 --2- 172.21.15.133:0/884724755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b3c10 0x7f6f9c071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.233+0000 7f6f98a00640 1 -- 172.21.15.133:0/884724755 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691224380 0 0) 0x7f6f9c071fe0 con 0x7f6f9c165ed0 2024-03-07T06:53:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 -- 172.21.15.133:0/884724755 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f80003520 con 0x7f6f9c165ed0 2024-03-07T06:53:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 -- 172.21.15.133:0/884724755 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2399231875 0 0) 0x7f6f80003520 con 0x7f6f9c165ed0 2024-03-07T06:53:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 -- 172.21.15.133:0/884724755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b3c10 msgr2=0x7f6f9c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 --2- 172.21.15.133:0/884724755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b3c10 0x7f6f9c071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 -- 172.21.15.133:0/884724755 >> v2:172.21.15.161:3300/0 conn(0x7f6f9c071b50 msgr2=0x7f6f9c162f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 --2- 172.21.15.133:0/884724755 >> v2:172.21.15.161:3300/0 conn(0x7f6f9c071b50 0x7f6f9c162f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 -- 172.21.15.133:0/884724755 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c0cdf80 con 0x7f6f9c165ed0 2024-03-07T06:53:31.235 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 -- 172.21.15.133:0/884724755 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f6f8c0019f0 con 0x7f6f9c165ed0 2024-03-07T06:53:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 -- 172.21.15.133:0/884724755 >> v1:172.21.15.133:6789/0 conn(0x7f6f9c165ed0 legacy=0x7f6f9c06ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 --2- 172.21.15.133:0/884724755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b3c10 0x7f6f800039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 --2- 172.21.15.133:0/884724755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f80003f80 0x7f6f80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 --2- 172.21.15.133:0/884724755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f80004d40 0x7f6f800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 -- 172.21.15.133:0/884724755 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0734a0 con 0x7f6f80004d40 2024-03-07T06:53:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 -- 172.21.15.133:0/884724755 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c0ce540 con 0x7f6f80003f80 2024-03-07T06:53:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f98a00640 1 -- 172.21.15.133:0/884724755 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8000b9c0 con 0x7f6f9c0b3c10 2024-03-07T06:53:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.234+0000 7f6f9a800640 1 --2- 172.21.15.133:0/884724755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f80004d40 0x7f6f800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.235+0000 7f6f99e00640 1 --2- 172.21.15.133:0/884724755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b3c10 0x7f6f800039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.235+0000 7f6f9a800640 1 -- 172.21.15.133:0/884724755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b3c10 msgr2=0x7f6f800039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.235+0000 7f6f9a800640 1 --2- 172.21.15.133:0/884724755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b3c10 0x7f6f800039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.235+0000 7f6f99400640 1 --2- 172.21.15.133:0/884724755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f80003f80 0x7f6f80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.236 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.235+0000 7f6f9a800640 1 -- 172.21.15.133:0/884724755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f80003f80 msgr2=0x7f6f80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:31.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.235+0000 7f6f9a800640 1 --2- 172.21.15.133:0/884724755 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f80003f80 0x7f6f80004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.235+0000 7f6f9a800640 1 -- 172.21.15.133:0/884724755 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f80003520 con 0x7f6f80004d40 2024-03-07T06:53:31.237 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.235+0000 7f6f99e00640 1 --2- 172.21.15.133:0/884724755 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b3c10 0x7f6f800039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.236+0000 7f6f9a800640 1 --2- 172.21.15.133:0/884724755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f80004d40 0x7f6f800051c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009af0 tx=0x7f6f80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7cfa7ecc4f7492f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.236+0000 7f6f98a00640 1 -- 172.21.15.133:0/884724755 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94039070 con 0x7f6f80004d40 2024-03-07T06:53:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.236+0000 7f6f98a00640 1 -- 172.21.15.133:0/884724755 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9402c710 con 0x7f6f80004d40 2024-03-07T06:53:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.236+0000 7f6f98a00640 1 -- 172.21.15.133:0/884724755 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f94009690 con 0x7f6f80004d40 2024-03-07T06:53:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.237+0000 7f6fa0e00640 1 -- 172.21.15.133:0/884724755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f80004d40 msgr2=0x7f6f800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.237+0000 7f6fa0e00640 1 --2- 172.21.15.133:0/884724755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f80004d40 0x7f6f800051c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6f94009af0 tx=0x7f6f80005830 comp rx=0 tx=0).stop 2024-03-07T06:53:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.237+0000 7f6f9a800640 1 -- 172.21.15.133:0/884724755 reap_dead start 2024-03-07T06:53:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.237+0000 7f6fa0e00640 1 -- 172.21.15.133:0/884724755 shutdown_connections 2024-03-07T06:53:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.237+0000 7f6fa0e00640 1 -- 172.21.15.133:0/884724755 >> 172.21.15.133:0/884724755 conn(0x7f6f9c0b3120 msgr2=0x7f6f9c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.237+0000 7f6fa0e00640 1 -- 172.21.15.133:0/884724755 shutdown_connections 2024-03-07T06:53:31.238 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.237+0000 7f6fa0e00640 1 -- 172.21.15.133:0/884724755 wait complete. 2024-03-07T06:53:31.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.239+0000 7f6fa0e00640 1 Processor -- start 2024-03-07T06:53:31.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.239+0000 7f6fa0e00640 1 -- start start 2024-03-07T06:53:31.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.240+0000 7f6fa0e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f9c15f5e0 0x7f6f9c0b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.240+0000 7f6fa0e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b5f00 0x7f6f9c1f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.240+0000 7f6fa0e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f9c1f8550 0x7f6f9c0acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.240+0000 7f6fa0e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c072550 con 0x7f6f9c15f5e0 2024-03-07T06:53:31.240 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.240+0000 7f6fa0e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c072270 con 0x7f6f9c1f8550 2024-03-07T06:53:31.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.240+0000 7f6fa0e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9c071610 con 0x7f6f9c0b5f00 2024-03-07T06:53:31.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.240+0000 7f6f99e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f9c15f5e0 0x7f6f9c0b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.240+0000 7f6f99e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f9c15f5e0 0x7f6f9c0b59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35770/0 (socket says 172.21.15.133:35770) 2024-03-07T06:53:31.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.240+0000 7f6f99e00640 1 -- 172.21.15.133:0/3435973166 learned_addr learned my addr 172.21.15.133:0/3435973166 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:31.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.240+0000 7f6f99400640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b5f00 0x7f6f9c1f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.241+0000 7f6f99e00640 1 -- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b5f00 msgr2=0x7f6f9c1f8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:31.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.241+0000 7f6f9a800640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f9c1f8550 0x7f6f9c0acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.241+0000 7f6f99e00640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b5f00 0x7f6f9c1f8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.241+0000 7f6f99e00640 1 -- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f9c1f8550 msgr2=0x7f6f9c0acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:31.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.241+0000 7f6f99e00640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f9c1f8550 0x7f6f9c0acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.241+0000 7f6f99e00640 1 -- 172.21.15.133:0/3435973166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c1f8bf0 con 0x7f6f9c15f5e0 2024-03-07T06:53:31.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.241+0000 7f6f99400640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b5f00 0x7f6f9c1f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:31.241 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.241+0000 7f6f99e00640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f9c15f5e0 0x7f6f9c0b59c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c004bc0 tx=0x7f6f8c00b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:31.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.241+0000 7f6f92000640 1 -- 172.21.15.133:0/3435973166 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c019070 con 0x7f6f9c15f5e0 2024-03-07T06:53:31.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.241+0000 7f6f92000640 1 -- 172.21.15.133:0/3435973166 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8c0072d0 con 0x7f6f9c15f5e0 2024-03-07T06:53:31.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.241+0000 7f6f92000640 1 -- 172.21.15.133:0/3435973166 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c007680 con 0x7f6f9c15f5e0 2024-03-07T06:53:31.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.241+0000 7f6fa0e00640 1 -- 172.21.15.133:0/3435973166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f9c1f8ed0 con 0x7f6f9c15f5e0 2024-03-07T06:53:31.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.241+0000 7f6fa0e00640 1 -- 172.21.15.133:0/3435973166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f9c0af000 con 0x7f6f9c15f5e0 2024-03-07T06:53:31.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.243+0000 7f6fa0e00640 1 -- 172.21.15.133:0/3435973166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9c062c90 con 0x7f6f9c15f5e0 2024-03-07T06:53:31.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.243+0000 7f6f92000640 1 -- 172.21.15.133:0/3435973166 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6f8c007820 con 0x7f6f9c15f5e0 2024-03-07T06:53:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.244+0000 7f6f92000640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6f7c06b500 0x7f6f7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.244+0000 7f6f92000640 1 -- 172.21.15.133:0/3435973166 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f8c08a770 con 0x7f6f9c15f5e0 2024-03-07T06:53:31.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.245+0000 7f6f99400640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6f7c06b500 0x7f6f7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.246+0000 7f6f99400640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6f7c06b500 0x7f6f7c06d9c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0add30 tx=0x7f6f88008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:31.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.249+0000 7f6f92000640 1 -- 172.21.15.133:0/3435973166 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6f8c0595f0 con 0x7f6f9c15f5e0 2024-03-07T06:53:31.549 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.549+0000 7f6fa0e00640 1 -- 172.21.15.133:0/3435973166 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6f9c0b1530 con 0x7f6f9c15f5e0 2024-03-07T06:53:31.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.550+0000 7f6f92000640 1 -- 172.21.15.133:0/3435973166 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1432 (secure 0 0 0) 0x7f6f8c058ef0 con 0x7f6f9c15f5e0 2024-03-07T06:53:31.550 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:31.550 INFO:teuthology.orchestra.run.smithi133.stdout:{"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":162,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":8,"num_up_osds":8,"osd_up_since":1709794262,"num_in_osds":8,"osd_in_since":1709794253,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":256778240,"bytes_avail":772837335040,"bytes_total":773094113280,"write_bytes_sec":2252,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":27,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5580}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-03-07T06:53:12.513285+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-07T06:53:31.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.553+0000 7f6fa0e00640 1 -- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6f7c06b500 msgr2=0x7f6f7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:31.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.553+0000 7f6fa0e00640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6f7c06b500 0x7f6f7c06d9c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0add30 tx=0x7f6f88008040 comp rx=0 tx=0).stop 2024-03-07T06:53:31.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.553+0000 7f6fa0e00640 1 -- 172.21.15.133:0/3435973166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f9c15f5e0 msgr2=0x7f6f9c0b59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:31.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.553+0000 7f6fa0e00640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f9c15f5e0 0x7f6f9c0b59c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c004bc0 tx=0x7f6f8c00b560 comp rx=0 tx=0).stop 2024-03-07T06:53:31.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.553+0000 7f6fa0e00640 1 -- 172.21.15.133:0/3435973166 shutdown_connections 2024-03-07T06:53:31.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.553+0000 7f6fa0e00640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6f7c06b500 0x7f6f7c06d9c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.553+0000 7f6fa0e00640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f9c1f8550 0x7f6f9c0acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.553+0000 7f6fa0e00640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f9c0b5f00 0x7f6f9c1f8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.553+0000 7f6fa0e00640 1 --2- 172.21.15.133:0/3435973166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f9c15f5e0 0x7f6f9c0b59c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.553+0000 7f6fa0e00640 1 -- 172.21.15.133:0/3435973166 >> 172.21.15.133:0/3435973166 conn(0x7f6f9c0b3120 msgr2=0x7f6f9c06e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:31.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.554+0000 7f6fa0e00640 1 -- 172.21.15.133:0/3435973166 shutdown_connections 2024-03-07T06:53:31.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.554+0000 7f6fa0e00640 1 -- 172.21.15.133:0/3435973166 wait complete. 2024-03-07T06:53:31.572 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-07T06:53:31.572 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:53:31.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.772+0000 7fbbc9800640 1 Processor -- start 2024-03-07T06:53:31.772 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.772+0000 7fbbc9800640 1 -- start start 2024-03-07T06:53:31.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.774+0000 7fbbc9800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbbc40b0bd0 0x7fbbc4165570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.774+0000 7fbbc9800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc406f560 con 0x7fbbc415e050 2024-03-07T06:53:31.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.774+0000 7fbbc9800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc4072620 con 0x7fbbc40aef50 2024-03-07T06:53:31.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.774+0000 7fbbc9800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc406e7a0 con 0x7fbbc40b0bd0 2024-03-07T06:53:31.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.775+0000 7fbbc2a00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fbbc415e050 0x7fbbc40ce4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47460/0 (socket says 172.21.15.133:47460) 2024-03-07T06:53:31.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.775+0000 7fbbc2a00640 1 -- 172.21.15.133:0/1350696012 learned_addr learned my addr 172.21.15.133:0/1350696012 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:31.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.775+0000 7fbbc3400640 1 --2- 172.21.15.133:0/1350696012 >> v2:172.21.15.161:3300/0 conn(0x7fbbc40b0bd0 0x7fbbc4165570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.775+0000 7fbbc1600640 1 -- 172.21.15.133:0/1350696012 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126936197 0 0) 0x7fbbc406f560 con 0x7fbbc415e050 2024-03-07T06:53:31.776 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.775+0000 7fbbc1600640 1 -- 172.21.15.133:0/1350696012 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbba8003520 con 0x7fbbc415e050 2024-03-07T06:53:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc1600640 1 -- 172.21.15.133:0/1350696012 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3627001062 0 0) 0x7fbba8003520 con 0x7fbbc415e050 2024-03-07T06:53:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc1600640 1 -- 172.21.15.133:0/1350696012 >> v2:172.21.15.161:3300/0 conn(0x7fbbc40b0bd0 msgr2=0x7fbbc4165570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc1600640 1 --2- 172.21.15.133:0/1350696012 >> v2:172.21.15.161:3300/0 conn(0x7fbbc40b0bd0 0x7fbbc4165570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc1600640 1 -- 172.21.15.133:0/1350696012 >> v1:172.21.15.161:6789/0 conn(0x7fbbc40aef50 legacy=0x7fbbc40b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc1600640 1 -- 172.21.15.133:0/1350696012 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc406ea80 con 0x7fbbc415e050 2024-03-07T06:53:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc1600640 1 -- 172.21.15.133:0/1350696012 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fbbb4001670 con 0x7fbbc415e050 2024-03-07T06:53:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc1600640 1 -- 172.21.15.133:0/1350696012 >> v1:172.21.15.133:6789/0 conn(0x7fbbc415e050 legacy=0x7fbbc40ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc1600640 1 --2- 172.21.15.133:0/1350696012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba80039b0 0x7fbba8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc1600640 1 --2- 172.21.15.133:0/1350696012 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbba8004800 0x7fbba8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc1600640 1 --2- 172.21.15.133:0/1350696012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbba80051c0 0x7fbba800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc1600640 1 -- 172.21.15.133:0/1350696012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc406e7a0 con 0x7fbba80039b0 2024-03-07T06:53:31.777 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc1600640 1 -- 172.21.15.133:0/1350696012 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba800c160 con 0x7fbba80051c0 2024-03-07T06:53:31.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc1600640 1 -- 172.21.15.133:0/1350696012 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba800c360 con 0x7fbba8004800 2024-03-07T06:53:31.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc2a00640 1 --2- 172.21.15.133:0/1350696012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba80039b0 0x7fbba8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc2a00640 1 -- 172.21.15.133:0/1350696012 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbba8004800 msgr2=0x7fbba8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:31.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc2a00640 1 --2- 172.21.15.133:0/1350696012 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbba8004800 0x7fbba8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc2a00640 1 -- 172.21.15.133:0/1350696012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbba80051c0 msgr2=0x7fbba800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:31.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc2a00640 1 --2- 172.21.15.133:0/1350696012 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbba80051c0 0x7fbba800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc3400640 1 -- 172.21.15.133:0/1350696012 reap_dead start 2024-03-07T06:53:31.778 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.776+0000 7fbbc2a00640 1 -- 172.21.15.133:0/1350696012 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba8003520 con 0x7fbba80039b0 2024-03-07T06:53:31.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.778+0000 7fbbc2a00640 1 --2- 172.21.15.133:0/1350696012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba80039b0 0x7fbba8003dd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4007fb0 tx=0x7fbba8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5e831dee6db648 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:31.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.778+0000 7fbbc1600640 1 -- 172.21.15.133:0/1350696012 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4007d30 con 0x7fbba80039b0 2024-03-07T06:53:31.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.778+0000 7fbbc1600640 1 -- 172.21.15.133:0/1350696012 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb4032070 con 0x7fbba80039b0 2024-03-07T06:53:31.779 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.778+0000 7fbbc1600640 1 -- 172.21.15.133:0/1350696012 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb40058d0 con 0x7fbba80039b0 2024-03-07T06:53:31.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.778+0000 7fbbc9800640 1 -- 172.21.15.133:0/1350696012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba80039b0 msgr2=0x7fbba8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:31.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.778+0000 7fbbc9800640 1 --2- 172.21.15.133:0/1350696012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba80039b0 0x7fbba8003dd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4007fb0 tx=0x7fbba8005830 comp rx=0 tx=0).stop 2024-03-07T06:53:31.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.778+0000 7fbbc9800640 1 -- 172.21.15.133:0/1350696012 shutdown_connections 2024-03-07T06:53:31.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.778+0000 7fbbc9800640 1 --2- 172.21.15.133:0/1350696012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba80039b0 0x7fbba8003dd0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.778+0000 7fbbc9800640 1 -- 172.21.15.133:0/1350696012 >> 172.21.15.133:0/1350696012 conn(0x7fbbc40b3120 msgr2=0x7fbbc40b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:31.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.779+0000 7fbbc9800640 1 -- 172.21.15.133:0/1350696012 shutdown_connections 2024-03-07T06:53:31.780 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.779+0000 7fbbc9800640 1 -- 172.21.15.133:0/1350696012 wait complete. 2024-03-07T06:53:31.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.781+0000 7fbbc9800640 1 Processor -- start 2024-03-07T06:53:31.781 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.781+0000 7fbbc9800640 1 -- start start 2024-03-07T06:53:31.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.782+0000 7fbbc9800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba80039b0 0x7fbbc415d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.783 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.782+0000 7fbbc9800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc41e31d0 0x7fbbc4146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc9800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbc40b5680 0x7fbbc40b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc9800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc40cece0 con 0x7fbba80039b0 2024-03-07T06:53:31.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc9800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc406f040 con 0x7fbbc41e31d0 2024-03-07T06:53:31.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc9800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc40cee60 con 0x7fbbc40b5680 2024-03-07T06:53:31.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba80039b0 0x7fbbc415d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba80039b0 0x7fbbc415d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35792/0 (socket says 172.21.15.133:35792) 2024-03-07T06:53:31.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc2a00640 1 -- 172.21.15.133:0/2225726269 learned_addr learned my addr 172.21.15.133:0/2225726269 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:31.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc3400640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbc40b5680 0x7fbbc40b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc2000640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc41e31d0 0x7fbbc4146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc2a00640 1 -- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbc40b5680 msgr2=0x7fbbc40b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:31.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc2a00640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbc40b5680 0x7fbbc40b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc2a00640 1 -- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc41e31d0 msgr2=0x7fbbc4146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:31.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc2a00640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc41e31d0 0x7fbbc4146920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:31.784 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc2a00640 1 -- 172.21.15.133:0/2225726269 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc415d090 con 0x7fbba80039b0 2024-03-07T06:53:31.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc3400640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbc40b5680 0x7fbbc40b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:31.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.783+0000 7fbbc2000640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc41e31d0 0x7fbbc4146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:31.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.784+0000 7fbbc2a00640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba80039b0 0x7fbbc415d420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4000f60 tx=0x7fbbb4030070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:31.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.784+0000 7fbbbaa00640 1 -- 172.21.15.133:0/2225726269 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb4004cb0 con 0x7fbba80039b0 2024-03-07T06:53:31.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.784+0000 7fbbbaa00640 1 -- 172.21.15.133:0/2225726269 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb4004e50 con 0x7fbba80039b0 2024-03-07T06:53:31.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.784+0000 7fbbc9800640 1 -- 172.21.15.133:0/2225726269 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc4146e60 con 0x7fbba80039b0 2024-03-07T06:53:31.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.784+0000 7fbbbaa00640 1 -- 172.21.15.133:0/2225726269 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb40072b0 con 0x7fbba80039b0 2024-03-07T06:53:31.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.784+0000 7fbbc9800640 1 -- 172.21.15.133:0/2225726269 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc41e3a70 con 0x7fbba80039b0 2024-03-07T06:53:31.785 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.785+0000 7fbbc9800640 1 -- 172.21.15.133:0/2225726269 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc4062c90 con 0x7fbba80039b0 2024-03-07T06:53:31.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.786+0000 7fbbbaa00640 1 -- 172.21.15.133:0/2225726269 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbbb4007450 con 0x7fbba80039b0 2024-03-07T06:53:31.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.787+0000 7fbbbaa00640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbba406b500 0x7fbba406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:31.786 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.787+0000 7fbbbaa00640 1 -- 172.21.15.133:0/2225726269 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbb40ad650 con 0x7fbba80039b0 2024-03-07T06:53:31.787 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.788+0000 7fbbc2000640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbba406b500 0x7fbba406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:31.788 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.789+0000 7fbbc2000640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbba406b500 0x7fbba406d9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbbc406e770 tx=0x7fbbb0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:31.791 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.791+0000 7fbbbaa00640 1 -- 172.21.15.133:0/2225726269 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbbc4146e60 con 0x7fbba80039b0 2024-03-07T06:53:31.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:31.982+0000 7fbbc9800640 1 -- 172.21.15.133:0/2225726269 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbbc40731c0 con 0x7fbba406b500 2024-03-07T06:53:32.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:32.000+0000 7fbbbaa00640 1 -- 172.21.15.133:0/2225726269 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251960 (secure 0 0 0) 0x7fbbc40731c0 con 0x7fbba406b500 2024-03-07T06:53:32.004 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:32.005 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:53:32.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:32.006+0000 7fbbc9800640 1 -- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbba406b500 msgr2=0x7fbba406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:32.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:32.006+0000 7fbbc9800640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbba406b500 0x7fbba406d9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbbc406e770 tx=0x7fbbb0009040 comp rx=0 tx=0).stop 2024-03-07T06:53:32.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:32.006+0000 7fbbc9800640 1 -- 172.21.15.133:0/2225726269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba80039b0 msgr2=0x7fbbc415d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:32.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:32.006+0000 7fbbc9800640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba80039b0 0x7fbbc415d420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4000f60 tx=0x7fbbb4030070 comp rx=0 tx=0).stop 2024-03-07T06:53:32.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:32.006+0000 7fbbc9800640 1 -- 172.21.15.133:0/2225726269 shutdown_connections 2024-03-07T06:53:32.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:32.006+0000 7fbbc9800640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbc40b5680 0x7fbbc40b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:32.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:32.006+0000 7fbbc9800640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbba406b500 0x7fbba406d9c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:32.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:32.006+0000 7fbbc9800640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbc41e31d0 0x7fbbc4146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:32.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:32.006+0000 7fbbc9800640 1 --2- 172.21.15.133:0/2225726269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbba80039b0 0x7fbbc415d420 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:32.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:32.006+0000 7fbbc9800640 1 -- 172.21.15.133:0/2225726269 >> 172.21.15.133:0/2225726269 conn(0x7fbbc40b3120 msgr2=0x7fbbc40740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:32.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:32.007+0000 7fbbc9800640 1 -- 172.21.15.133:0/2225726269 shutdown_connections 2024-03-07T06:53:32.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:32.007+0000 7fbbc9800640 1 -- 172.21.15.133:0/2225726269 wait complete. 2024-03-07T06:53:32.027 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-03-07T06:53:30.518666+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250760,"kb_used_data":5448,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754723960,"statfs":{"total":773094113280,"available":772837335040,"internally_reserved":0,"allocated":5578752,"data_stored":2943908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.363674"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194303+0000","last_change":"2024-03-07T06:53:25.202958+0000","last_active":"2024-03-07T06:53:26.194303+0000","last_peered":"2024-03-07T06:53:26.194303+0000","last_clean":"2024-03-07T06:53:26.194303+0000","last_became_active":"2024-03-07T06:53:25.202689+0000","last_became_peered":"2024-03-07T06:53:25.202689+0000","last_unstale":"2024-03-07T06:53:26.194303+0000","last_undegraded":"2024-03-07T06:53:26.194303+0000","last_fullsized":"2024-03-07T06:53:26.194303+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:51:54.139595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192936+0000","last_change":"2024-03-07T06:53:25.205060+0000","last_active":"2024-03-07T06:53:26.192936+0000","last_peered":"2024-03-07T06:53:26.192936+0000","last_clean":"2024-03-07T06:53:26.192936+0000","last_became_active":"2024-03-07T06:53:25.204801+0000","last_became_peered":"2024-03-07T06:53:25.204801+0000","last_unstale":"2024-03-07T06:53:26.192936+0000","last_undegraded":"2024-03-07T06:53:26.192936+0000","last_fullsized":"2024-03-07T06:53:26.192936+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:14.520156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796955+0000","last_change":"2024-03-07T06:53:25.189468+0000","last_active":"2024-03-07T06:53:26.796955+0000","last_peered":"2024-03-07T06:53:26.796955+0000","last_clean":"2024-03-07T06:53:26.796955+0000","last_became_active":"2024-03-07T06:53:25.189178+0000","last_became_peered":"2024-03-07T06:53:25.189178+0000","last_unstale":"2024-03-07T06:53:26.796955+0000","last_undegraded":"2024-03-07T06:53:26.796955+0000","last_fullsized":"2024-03-07T06:53:26.796955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:04.729008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556013+0000","last_change":"2024-03-07T06:53:25.193381+0000","last_active":"2024-03-07T06:53:26.556013+0000","last_peered":"2024-03-07T06:53:26.556013+0000","last_clean":"2024-03-07T06:53:26.556013+0000","last_became_active":"2024-03-07T06:53:25.193079+0000","last_became_peered":"2024-03-07T06:53:25.193079+0000","last_unstale":"2024-03-07T06:53:26.556013+0000","last_undegraded":"2024-03-07T06:53:26.556013+0000","last_fullsized":"2024-03-07T06:53:26.556013+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:14.520156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193500+0000","last_change":"2024-03-07T06:53:25.202836+0000","last_active":"2024-03-07T06:53:26.193500+0000","last_peered":"2024-03-07T06:53:26.193500+0000","last_clean":"2024-03-07T06:53:26.193500+0000","last_became_active":"2024-03-07T06:53:25.202463+0000","last_became_peered":"2024-03-07T06:53:25.202463+0000","last_unstale":"2024-03-07T06:53:26.193500+0000","last_undegraded":"2024-03-07T06:53:26.193500+0000","last_fullsized":"2024-03-07T06:53:26.193500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:44.090672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568638+0000","last_change":"2024-03-07T06:53:25.203678+0000","last_active":"2024-03-07T06:53:26.568638+0000","last_peered":"2024-03-07T06:53:26.568638+0000","last_clean":"2024-03-07T06:53:26.568638+0000","last_became_active":"2024-03-07T06:53:25.198295+0000","last_became_peered":"2024-03-07T06:53:25.198295+0000","last_unstale":"2024-03-07T06:53:26.568638+0000","last_undegraded":"2024-03-07T06:53:26.568638+0000","last_fullsized":"2024-03-07T06:53:26.568638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:49:12.380843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.190260+0000","last_change":"2024-03-07T06:53:25.193112+0000","last_active":"2024-03-07T06:53:26.190260+0000","last_peered":"2024-03-07T06:53:26.190260+0000","last_clean":"2024-03-07T06:53:26.190260+0000","last_became_active":"2024-03-07T06:53:25.192861+0000","last_became_peered":"2024-03-07T06:53:25.192861+0000","last_unstale":"2024-03-07T06:53:26.190260+0000","last_undegraded":"2024-03-07T06:53:26.190260+0000","last_fullsized":"2024-03-07T06:53:26.190260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:08:50.099907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555336+0000","last_change":"2024-03-07T06:53:25.200927+0000","last_active":"2024-03-07T06:53:26.555336+0000","last_peered":"2024-03-07T06:53:26.555336+0000","last_clean":"2024-03-07T06:53:26.555336+0000","last_became_active":"2024-03-07T06:53:25.200535+0000","last_became_peered":"2024-03-07T06:53:25.200535+0000","last_unstale":"2024-03-07T06:53:26.555336+0000","last_undegraded":"2024-03-07T06:53:26.555336+0000","last_fullsized":"2024-03-07T06:53:26.555336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:55.274941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556694+0000","last_change":"2024-03-07T06:53:25.179342+0000","last_active":"2024-03-07T06:53:26.556694+0000","last_peered":"2024-03-07T06:53:26.556694+0000","last_clean":"2024-03-07T06:53:26.556694+0000","last_became_active":"2024-03-07T06:53:25.179031+0000","last_became_peered":"2024-03-07T06:53:25.179031+0000","last_unstale":"2024-03-07T06:53:26.556694+0000","last_undegraded":"2024-03-07T06:53:26.556694+0000","last_fullsized":"2024-03-07T06:53:26.556694+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:54:32.067212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556561+0000","last_change":"2024-03-07T06:53:25.199475+0000","last_active":"2024-03-07T06:53:26.556561+0000","last_peered":"2024-03-07T06:53:26.556561+0000","last_clean":"2024-03-07T06:53:26.556561+0000","last_became_active":"2024-03-07T06:53:25.193080+0000","last_became_peered":"2024-03-07T06:53:25.193080+0000","last_unstale":"2024-03-07T06:53:26.556561+0000","last_undegraded":"2024-03-07T06:53:26.556561+0000","last_fullsized":"2024-03-07T06:53:26.556561+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:08:50.099907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555759+0000","last_change":"2024-03-07T06:53:25.177187+0000","last_active":"2024-03-07T06:53:26.555759+0000","last_peered":"2024-03-07T06:53:26.555759+0000","last_clean":"2024-03-07T06:53:26.555759+0000","last_became_active":"2024-03-07T06:53:25.176924+0000","last_became_peered":"2024-03-07T06:53:25.176924+0000","last_unstale":"2024-03-07T06:53:26.555759+0000","last_undegraded":"2024-03-07T06:53:26.555759+0000","last_fullsized":"2024-03-07T06:53:26.555759+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:04.729008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193384+0000","last_change":"2024-03-07T06:53:25.197797+0000","last_active":"2024-03-07T06:53:26.193384+0000","last_peered":"2024-03-07T06:53:26.193384+0000","last_clean":"2024-03-07T06:53:26.193384+0000","last_became_active":"2024-03-07T06:53:25.197396+0000","last_became_peered":"2024-03-07T06:53:25.197396+0000","last_unstale":"2024-03-07T06:53:26.193384+0000","last_undegraded":"2024-03-07T06:53:26.193384+0000","last_fullsized":"2024-03-07T06:53:26.193384+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:49:12.962971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797908+0000","last_change":"2024-03-07T06:53:25.198602+0000","last_active":"2024-03-07T06:53:26.797908+0000","last_peered":"2024-03-07T06:53:26.797908+0000","last_clean":"2024-03-07T06:53:26.797908+0000","last_became_active":"2024-03-07T06:53:25.197653+0000","last_became_peered":"2024-03-07T06:53:25.197653+0000","last_unstale":"2024-03-07T06:53:26.797908+0000","last_undegraded":"2024-03-07T06:53:26.797908+0000","last_fullsized":"2024-03-07T06:53:26.797908+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:03:09.371531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555920+0000","last_change":"2024-03-07T06:53:25.180030+0000","last_active":"2024-03-07T06:53:26.555920+0000","last_peered":"2024-03-07T06:53:26.555920+0000","last_clean":"2024-03-07T06:53:26.555920+0000","last_became_active":"2024-03-07T06:53:25.178772+0000","last_became_peered":"2024-03-07T06:53:25.178772+0000","last_unstale":"2024-03-07T06:53:26.555920+0000","last_undegraded":"2024-03-07T06:53:26.555920+0000","last_fullsized":"2024-03-07T06:53:26.555920+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:46:45.216443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568373+0000","last_change":"2024-03-07T06:53:25.197311+0000","last_active":"2024-03-07T06:53:26.568373+0000","last_peered":"2024-03-07T06:53:26.568373+0000","last_clean":"2024-03-07T06:53:26.568373+0000","last_became_active":"2024-03-07T06:53:25.196985+0000","last_became_peered":"2024-03-07T06:53:25.196985+0000","last_unstale":"2024-03-07T06:53:26.568373+0000","last_undegraded":"2024-03-07T06:53:26.568373+0000","last_fullsized":"2024-03-07T06:53:26.568373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:05:54.805828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192024+0000","last_change":"2024-03-07T06:53:25.204127+0000","last_active":"2024-03-07T06:53:26.192024+0000","last_peered":"2024-03-07T06:53:26.192024+0000","last_clean":"2024-03-07T06:53:26.192024+0000","last_became_active":"2024-03-07T06:53:25.203266+0000","last_became_peered":"2024-03-07T06:53:25.203266+0000","last_unstale":"2024-03-07T06:53:26.192024+0000","last_undegraded":"2024-03-07T06:53:26.192024+0000","last_fullsized":"2024-03-07T06:53:26.192024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:13:15.845709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797626+0000","last_change":"2024-03-07T06:53:25.194438+0000","last_active":"2024-03-07T06:53:26.797626+0000","last_peered":"2024-03-07T06:53:26.797626+0000","last_clean":"2024-03-07T06:53:26.797626+0000","last_became_active":"2024-03-07T06:53:25.194130+0000","last_became_peered":"2024-03-07T06:53:25.194130+0000","last_unstale":"2024-03-07T06:53:26.797626+0000","last_undegraded":"2024-03-07T06:53:26.797626+0000","last_fullsized":"2024-03-07T06:53:26.797626+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:19:59.756501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797261+0000","last_change":"2024-03-07T06:53:25.206131+0000","last_active":"2024-03-07T06:53:26.797261+0000","last_peered":"2024-03-07T06:53:26.797261+0000","last_clean":"2024-03-07T06:53:26.797261+0000","last_became_active":"2024-03-07T06:53:25.205578+0000","last_became_peered":"2024-03-07T06:53:25.205578+0000","last_unstale":"2024-03-07T06:53:26.797261+0000","last_undegraded":"2024-03-07T06:53:26.797261+0000","last_fullsized":"2024-03-07T06:53:26.797261+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:44.090672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195955+0000","last_change":"2024-03-07T06:53:25.188986+0000","last_active":"2024-03-07T06:53:26.195955+0000","last_peered":"2024-03-07T06:53:26.195955+0000","last_clean":"2024-03-07T06:53:26.195955+0000","last_became_active":"2024-03-07T06:53:25.188678+0000","last_became_peered":"2024-03-07T06:53:25.188678+0000","last_unstale":"2024-03-07T06:53:26.195955+0000","last_undegraded":"2024-03-07T06:53:26.195955+0000","last_fullsized":"2024-03-07T06:53:26.195955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:32.048437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798581+0000","last_change":"2024-03-07T06:53:25.204686+0000","last_active":"2024-03-07T06:53:26.798581+0000","last_peered":"2024-03-07T06:53:26.798581+0000","last_clean":"2024-03-07T06:53:26.798581+0000","last_became_active":"2024-03-07T06:53:25.203607+0000","last_became_peered":"2024-03-07T06:53:25.203607+0000","last_unstale":"2024-03-07T06:53:26.798581+0000","last_undegraded":"2024-03-07T06:53:26.798581+0000","last_fullsized":"2024-03-07T06:53:26.798581+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:27:56.445897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568794+0000","last_change":"2024-03-07T06:53:25.190607+0000","last_active":"2024-03-07T06:53:26.568794+0000","last_peered":"2024-03-07T06:53:26.568794+0000","last_clean":"2024-03-07T06:53:26.568794+0000","last_became_active":"2024-03-07T06:53:25.190083+0000","last_became_peered":"2024-03-07T06:53:25.190083+0000","last_unstale":"2024-03-07T06:53:26.568794+0000","last_undegraded":"2024-03-07T06:53:26.568794+0000","last_fullsized":"2024-03-07T06:53:26.568794+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:47:02.991917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555545+0000","last_change":"2024-03-07T06:53:25.200334+0000","last_active":"2024-03-07T06:53:26.555545+0000","last_peered":"2024-03-07T06:53:26.555545+0000","last_clean":"2024-03-07T06:53:26.555545+0000","last_became_active":"2024-03-07T06:53:25.200096+0000","last_became_peered":"2024-03-07T06:53:25.200096+0000","last_unstale":"2024-03-07T06:53:26.555545+0000","last_undegraded":"2024-03-07T06:53:26.555545+0000","last_fullsized":"2024-03-07T06:53:26.555545+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:33:47.684117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193801+0000","last_change":"2024-03-07T06:53:25.202494+0000","last_active":"2024-03-07T06:53:26.193801+0000","last_peered":"2024-03-07T06:53:26.193801+0000","last_clean":"2024-03-07T06:53:26.193801+0000","last_became_active":"2024-03-07T06:53:25.202185+0000","last_became_peered":"2024-03-07T06:53:25.202185+0000","last_unstale":"2024-03-07T06:53:26.193801+0000","last_undegraded":"2024-03-07T06:53:26.193801+0000","last_fullsized":"2024-03-07T06:53:26.193801+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:36.565865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192462+0000","last_change":"2024-03-07T06:53:25.203558+0000","last_active":"2024-03-07T06:53:26.192462+0000","last_peered":"2024-03-07T06:53:26.192462+0000","last_clean":"2024-03-07T06:53:26.192462+0000","last_became_active":"2024-03-07T06:53:25.202854+0000","last_became_peered":"2024-03-07T06:53:25.202854+0000","last_unstale":"2024-03-07T06:53:26.192462+0000","last_undegraded":"2024-03-07T06:53:26.192462+0000","last_fullsized":"2024-03-07T06:53:26.192462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:34:02.638312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195196+0000","last_change":"2024-03-07T06:53:25.193176+0000","last_active":"2024-03-07T06:53:26.195196+0000","last_peered":"2024-03-07T06:53:26.195196+0000","last_clean":"2024-03-07T06:53:26.195196+0000","last_became_active":"2024-03-07T06:53:25.192881+0000","last_became_peered":"2024-03-07T06:53:25.192881+0000","last_unstale":"2024-03-07T06:53:26.195196+0000","last_undegraded":"2024-03-07T06:53:26.195196+0000","last_fullsized":"2024-03-07T06:53:26.195196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:07:11.833897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798298+0000","last_change":"2024-03-07T06:53:25.205870+0000","last_active":"2024-03-07T06:53:26.798298+0000","last_peered":"2024-03-07T06:53:26.798298+0000","last_clean":"2024-03-07T06:53:26.798298+0000","last_became_active":"2024-03-07T06:53:25.205595+0000","last_became_peered":"2024-03-07T06:53:25.205595+0000","last_unstale":"2024-03-07T06:53:26.798298+0000","last_undegraded":"2024-03-07T06:53:26.798298+0000","last_fullsized":"2024-03-07T06:53:26.798298+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:13:01.272121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797104+0000","last_change":"2024-03-07T06:53:25.187957+0000","last_active":"2024-03-07T06:53:26.797104+0000","last_peered":"2024-03-07T06:53:26.797104+0000","last_clean":"2024-03-07T06:53:26.797104+0000","last_became_active":"2024-03-07T06:53:25.184716+0000","last_became_peered":"2024-03-07T06:53:25.184716+0000","last_unstale":"2024-03-07T06:53:26.797104+0000","last_undegraded":"2024-03-07T06:53:26.797104+0000","last_fullsized":"2024-03-07T06:53:26.797104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:14.520156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194874+0000","last_change":"2024-03-07T06:53:25.194725+0000","last_active":"2024-03-07T06:53:26.194874+0000","last_peered":"2024-03-07T06:53:26.194874+0000","last_clean":"2024-03-07T06:53:26.194874+0000","last_became_active":"2024-03-07T06:53:25.194434+0000","last_became_peered":"2024-03-07T06:53:25.194434+0000","last_unstale":"2024-03-07T06:53:26.194874+0000","last_undegraded":"2024-03-07T06:53:26.194874+0000","last_fullsized":"2024-03-07T06:53:26.194874+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:08:50.099907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195182+0000","last_change":"2024-03-07T06:53:25.188058+0000","last_active":"2024-03-07T06:53:26.195182+0000","last_peered":"2024-03-07T06:53:26.195182+0000","last_clean":"2024-03-07T06:53:26.195182+0000","last_became_active":"2024-03-07T06:53:25.187778+0000","last_became_peered":"2024-03-07T06:53:25.187778+0000","last_unstale":"2024-03-07T06:53:26.195182+0000","last_undegraded":"2024-03-07T06:53:26.195182+0000","last_fullsized":"2024-03-07T06:53:26.195182+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:41:07.796024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796976+0000","last_change":"2024-03-07T06:53:25.203241+0000","last_active":"2024-03-07T06:53:26.796976+0000","last_peered":"2024-03-07T06:53:26.796976+0000","last_clean":"2024-03-07T06:53:26.796976+0000","last_became_active":"2024-03-07T06:53:25.202804+0000","last_became_peered":"2024-03-07T06:53:25.202804+0000","last_unstale":"2024-03-07T06:53:26.796976+0000","last_undegraded":"2024-03-07T06:53:26.796976+0000","last_fullsized":"2024-03-07T06:53:26.796976+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:46:45.216443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568377+0000","last_change":"2024-03-07T06:53:25.193410+0000","last_active":"2024-03-07T06:53:26.568377+0000","last_peered":"2024-03-07T06:53:26.568377+0000","last_clean":"2024-03-07T06:53:26.568377+0000","last_became_active":"2024-03-07T06:53:25.193130+0000","last_became_peered":"2024-03-07T06:53:25.193130+0000","last_unstale":"2024-03-07T06:53:26.568377+0000","last_undegraded":"2024-03-07T06:53:26.568377+0000","last_fullsized":"2024-03-07T06:53:26.568377+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:17:12.387823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797108+0000","last_change":"2024-03-07T06:53:25.181460+0000","last_active":"2024-03-07T06:53:26.797108+0000","last_peered":"2024-03-07T06:53:26.797108+0000","last_clean":"2024-03-07T06:53:26.797108+0000","last_became_active":"2024-03-07T06:53:25.181176+0000","last_became_peered":"2024-03-07T06:53:25.181176+0000","last_unstale":"2024-03-07T06:53:26.797108+0000","last_undegraded":"2024-03-07T06:53:26.797108+0000","last_fullsized":"2024-03-07T06:53:26.797108+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:31:32.369129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797278+0000","last_change":"2024-03-07T06:53:25.185869+0000","last_active":"2024-03-07T06:53:26.797278+0000","last_peered":"2024-03-07T06:53:26.797278+0000","last_clean":"2024-03-07T06:53:26.797278+0000","last_became_active":"2024-03-07T06:53:25.185574+0000","last_became_peered":"2024-03-07T06:53:25.185574+0000","last_unstale":"2024-03-07T06:53:26.797278+0000","last_undegraded":"2024-03-07T06:53:26.797278+0000","last_fullsized":"2024-03-07T06:53:26.797278+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:32.048437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569830+0000","last_change":"2024-03-07T06:53:24.190584+0000","last_active":"2024-03-07T06:53:26.569830+0000","last_peered":"2024-03-07T06:53:26.569830+0000","last_clean":"2024-03-07T06:53:26.569830+0000","last_became_active":"2024-03-07T06:53:24.190262+0000","last_became_peered":"2024-03-07T06:53:24.190262+0000","last_unstale":"2024-03-07T06:53:26.569830+0000","last_undegraded":"2024-03-07T06:53:26.569830+0000","last_fullsized":"2024-03-07T06:53:26.569830+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:53:36.898081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.196313+0000","last_change":"2024-03-07T06:53:25.195388+0000","last_active":"2024-03-07T06:53:26.196313+0000","last_peered":"2024-03-07T06:53:26.196313+0000","last_clean":"2024-03-07T06:53:26.196313+0000","last_became_active":"2024-03-07T06:53:25.192645+0000","last_became_peered":"2024-03-07T06:53:25.192645+0000","last_unstale":"2024-03-07T06:53:26.196313+0000","last_undegraded":"2024-03-07T06:53:26.196313+0000","last_fullsized":"2024-03-07T06:53:26.196313+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:12:49.521040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797418+0000","last_change":"2024-03-07T06:53:24.190898+0000","last_active":"2024-03-07T06:53:26.797418+0000","last_peered":"2024-03-07T06:53:26.797418+0000","last_clean":"2024-03-07T06:53:26.797418+0000","last_became_active":"2024-03-07T06:53:24.190216+0000","last_became_peered":"2024-03-07T06:53:24.190216+0000","last_unstale":"2024-03-07T06:53:26.797418+0000","last_undegraded":"2024-03-07T06:53:26.797418+0000","last_fullsized":"2024-03-07T06:53:26.797418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:14:52.563314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193786+0000","last_change":"2024-03-07T06:53:25.206962+0000","last_active":"2024-03-07T06:53:26.193786+0000","last_peered":"2024-03-07T06:53:26.193786+0000","last_clean":"2024-03-07T06:53:26.193786+0000","last_became_active":"2024-03-07T06:53:25.206428+0000","last_became_peered":"2024-03-07T06:53:25.206428+0000","last_unstale":"2024-03-07T06:53:26.193786+0000","last_undegraded":"2024-03-07T06:53:26.193786+0000","last_fullsized":"2024-03-07T06:53:26.193786+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:19:59.756501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798392+0000","last_change":"2024-03-07T06:53:24.189931+0000","last_active":"2024-03-07T06:53:26.798392+0000","last_peered":"2024-03-07T06:53:26.798392+0000","last_clean":"2024-03-07T06:53:26.798392+0000","last_became_active":"2024-03-07T06:53:24.189610+0000","last_became_peered":"2024-03-07T06:53:24.189610+0000","last_unstale":"2024-03-07T06:53:26.798392+0000","last_undegraded":"2024-03-07T06:53:26.798392+0000","last_fullsized":"2024-03-07T06:53:26.798392+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:36:40.695222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798250+0000","last_change":"2024-03-07T06:53:25.190984+0000","last_active":"2024-03-07T06:53:26.798250+0000","last_peered":"2024-03-07T06:53:26.798250+0000","last_clean":"2024-03-07T06:53:26.798250+0000","last_became_active":"2024-03-07T06:53:25.190624+0000","last_became_peered":"2024-03-07T06:53:25.190624+0000","last_unstale":"2024-03-07T06:53:26.798250+0000","last_undegraded":"2024-03-07T06:53:26.798250+0000","last_fullsized":"2024-03-07T06:53:26.798250+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:36:14.943170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.189900+0000","last_change":"2024-03-07T06:53:24.192196+0000","last_active":"2024-03-07T06:53:26.189900+0000","last_peered":"2024-03-07T06:53:26.189900+0000","last_clean":"2024-03-07T06:53:26.189900+0000","last_became_active":"2024-03-07T06:53:24.191698+0000","last_became_peered":"2024-03-07T06:53:24.191698+0000","last_unstale":"2024-03-07T06:53:26.189900+0000","last_undegraded":"2024-03-07T06:53:26.189900+0000","last_fullsized":"2024-03-07T06:53:26.189900+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:45:09.141933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796861+0000","last_change":"2024-03-07T06:53:25.193044+0000","last_active":"2024-03-07T06:53:26.796861+0000","last_peered":"2024-03-07T06:53:26.796861+0000","last_clean":"2024-03-07T06:53:26.796861+0000","last_became_active":"2024-03-07T06:53:25.192696+0000","last_became_peered":"2024-03-07T06:53:25.192696+0000","last_unstale":"2024-03-07T06:53:26.796861+0000","last_undegraded":"2024-03-07T06:53:26.796861+0000","last_fullsized":"2024-03-07T06:53:26.796861+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:45:10.156912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569421+0000","last_change":"2024-03-07T06:53:24.180332+0000","last_active":"2024-03-07T06:53:26.569421+0000","last_peered":"2024-03-07T06:53:26.569421+0000","last_clean":"2024-03-07T06:53:26.569421+0000","last_became_active":"2024-03-07T06:53:24.179783+0000","last_became_peered":"2024-03-07T06:53:24.179783+0000","last_unstale":"2024-03-07T06:53:26.569421+0000","last_undegraded":"2024-03-07T06:53:26.569421+0000","last_fullsized":"2024-03-07T06:53:26.569421+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:03.714029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556363+0000","last_change":"2024-03-07T06:53:25.193219+0000","last_active":"2024-03-07T06:53:26.556363+0000","last_peered":"2024-03-07T06:53:26.556363+0000","last_clean":"2024-03-07T06:53:26.556363+0000","last_became_active":"2024-03-07T06:53:25.192945+0000","last_became_peered":"2024-03-07T06:53:25.192945+0000","last_unstale":"2024-03-07T06:53:26.556363+0000","last_undegraded":"2024-03-07T06:53:26.556363+0000","last_fullsized":"2024-03-07T06:53:26.556363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:44.090672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797025+0000","last_change":"2024-03-07T06:53:24.186564+0000","last_active":"2024-03-07T06:53:26.797025+0000","last_peered":"2024-03-07T06:53:26.797025+0000","last_clean":"2024-03-07T06:53:26.797025+0000","last_became_active":"2024-03-07T06:53:24.184717+0000","last_became_peered":"2024-03-07T06:53:24.184717+0000","last_unstale":"2024-03-07T06:53:26.797025+0000","last_undegraded":"2024-03-07T06:53:26.797025+0000","last_fullsized":"2024-03-07T06:53:26.797025+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:07:10.818918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555572+0000","last_change":"2024-03-07T06:53:25.196813+0000","last_active":"2024-03-07T06:53:26.555572+0000","last_peered":"2024-03-07T06:53:26.555572+0000","last_clean":"2024-03-07T06:53:26.555572+0000","last_became_active":"2024-03-07T06:53:25.195979+0000","last_became_peered":"2024-03-07T06:53:25.195979+0000","last_unstale":"2024-03-07T06:53:26.555572+0000","last_undegraded":"2024-03-07T06:53:26.555572+0000","last_fullsized":"2024-03-07T06:53:26.555572+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:53:37.913060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193612+0000","last_change":"2024-03-07T06:53:24.187243+0000","last_active":"2024-03-07T06:53:26.193612+0000","last_peered":"2024-03-07T06:53:26.193612+0000","last_clean":"2024-03-07T06:53:26.193612+0000","last_became_active":"2024-03-07T06:53:24.186379+0000","last_became_peered":"2024-03-07T06:53:24.186379+0000","last_unstale":"2024-03-07T06:53:26.193612+0000","last_undegraded":"2024-03-07T06:53:26.193612+0000","last_fullsized":"2024-03-07T06:53:26.193612+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:12:48.506061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195332+0000","last_change":"2024-03-07T06:53:25.203867+0000","last_active":"2024-03-07T06:53:26.195332+0000","last_peered":"2024-03-07T06:53:26.195332+0000","last_clean":"2024-03-07T06:53:26.195332+0000","last_became_active":"2024-03-07T06:53:25.203567+0000","last_became_peered":"2024-03-07T06:53:25.203567+0000","last_unstale":"2024-03-07T06:53:26.195332+0000","last_undegraded":"2024-03-07T06:53:26.195332+0000","last_fullsized":"2024-03-07T06:53:26.195332+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:43:40.476283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555610+0000","last_change":"2024-03-07T06:53:24.176450+0000","last_active":"2024-03-07T06:53:26.555610+0000","last_peered":"2024-03-07T06:53:26.555610+0000","last_clean":"2024-03-07T06:53:26.555610+0000","last_became_active":"2024-03-07T06:53:24.175876+0000","last_became_peered":"2024-03-07T06:53:24.175876+0000","last_unstale":"2024-03-07T06:53:26.555610+0000","last_undegraded":"2024-03-07T06:53:26.555610+0000","last_fullsized":"2024-03-07T06:53:26.555610+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:20:37.347168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797243+0000","last_change":"2024-03-07T06:53:25.205883+0000","last_active":"2024-03-07T06:53:26.797243+0000","last_peered":"2024-03-07T06:53:26.797243+0000","last_clean":"2024-03-07T06:53:26.797243+0000","last_became_active":"2024-03-07T06:53:25.205193+0000","last_became_peered":"2024-03-07T06:53:25.205193+0000","last_unstale":"2024-03-07T06:53:26.797243+0000","last_undegraded":"2024-03-07T06:53:26.797243+0000","last_fullsized":"2024-03-07T06:53:26.797243+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:49:12.962971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797587+0000","last_change":"2024-03-07T06:53:24.184752+0000","last_active":"2024-03-07T06:53:26.797587+0000","last_peered":"2024-03-07T06:53:26.797587+0000","last_clean":"2024-03-07T06:53:26.797587+0000","last_became_active":"2024-03-07T06:53:24.184338+0000","last_became_peered":"2024-03-07T06:53:24.184338+0000","last_unstale":"2024-03-07T06:53:26.797587+0000","last_undegraded":"2024-03-07T06:53:26.797587+0000","last_fullsized":"2024-03-07T06:53:26.797587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:41:06.781045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194726+0000","last_change":"2024-03-07T06:53:25.203839+0000","last_active":"2024-03-07T06:53:26.194726+0000","last_peered":"2024-03-07T06:53:26.194726+0000","last_clean":"2024-03-07T06:53:26.194726+0000","last_became_active":"2024-03-07T06:53:25.203570+0000","last_became_peered":"2024-03-07T06:53:25.203570+0000","last_unstale":"2024-03-07T06:53:26.194726+0000","last_undegraded":"2024-03-07T06:53:26.194726+0000","last_fullsized":"2024-03-07T06:53:26.194726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:25:16.302702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556704+0000","last_change":"2024-03-07T06:53:24.180862+0000","last_active":"2024-03-07T06:53:26.556704+0000","last_peered":"2024-03-07T06:53:26.556704+0000","last_clean":"2024-03-07T06:53:26.556704+0000","last_became_active":"2024-03-07T06:53:24.180558+0000","last_became_peered":"2024-03-07T06:53:24.180558+0000","last_unstale":"2024-03-07T06:53:26.556704+0000","last_undegraded":"2024-03-07T06:53:26.556704+0000","last_fullsized":"2024-03-07T06:53:26.556704+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:23:18.942189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569862+0000","last_change":"2024-03-07T06:53:25.193872+0000","last_active":"2024-03-07T06:53:26.569862+0000","last_peered":"2024-03-07T06:53:26.569862+0000","last_clean":"2024-03-07T06:53:26.569862+0000","last_became_active":"2024-03-07T06:53:25.193677+0000","last_became_peered":"2024-03-07T06:53:25.193677+0000","last_unstale":"2024-03-07T06:53:26.569862+0000","last_undegraded":"2024-03-07T06:53:26.569862+0000","last_fullsized":"2024-03-07T06:53:26.569862+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:40:13.617392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192612+0000","last_change":"2024-03-07T06:53:24.193883+0000","last_active":"2024-03-07T06:53:26.192612+0000","last_peered":"2024-03-07T06:53:26.192612+0000","last_clean":"2024-03-07T06:53:26.192612+0000","last_became_active":"2024-03-07T06:53:24.193619+0000","last_became_peered":"2024-03-07T06:53:24.193619+0000","last_unstale":"2024-03-07T06:53:26.192612+0000","last_undegraded":"2024-03-07T06:53:26.192612+0000","last_fullsized":"2024-03-07T06:53:26.192612+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:33:46.669137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.191713+0000","last_change":"2024-03-07T06:53:25.208536+0000","last_active":"2024-03-07T06:53:26.191713+0000","last_peered":"2024-03-07T06:53:26.191713+0000","last_clean":"2024-03-07T06:53:26.191713+0000","last_became_active":"2024-03-07T06:53:25.208127+0000","last_became_peered":"2024-03-07T06:53:25.208127+0000","last_unstale":"2024-03-07T06:53:26.191713+0000","last_undegraded":"2024-03-07T06:53:26.191713+0000","last_fullsized":"2024-03-07T06:53:26.191713+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:54:32.067212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798314+0000","last_change":"2024-03-07T06:53:24.182955+0000","last_active":"2024-03-07T06:53:26.798314+0000","last_peered":"2024-03-07T06:53:26.798314+0000","last_clean":"2024-03-07T06:53:26.798314+0000","last_became_active":"2024-03-07T06:53:24.182685+0000","last_became_peered":"2024-03-07T06:53:24.182685+0000","last_unstale":"2024-03-07T06:53:26.798314+0000","last_undegraded":"2024-03-07T06:53:26.798314+0000","last_fullsized":"2024-03-07T06:53:26.798314+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:35:03.451977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194621+0000","last_change":"2024-03-07T06:53:25.190435+0000","last_active":"2024-03-07T06:53:26.194621+0000","last_peered":"2024-03-07T06:53:26.194621+0000","last_clean":"2024-03-07T06:53:26.194621+0000","last_became_active":"2024-03-07T06:53:25.189951+0000","last_became_peered":"2024-03-07T06:53:25.189951+0000","last_unstale":"2024-03-07T06:53:26.194621+0000","last_undegraded":"2024-03-07T06:53:26.194621+0000","last_fullsized":"2024-03-07T06:53:26.194621+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:14:53.578293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569710+0000","last_change":"2024-03-07T06:53:24.177161+0000","last_active":"2024-03-07T06:53:26.569710+0000","last_peered":"2024-03-07T06:53:26.569710+0000","last_clean":"2024-03-07T06:53:26.569710+0000","last_became_active":"2024-03-07T06:53:24.176716+0000","last_became_peered":"2024-03-07T06:53:24.176716+0000","last_unstale":"2024-03-07T06:53:26.569710+0000","last_undegraded":"2024-03-07T06:53:26.569710+0000","last_fullsized":"2024-03-07T06:53:26.569710+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:46:44.201464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556482+0000","last_change":"2024-03-07T06:53:25.203128+0000","last_active":"2024-03-07T06:53:26.556482+0000","last_peered":"2024-03-07T06:53:26.556482+0000","last_clean":"2024-03-07T06:53:26.556482+0000","last_became_active":"2024-03-07T06:53:25.202924+0000","last_became_peered":"2024-03-07T06:53:25.202924+0000","last_unstale":"2024-03-07T06:53:26.556482+0000","last_undegraded":"2024-03-07T06:53:26.556482+0000","last_fullsized":"2024-03-07T06:53:26.556482+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:19:59.756501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.190349+0000","last_change":"2024-03-07T06:53:24.188380+0000","last_active":"2024-03-07T06:53:26.190349+0000","last_peered":"2024-03-07T06:53:26.190349+0000","last_clean":"2024-03-07T06:53:26.190349+0000","last_became_active":"2024-03-07T06:53:24.187005+0000","last_became_peered":"2024-03-07T06:53:24.187005+0000","last_unstale":"2024-03-07T06:53:26.190349+0000","last_undegraded":"2024-03-07T06:53:26.190349+0000","last_fullsized":"2024-03-07T06:53:26.190349+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:43:50.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568329+0000","last_change":"2024-03-07T06:53:25.185049+0000","last_active":"2024-03-07T06:53:26.568329+0000","last_peered":"2024-03-07T06:53:26.568329+0000","last_clean":"2024-03-07T06:53:26.568329+0000","last_became_active":"2024-03-07T06:53:25.184192+0000","last_became_peered":"2024-03-07T06:53:25.184192+0000","last_unstale":"2024-03-07T06:53:26.568329+0000","last_undegraded":"2024-03-07T06:53:26.568329+0000","last_fullsized":"2024-03-07T06:53:26.568329+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:45.850119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568464+0000","last_change":"2024-03-07T06:53:24.177157+0000","last_active":"2024-03-07T06:53:26.568464+0000","last_peered":"2024-03-07T06:53:26.568464+0000","last_clean":"2024-03-07T06:53:26.568464+0000","last_became_active":"2024-03-07T06:53:24.176724+0000","last_became_peered":"2024-03-07T06:53:24.176724+0000","last_unstale":"2024-03-07T06:53:26.568464+0000","last_undegraded":"2024-03-07T06:53:26.568464+0000","last_fullsized":"2024-03-07T06:53:26.568464+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:54.259961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.190647+0000","last_change":"2024-03-07T06:53:25.194921+0000","last_active":"2024-03-07T06:53:26.190647+0000","last_peered":"2024-03-07T06:53:26.190647+0000","last_clean":"2024-03-07T06:53:26.190647+0000","last_became_active":"2024-03-07T06:53:25.193907+0000","last_became_peered":"2024-03-07T06:53:25.193907+0000","last_unstale":"2024-03-07T06:53:26.190647+0000","last_undegraded":"2024-03-07T06:53:26.190647+0000","last_fullsized":"2024-03-07T06:53:26.190647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:21:58.317633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":22,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555716+0000","last_change":"2024-03-07T06:53:24.181056+0000","last_active":"2024-03-07T06:53:26.555716+0000","last_peered":"2024-03-07T06:53:26.555716+0000","last_clean":"2024-03-07T06:53:26.555716+0000","last_became_active":"2024-03-07T06:53:24.180668+0000","last_became_peered":"2024-03-07T06:53:24.180668+0000","last_unstale":"2024-03-07T06:53:26.555716+0000","last_undegraded":"2024-03-07T06:53:26.555716+0000","last_fullsized":"2024-03-07T06:53:26.555716+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:27:55.430918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798458+0000","last_change":"2024-03-07T06:53:25.196095+0000","last_active":"2024-03-07T06:53:26.798458+0000","last_peered":"2024-03-07T06:53:26.798458+0000","last_clean":"2024-03-07T06:53:26.798458+0000","last_became_active":"2024-03-07T06:53:25.195557+0000","last_became_peered":"2024-03-07T06:53:25.195557+0000","last_unstale":"2024-03-07T06:53:26.798458+0000","last_undegraded":"2024-03-07T06:53:26.798458+0000","last_fullsized":"2024-03-07T06:53:26.798458+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:20:38.362147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798600+0000","last_change":"2024-03-07T06:53:24.179843+0000","last_active":"2024-03-07T06:53:26.798600+0000","last_peered":"2024-03-07T06:53:26.798600+0000","last_clean":"2024-03-07T06:53:26.798600+0000","last_became_active":"2024-03-07T06:53:24.179393+0000","last_became_peered":"2024-03-07T06:53:24.179393+0000","last_unstale":"2024-03-07T06:53:26.798600+0000","last_undegraded":"2024-03-07T06:53:26.798600+0000","last_fullsized":"2024-03-07T06:53:26.798600+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:51:11.895335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556574+0000","last_change":"2024-03-07T06:53:25.205258+0000","last_active":"2024-03-07T06:53:26.556574+0000","last_peered":"2024-03-07T06:53:26.556574+0000","last_clean":"2024-03-07T06:53:26.556574+0000","last_became_active":"2024-03-07T06:53:25.204991+0000","last_became_peered":"2024-03-07T06:53:25.204991+0000","last_unstale":"2024-03-07T06:53:26.556574+0000","last_undegraded":"2024-03-07T06:53:26.556574+0000","last_fullsized":"2024-03-07T06:53:26.556574+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:21:58.317633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797419+0000","last_change":"2024-03-07T06:53:24.185672+0000","last_active":"2024-03-07T06:53:26.797419+0000","last_peered":"2024-03-07T06:53:26.797419+0000","last_clean":"2024-03-07T06:53:26.797419+0000","last_became_active":"2024-03-07T06:53:24.185368+0000","last_became_peered":"2024-03-07T06:53:24.185368+0000","last_unstale":"2024-03-07T06:53:26.797419+0000","last_undegraded":"2024-03-07T06:53:26.797419+0000","last_fullsized":"2024-03-07T06:53:26.797419+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:36:40.695222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195109+0000","last_change":"2024-03-07T06:53:25.186795+0000","last_active":"2024-03-07T06:53:26.195109+0000","last_peered":"2024-03-07T06:53:26.195109+0000","last_clean":"2024-03-07T06:53:26.195109+0000","last_became_active":"2024-03-07T06:53:25.186502+0000","last_became_peered":"2024-03-07T06:53:25.186502+0000","last_unstale":"2024-03-07T06:53:26.195109+0000","last_undegraded":"2024-03-07T06:53:26.195109+0000","last_fullsized":"2024-03-07T06:53:26.195109+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:36:41.710201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193159+0000","last_change":"2024-03-07T06:53:24.186577+0000","last_active":"2024-03-07T06:53:26.193159+0000","last_peered":"2024-03-07T06:53:26.193159+0000","last_clean":"2024-03-07T06:53:26.193159+0000","last_became_active":"2024-03-07T06:53:24.186258+0000","last_became_peered":"2024-03-07T06:53:24.186258+0000","last_unstale":"2024-03-07T06:53:26.193159+0000","last_undegraded":"2024-03-07T06:53:26.193159+0000","last_fullsized":"2024-03-07T06:53:26.193159+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:31.033458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195414+0000","last_change":"2024-03-07T06:53:25.189444+0000","last_active":"2024-03-07T06:53:26.195414+0000","last_peered":"2024-03-07T06:53:26.195414+0000","last_clean":"2024-03-07T06:53:26.195414+0000","last_became_active":"2024-03-07T06:53:25.187460+0000","last_became_peered":"2024-03-07T06:53:25.187460+0000","last_unstale":"2024-03-07T06:53:26.195414+0000","last_undegraded":"2024-03-07T06:53:26.195414+0000","last_fullsized":"2024-03-07T06:53:26.195414+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:28:41.703936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569599+0000","last_change":"2024-03-07T06:53:24.189372+0000","last_active":"2024-03-07T06:53:26.569599+0000","last_peered":"2024-03-07T06:53:26.569599+0000","last_clean":"2024-03-07T06:53:26.569599+0000","last_became_active":"2024-03-07T06:53:24.189061+0000","last_became_peered":"2024-03-07T06:53:24.189061+0000","last_unstale":"2024-03-07T06:53:26.569599+0000","last_undegraded":"2024-03-07T06:53:26.569599+0000","last_fullsized":"2024-03-07T06:53:26.569599+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:45:09.141933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568500+0000","last_change":"2024-03-07T06:53:25.193713+0000","last_active":"2024-03-07T06:53:26.568500+0000","last_peered":"2024-03-07T06:53:26.568500+0000","last_clean":"2024-03-07T06:53:26.568500+0000","last_became_active":"2024-03-07T06:53:25.186210+0000","last_became_peered":"2024-03-07T06:53:25.186210+0000","last_unstale":"2024-03-07T06:53:26.568500+0000","last_undegraded":"2024-03-07T06:53:26.568500+0000","last_fullsized":"2024-03-07T06:53:26.568500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:33:15.293206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.570025+0000","last_change":"2024-03-07T06:53:24.180089+0000","last_active":"2024-03-07T06:53:26.570025+0000","last_peered":"2024-03-07T06:53:26.570025+0000","last_clean":"2024-03-07T06:53:26.570025+0000","last_became_active":"2024-03-07T06:53:24.179418+0000","last_became_peered":"2024-03-07T06:53:24.179418+0000","last_unstale":"2024-03-07T06:53:26.570025+0000","last_undegraded":"2024-03-07T06:53:26.570025+0000","last_fullsized":"2024-03-07T06:53:26.570025+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:13.505176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797763+0000","last_change":"2024-03-07T06:53:25.196193+0000","last_active":"2024-03-07T06:53:26.797763+0000","last_peered":"2024-03-07T06:53:26.797763+0000","last_clean":"2024-03-07T06:53:26.797763+0000","last_became_active":"2024-03-07T06:53:25.195898+0000","last_became_peered":"2024-03-07T06:53:25.195898+0000","last_unstale":"2024-03-07T06:53:26.797763+0000","last_undegraded":"2024-03-07T06:53:26.797763+0000","last_fullsized":"2024-03-07T06:53:26.797763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:12:49.521040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193010+0000","last_change":"2024-03-07T06:53:24.191253+0000","last_active":"2024-03-07T06:53:26.193010+0000","last_peered":"2024-03-07T06:53:26.193010+0000","last_clean":"2024-03-07T06:53:26.193010+0000","last_became_active":"2024-03-07T06:53:24.190919+0000","last_became_peered":"2024-03-07T06:53:24.190919+0000","last_unstale":"2024-03-07T06:53:26.193010+0000","last_undegraded":"2024-03-07T06:53:26.193010+0000","last_fullsized":"2024-03-07T06:53:26.193010+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:43:50.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797624+0000","last_change":"2024-03-07T06:53:25.202509+0000","last_active":"2024-03-07T06:53:26.797624+0000","last_peered":"2024-03-07T06:53:26.797624+0000","last_clean":"2024-03-07T06:53:26.797624+0000","last_became_active":"2024-03-07T06:53:25.202242+0000","last_became_peered":"2024-03-07T06:53:25.202242+0000","last_unstale":"2024-03-07T06:53:26.797624+0000","last_undegraded":"2024-03-07T06:53:26.797624+0000","last_fullsized":"2024-03-07T06:53:26.797624+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:01:23.009414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555290+0000","last_change":"2024-03-07T06:53:24.181121+0000","last_active":"2024-03-07T06:53:26.555290+0000","last_peered":"2024-03-07T06:53:26.555290+0000","last_clean":"2024-03-07T06:53:26.555290+0000","last_became_active":"2024-03-07T06:53:24.180669+0000","last_became_peered":"2024-03-07T06:53:24.180669+0000","last_unstale":"2024-03-07T06:53:26.555290+0000","last_undegraded":"2024-03-07T06:53:26.555290+0000","last_fullsized":"2024-03-07T06:53:26.555290+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:24:07.355696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555633+0000","last_change":"2024-03-07T06:53:25.177613+0000","last_active":"2024-03-07T06:53:26.555633+0000","last_peered":"2024-03-07T06:53:26.555633+0000","last_clean":"2024-03-07T06:53:26.555633+0000","last_became_active":"2024-03-07T06:53:25.177320+0000","last_became_peered":"2024-03-07T06:53:25.177320+0000","last_unstale":"2024-03-07T06:53:26.555633+0000","last_undegraded":"2024-03-07T06:53:26.555633+0000","last_fullsized":"2024-03-07T06:53:26.555633+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:45:10.156912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.189619+0000","last_change":"2024-03-07T06:53:24.184705+0000","last_active":"2024-03-07T06:53:26.189619+0000","last_peered":"2024-03-07T06:53:26.189619+0000","last_clean":"2024-03-07T06:53:26.189619+0000","last_became_active":"2024-03-07T06:53:24.184354+0000","last_became_peered":"2024-03-07T06:53:24.184354+0000","last_unstale":"2024-03-07T06:53:26.189619+0000","last_undegraded":"2024-03-07T06:53:26.189619+0000","last_fullsized":"2024-03-07T06:53:26.189619+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:37.281189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.196424+0000","last_change":"2024-03-07T06:53:25.204962+0000","last_active":"2024-03-07T06:53:26.196424+0000","last_peered":"2024-03-07T06:53:26.196424+0000","last_clean":"2024-03-07T06:53:26.196424+0000","last_became_active":"2024-03-07T06:53:25.203898+0000","last_became_peered":"2024-03-07T06:53:25.203898+0000","last_unstale":"2024-03-07T06:53:26.196424+0000","last_undegraded":"2024-03-07T06:53:26.196424+0000","last_fullsized":"2024-03-07T06:53:26.196424+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:36:14.943170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569903+0000","last_change":"2024-03-07T06:53:24.182347+0000","last_active":"2024-03-07T06:53:26.569903+0000","last_peered":"2024-03-07T06:53:26.569903+0000","last_clean":"2024-03-07T06:53:26.569903+0000","last_became_active":"2024-03-07T06:53:24.182031+0000","last_became_peered":"2024-03-07T06:53:24.182031+0000","last_unstale":"2024-03-07T06:53:26.569903+0000","last_undegraded":"2024-03-07T06:53:26.569903+0000","last_fullsized":"2024-03-07T06:53:26.569903+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:33:46.669137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556121+0000","last_change":"2024-03-07T06:53:25.184985+0000","last_active":"2024-03-07T06:53:26.556121+0000","last_peered":"2024-03-07T06:53:26.556121+0000","last_clean":"2024-03-07T06:53:26.556121+0000","last_became_active":"2024-03-07T06:53:25.184052+0000","last_became_peered":"2024-03-07T06:53:25.184052+0000","last_unstale":"2024-03-07T06:53:26.556121+0000","last_undegraded":"2024-03-07T06:53:26.556121+0000","last_fullsized":"2024-03-07T06:53:26.556121+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:49:12.962971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192906+0000","last_change":"2024-03-07T06:53:24.194031+0000","last_active":"2024-03-07T06:53:26.192906+0000","last_peered":"2024-03-07T06:53:26.192906+0000","last_clean":"2024-03-07T06:53:26.192906+0000","last_became_active":"2024-03-07T06:53:24.193718+0000","last_became_peered":"2024-03-07T06:53:24.193718+0000","last_unstale":"2024-03-07T06:53:26.192906+0000","last_undegraded":"2024-03-07T06:53:26.192906+0000","last_fullsized":"2024-03-07T06:53:26.192906+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:53:36.898081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193172+0000","last_change":"2024-03-07T06:53:25.202526+0000","last_active":"2024-03-07T06:53:26.193172+0000","last_peered":"2024-03-07T06:53:26.193172+0000","last_clean":"2024-03-07T06:53:26.193172+0000","last_became_active":"2024-03-07T06:53:25.202243+0000","last_became_peered":"2024-03-07T06:53:25.202243+0000","last_unstale":"2024-03-07T06:53:26.193172+0000","last_undegraded":"2024-03-07T06:53:26.193172+0000","last_fullsized":"2024-03-07T06:53:26.193172+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:55.587418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796596+0000","last_change":"2024-03-07T06:53:24.183011+0000","last_active":"2024-03-07T06:53:26.796596+0000","last_peered":"2024-03-07T06:53:26.796596+0000","last_clean":"2024-03-07T06:53:26.796596+0000","last_became_active":"2024-03-07T06:53:24.182772+0000","last_became_peered":"2024-03-07T06:53:24.182772+0000","last_unstale":"2024-03-07T06:53:26.796596+0000","last_undegraded":"2024-03-07T06:53:26.796596+0000","last_fullsized":"2024-03-07T06:53:26.796596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:14:52.563314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568643+0000","last_change":"2024-03-07T06:53:25.173467+0000","last_active":"2024-03-07T06:53:26.568643+0000","last_peered":"2024-03-07T06:53:26.568643+0000","last_clean":"2024-03-07T06:53:26.568643+0000","last_became_active":"2024-03-07T06:53:25.173157+0000","last_became_peered":"2024-03-07T06:53:25.173157+0000","last_unstale":"2024-03-07T06:53:26.568643+0000","last_undegraded":"2024-03-07T06:53:26.568643+0000","last_fullsized":"2024-03-07T06:53:26.568643+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:04:56.048641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.191151+0000","last_change":"2024-03-07T06:53:24.196702+0000","last_active":"2024-03-07T06:53:26.191151+0000","last_peered":"2024-03-07T06:53:26.191151+0000","last_clean":"2024-03-07T06:53:26.191151+0000","last_became_active":"2024-03-07T06:53:24.196356+0000","last_became_peered":"2024-03-07T06:53:24.196356+0000","last_unstale":"2024-03-07T06:53:26.191151+0000","last_undegraded":"2024-03-07T06:53:26.191151+0000","last_fullsized":"2024-03-07T06:53:26.191151+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:54.259961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569797+0000","last_change":"2024-03-07T06:53:25.190375+0000","last_active":"2024-03-07T06:53:26.569797+0000","last_peered":"2024-03-07T06:53:26.569797+0000","last_clean":"2024-03-07T06:53:26.569797+0000","last_became_active":"2024-03-07T06:53:25.189627+0000","last_became_peered":"2024-03-07T06:53:25.189627+0000","last_unstale":"2024-03-07T06:53:26.569797+0000","last_undegraded":"2024-03-07T06:53:26.569797+0000","last_fullsized":"2024-03-07T06:53:26.569797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:39:28.721920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797059+0000","last_change":"2024-03-07T06:53:24.180383+0000","last_active":"2024-03-07T06:53:26.797059+0000","last_peered":"2024-03-07T06:53:26.797059+0000","last_clean":"2024-03-07T06:53:26.797059+0000","last_became_active":"2024-03-07T06:53:24.180057+0000","last_became_peered":"2024-03-07T06:53:24.180057+0000","last_unstale":"2024-03-07T06:53:26.797059+0000","last_undegraded":"2024-03-07T06:53:26.797059+0000","last_fullsized":"2024-03-07T06:53:26.797059+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:43:39.461304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192907+0000","last_change":"2024-03-07T06:53:25.195366+0000","last_active":"2024-03-07T06:53:26.192907+0000","last_peered":"2024-03-07T06:53:26.192907+0000","last_clean":"2024-03-07T06:53:26.192907+0000","last_became_active":"2024-03-07T06:53:25.194848+0000","last_became_peered":"2024-03-07T06:53:25.194848+0000","last_unstale":"2024-03-07T06:53:26.192907+0000","last_undegraded":"2024-03-07T06:53:26.192907+0000","last_fullsized":"2024-03-07T06:53:26.192907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:46:45.216443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796937+0000","last_change":"2024-03-07T06:53:24.185042+0000","last_active":"2024-03-07T06:53:26.796937+0000","last_peered":"2024-03-07T06:53:26.796937+0000","last_clean":"2024-03-07T06:53:26.796937+0000","last_became_active":"2024-03-07T06:53:24.184633+0000","last_became_peered":"2024-03-07T06:53:24.184633+0000","last_unstale":"2024-03-07T06:53:26.796937+0000","last_undegraded":"2024-03-07T06:53:26.796937+0000","last_fullsized":"2024-03-07T06:53:26.796937+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:35.550886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796752+0000","last_change":"2024-03-07T06:53:25.186793+0000","last_active":"2024-03-07T06:53:26.796752+0000","last_peered":"2024-03-07T06:53:26.796752+0000","last_clean":"2024-03-07T06:53:26.796752+0000","last_became_active":"2024-03-07T06:53:25.182526+0000","last_became_peered":"2024-03-07T06:53:25.182526+0000","last_unstale":"2024-03-07T06:53:26.796752+0000","last_undegraded":"2024-03-07T06:53:26.796752+0000","last_fullsized":"2024-03-07T06:53:26.796752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:53:37.913060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555749+0000","last_change":"2024-03-07T06:53:24.176113+0000","last_active":"2024-03-07T06:53:26.555749+0000","last_peered":"2024-03-07T06:53:26.555749+0000","last_clean":"2024-03-07T06:53:26.555749+0000","last_became_active":"2024-03-07T06:53:24.175506+0000","last_became_peered":"2024-03-07T06:53:24.175506+0000","last_unstale":"2024-03-07T06:53:26.555749+0000","last_undegraded":"2024-03-07T06:53:26.555749+0000","last_fullsized":"2024-03-07T06:53:26.555749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:20:28.607718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556710+0000","last_change":"2024-03-07T06:53:25.203466+0000","last_active":"2024-03-07T06:53:26.556710+0000","last_peered":"2024-03-07T06:53:26.556710+0000","last_clean":"2024-03-07T06:53:26.556710+0000","last_became_active":"2024-03-07T06:53:25.203113+0000","last_became_peered":"2024-03-07T06:53:25.203113+0000","last_unstale":"2024-03-07T06:53:26.556710+0000","last_undegraded":"2024-03-07T06:53:26.556710+0000","last_fullsized":"2024-03-07T06:53:26.556710+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:55.587418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194351+0000","last_change":"2024-03-07T06:53:24.186782+0000","last_active":"2024-03-07T06:53:26.194351+0000","last_peered":"2024-03-07T06:53:26.194351+0000","last_clean":"2024-03-07T06:53:26.194351+0000","last_became_active":"2024-03-07T06:53:24.186396+0000","last_became_peered":"2024-03-07T06:53:24.186396+0000","last_unstale":"2024-03-07T06:53:26.194351+0000","last_undegraded":"2024-03-07T06:53:26.194351+0000","last_fullsized":"2024-03-07T06:53:26.194351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:36:13.928191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.189635+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:53:26.189635+0000","last_peered":"2024-03-07T06:53:26.189635+0000","last_clean":"2024-03-07T06:53:26.189635+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:53:26.189635+0000","last_undegraded":"2024-03-07T06:53:26.189635+0000","last_fullsized":"2024-03-07T06:53:26.189635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":810252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":548864,"data_stored":220466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":219864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":552960,"data_stored":221406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":552960,"data_stored":220432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":221428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":219864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1138688,"data_stored":810196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:53:35.032 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:53:35.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.242+0000 7f9a8fe00640 1 Processor -- start 2024-03-07T06:53:35.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.242+0000 7f9a8fe00640 1 -- start start 2024-03-07T06:53:35.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.244+0000 7f9a8fe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9a900cf5d0 0x7f9a90072830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:35.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.244+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a90072d70 0x7f9a9015c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:35.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.244+0000 7f9a8fe00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a90160760 con 0x7f9a9015bd30 2024-03-07T06:53:35.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.244+0000 7f9a8fe00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a900731c0 con 0x7f9a900cf5d0 2024-03-07T06:53:35.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.244+0000 7f9a8fe00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9015f9a0 con 0x7f9a90072d70 2024-03-07T06:53:35.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.245+0000 7f9a8e000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9a900cf5d0 0x7f9a90072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:35.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.245+0000 7f9a8e000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9a900cf5d0 0x7f9a90072830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35806/0 (socket says 172.21.15.133:35806) 2024-03-07T06:53:35.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.245+0000 7f9a8e000640 1 -- 172.21.15.133:0/365524470 learned_addr learned my addr 172.21.15.133:0/365524470 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:35.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.245+0000 7f9a8ea00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f9a9015bd30 0x7f9a9006e8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47462/0 (socket says 172.21.15.133:47462) 2024-03-07T06:53:35.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.245+0000 7f9a8f400640 1 --2- 172.21.15.133:0/365524470 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a90072d70 0x7f9a9015c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:35.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.245+0000 7f9a8e000640 1 -- 172.21.15.133:0/365524470 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a90072d70 msgr2=0x7f9a9015c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:35.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.245+0000 7f9a8e000640 1 --2- 172.21.15.133:0/365524470 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a90072d70 0x7f9a9015c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:35.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.245+0000 7f9a8e000640 1 -- 172.21.15.133:0/365524470 >> v1:172.21.15.133:6789/0 conn(0x7f9a9015bd30 legacy=0x7f9a9006e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:35.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.245+0000 7f9a8e000640 1 -- 172.21.15.133:0/365524470 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9015fc80 con 0x7f9a900cf5d0 2024-03-07T06:53:35.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.245+0000 7f9a8f400640 1 --2- 172.21.15.133:0/365524470 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a90072d70 0x7f9a9015c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:35.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.247+0000 7f9a8e000640 1 --2- 172.21.15.133:0/365524470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a900cf5d0 0x7f9a90072830 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009870 tx=0x7f9a7c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d834b9d37c7ef4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:35.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.249+0000 7f9a8d600640 1 -- 172.21.15.133:0/365524470 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c034070 con 0x7f9a900cf5d0 2024-03-07T06:53:35.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.249+0000 7f9a8d600640 1 -- 172.21.15.133:0/365524470 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c030840 con 0x7f9a900cf5d0 2024-03-07T06:53:35.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.249+0000 7f9a8d600640 1 -- 172.21.15.133:0/365524470 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c030c10 con 0x7f9a900cf5d0 2024-03-07T06:53:35.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.249+0000 7f9a8fe00640 1 -- 172.21.15.133:0/365524470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a900cf5d0 msgr2=0x7f9a90072830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:35.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.249+0000 7f9a8fe00640 1 --2- 172.21.15.133:0/365524470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a900cf5d0 0x7f9a90072830 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009870 tx=0x7f9a7c02b980 comp rx=0 tx=0).stop 2024-03-07T06:53:35.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.249+0000 7f9a8fe00640 1 -- 172.21.15.133:0/365524470 shutdown_connections 2024-03-07T06:53:35.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.249+0000 7f9a8fe00640 1 --2- 172.21.15.133:0/365524470 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a90072d70 0x7f9a9015c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:35.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.249+0000 7f9a8fe00640 1 --2- 172.21.15.133:0/365524470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a900cf5d0 0x7f9a90072830 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:35.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.249+0000 7f9a8fe00640 1 -- 172.21.15.133:0/365524470 >> 172.21.15.133:0/365524470 conn(0x7f9a900b2e90 msgr2=0x7f9a900b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:35.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.249+0000 7f9a8fe00640 1 -- 172.21.15.133:0/365524470 shutdown_connections 2024-03-07T06:53:35.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.249+0000 7f9a8fe00640 1 -- 172.21.15.133:0/365524470 wait complete. 2024-03-07T06:53:35.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.251+0000 7f9a8fe00640 1 Processor -- start 2024-03-07T06:53:35.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.251+0000 7f9a8fe00640 1 -- start start 2024-03-07T06:53:35.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.252+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a90072d70 0x7f9a9014f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.252+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a900cf5d0 0x7f9a9014a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.252+0000 7f9a8fe00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a9015bd30 0x7f9a9014ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.252+0000 7f9a8fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9006f130 con 0x7f9a900cf5d0 2024-03-07T06:53:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.252+0000 7f9a8fe00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a90160240 con 0x7f9a90072d70 2024-03-07T06:53:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.252+0000 7f9a8fe00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9015c650 con 0x7f9a9015bd30 2024-03-07T06:53:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.253+0000 7f9a8e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a900cf5d0 0x7f9a9014a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.253+0000 7f9a8e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a900cf5d0 0x7f9a9014a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35820/0 (socket says 172.21.15.133:35820) 2024-03-07T06:53:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.253+0000 7f9a8e000640 1 -- 172.21.15.133:0/2365774439 learned_addr learned my addr 172.21.15.133:0/2365774439 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.253+0000 7f9a8f400640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a9015bd30 0x7f9a9014ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.253+0000 7f9a8ea00640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a90072d70 0x7f9a9014f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.253+0000 7f9a8e000640 1 -- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a9015bd30 msgr2=0x7f9a9014ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.253+0000 7f9a8e000640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a9015bd30 0x7f9a9014ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.253+0000 7f9a8e000640 1 -- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a90072d70 msgr2=0x7f9a9014f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:35.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.253+0000 7f9a8e000640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a90072d70 0x7f9a9014f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:35.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.253+0000 7f9a8e000640 1 -- 172.21.15.133:0/2365774439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9014fd10 con 0x7f9a900cf5d0 2024-03-07T06:53:35.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.253+0000 7f9a8ea00640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a90072d70 0x7f9a9014f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:35.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.254+0000 7f9a8e000640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a900cf5d0 0x7f9a9014a310 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c02b9c0 tx=0x7f9a7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:35.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.254+0000 7f9a86a00640 1 -- 172.21.15.133:0/2365774439 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c048070 con 0x7f9a900cf5d0 2024-03-07T06:53:35.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.254+0000 7f9a86a00640 1 -- 172.21.15.133:0/2365774439 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c030640 con 0x7f9a900cf5d0 2024-03-07T06:53:35.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.254+0000 7f9a86a00640 1 -- 172.21.15.133:0/2365774439 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c034040 con 0x7f9a900cf5d0 2024-03-07T06:53:35.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.254+0000 7f9a8fe00640 1 -- 172.21.15.133:0/2365774439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9014fff0 con 0x7f9a900cf5d0 2024-03-07T06:53:35.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.254+0000 7f9a8fe00640 1 -- 172.21.15.133:0/2365774439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a901505b0 con 0x7f9a900cf5d0 2024-03-07T06:53:35.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.255+0000 7f9a8fe00640 1 -- 172.21.15.133:0/2365774439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a90062c90 con 0x7f9a900cf5d0 2024-03-07T06:53:35.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.256+0000 7f9a86a00640 1 -- 172.21.15.133:0/2365774439 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9a7c037070 con 0x7f9a900cf5d0 2024-03-07T06:53:35.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.257+0000 7f9a86a00640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9a7006b4d0 0x7f9a7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:35.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.257+0000 7f9a86a00640 1 -- 172.21.15.133:0/2365774439 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a7c0abfb0 con 0x7f9a900cf5d0 2024-03-07T06:53:35.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.258+0000 7f9a8ea00640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9a7006b4d0 0x7f9a7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:35.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.259+0000 7f9a8ea00640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9a7006b4d0 0x7f9a7006d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9a80000f60 tx=0x7f9a80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:35.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.267+0000 7f9a86a00640 1 -- 172.21.15.133:0/2365774439 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9a7c030810 con 0x7f9a900cf5d0 2024-03-07T06:53:35.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.456+0000 7f9a8fe00640 1 -- 172.21.15.133:0/2365774439 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a90166a70 con 0x7f9a7006b4d0 2024-03-07T06:53:35.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.475+0000 7f9a86a00640 1 -- 172.21.15.133:0/2365774439 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251848 (secure 0 0 0) 0x7f9a90166a70 con 0x7f9a7006b4d0 2024-03-07T06:53:35.478 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:35.479 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:53:35.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.479+0000 7f9a8fe00640 1 -- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9a7006b4d0 msgr2=0x7f9a7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:35.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.479+0000 7f9a8fe00640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9a7006b4d0 0x7f9a7006d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9a80000f60 tx=0x7f9a80009040 comp rx=0 tx=0).stop 2024-03-07T06:53:35.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.479+0000 7f9a8fe00640 1 -- 172.21.15.133:0/2365774439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a900cf5d0 msgr2=0x7f9a9014a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:35.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.479+0000 7f9a8fe00640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a900cf5d0 0x7f9a9014a310 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c02b9c0 tx=0x7f9a7c004730 comp rx=0 tx=0).stop 2024-03-07T06:53:35.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.479+0000 7f9a8fe00640 1 -- 172.21.15.133:0/2365774439 shutdown_connections 2024-03-07T06:53:35.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.479+0000 7f9a8fe00640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a9015bd30 0x7f9a9014ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:35.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.479+0000 7f9a8fe00640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a900cf5d0 0x7f9a9014a310 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:35.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.479+0000 7f9a8fe00640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9a7006b4d0 0x7f9a7006d990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:35.481 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.479+0000 7f9a8fe00640 1 --2- 172.21.15.133:0/2365774439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a90072d70 0x7f9a9014f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:35.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.479+0000 7f9a8fe00640 1 -- 172.21.15.133:0/2365774439 >> 172.21.15.133:0/2365774439 conn(0x7f9a900b2e90 msgr2=0x7f9a9006ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:35.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.480+0000 7f9a8fe00640 1 -- 172.21.15.133:0/2365774439 shutdown_connections 2024-03-07T06:53:35.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.480+0000 7f9a8fe00640 1 -- 172.21.15.133:0/2365774439 wait complete. 2024-03-07T06:53:35.497 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-03-07T06:53:34.520424+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250760,"kb_used_data":5448,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754723960,"statfs":{"total":773094113280,"available":772837335040,"internally_reserved":0,"allocated":5578752,"data_stored":2943908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.361778"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194303+0000","last_change":"2024-03-07T06:53:25.202958+0000","last_active":"2024-03-07T06:53:26.194303+0000","last_peered":"2024-03-07T06:53:26.194303+0000","last_clean":"2024-03-07T06:53:26.194303+0000","last_became_active":"2024-03-07T06:53:25.202689+0000","last_became_peered":"2024-03-07T06:53:25.202689+0000","last_unstale":"2024-03-07T06:53:26.194303+0000","last_undegraded":"2024-03-07T06:53:26.194303+0000","last_fullsized":"2024-03-07T06:53:26.194303+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:51:54.139595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192936+0000","last_change":"2024-03-07T06:53:25.205060+0000","last_active":"2024-03-07T06:53:26.192936+0000","last_peered":"2024-03-07T06:53:26.192936+0000","last_clean":"2024-03-07T06:53:26.192936+0000","last_became_active":"2024-03-07T06:53:25.204801+0000","last_became_peered":"2024-03-07T06:53:25.204801+0000","last_unstale":"2024-03-07T06:53:26.192936+0000","last_undegraded":"2024-03-07T06:53:26.192936+0000","last_fullsized":"2024-03-07T06:53:26.192936+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:14.520156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796955+0000","last_change":"2024-03-07T06:53:25.189468+0000","last_active":"2024-03-07T06:53:26.796955+0000","last_peered":"2024-03-07T06:53:26.796955+0000","last_clean":"2024-03-07T06:53:26.796955+0000","last_became_active":"2024-03-07T06:53:25.189178+0000","last_became_peered":"2024-03-07T06:53:25.189178+0000","last_unstale":"2024-03-07T06:53:26.796955+0000","last_undegraded":"2024-03-07T06:53:26.796955+0000","last_fullsized":"2024-03-07T06:53:26.796955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:04.729008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556013+0000","last_change":"2024-03-07T06:53:25.193381+0000","last_active":"2024-03-07T06:53:26.556013+0000","last_peered":"2024-03-07T06:53:26.556013+0000","last_clean":"2024-03-07T06:53:26.556013+0000","last_became_active":"2024-03-07T06:53:25.193079+0000","last_became_peered":"2024-03-07T06:53:25.193079+0000","last_unstale":"2024-03-07T06:53:26.556013+0000","last_undegraded":"2024-03-07T06:53:26.556013+0000","last_fullsized":"2024-03-07T06:53:26.556013+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:14.520156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193500+0000","last_change":"2024-03-07T06:53:25.202836+0000","last_active":"2024-03-07T06:53:26.193500+0000","last_peered":"2024-03-07T06:53:26.193500+0000","last_clean":"2024-03-07T06:53:26.193500+0000","last_became_active":"2024-03-07T06:53:25.202463+0000","last_became_peered":"2024-03-07T06:53:25.202463+0000","last_unstale":"2024-03-07T06:53:26.193500+0000","last_undegraded":"2024-03-07T06:53:26.193500+0000","last_fullsized":"2024-03-07T06:53:26.193500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:44.090672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568638+0000","last_change":"2024-03-07T06:53:25.203678+0000","last_active":"2024-03-07T06:53:26.568638+0000","last_peered":"2024-03-07T06:53:26.568638+0000","last_clean":"2024-03-07T06:53:26.568638+0000","last_became_active":"2024-03-07T06:53:25.198295+0000","last_became_peered":"2024-03-07T06:53:25.198295+0000","last_unstale":"2024-03-07T06:53:26.568638+0000","last_undegraded":"2024-03-07T06:53:26.568638+0000","last_fullsized":"2024-03-07T06:53:26.568638+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:49:12.380843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.190260+0000","last_change":"2024-03-07T06:53:25.193112+0000","last_active":"2024-03-07T06:53:26.190260+0000","last_peered":"2024-03-07T06:53:26.190260+0000","last_clean":"2024-03-07T06:53:26.190260+0000","last_became_active":"2024-03-07T06:53:25.192861+0000","last_became_peered":"2024-03-07T06:53:25.192861+0000","last_unstale":"2024-03-07T06:53:26.190260+0000","last_undegraded":"2024-03-07T06:53:26.190260+0000","last_fullsized":"2024-03-07T06:53:26.190260+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:08:50.099907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555336+0000","last_change":"2024-03-07T06:53:25.200927+0000","last_active":"2024-03-07T06:53:26.555336+0000","last_peered":"2024-03-07T06:53:26.555336+0000","last_clean":"2024-03-07T06:53:26.555336+0000","last_became_active":"2024-03-07T06:53:25.200535+0000","last_became_peered":"2024-03-07T06:53:25.200535+0000","last_unstale":"2024-03-07T06:53:26.555336+0000","last_undegraded":"2024-03-07T06:53:26.555336+0000","last_fullsized":"2024-03-07T06:53:26.555336+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:55.274941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556694+0000","last_change":"2024-03-07T06:53:25.179342+0000","last_active":"2024-03-07T06:53:26.556694+0000","last_peered":"2024-03-07T06:53:26.556694+0000","last_clean":"2024-03-07T06:53:26.556694+0000","last_became_active":"2024-03-07T06:53:25.179031+0000","last_became_peered":"2024-03-07T06:53:25.179031+0000","last_unstale":"2024-03-07T06:53:26.556694+0000","last_undegraded":"2024-03-07T06:53:26.556694+0000","last_fullsized":"2024-03-07T06:53:26.556694+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:54:32.067212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556561+0000","last_change":"2024-03-07T06:53:25.199475+0000","last_active":"2024-03-07T06:53:26.556561+0000","last_peered":"2024-03-07T06:53:26.556561+0000","last_clean":"2024-03-07T06:53:26.556561+0000","last_became_active":"2024-03-07T06:53:25.193080+0000","last_became_peered":"2024-03-07T06:53:25.193080+0000","last_unstale":"2024-03-07T06:53:26.556561+0000","last_undegraded":"2024-03-07T06:53:26.556561+0000","last_fullsized":"2024-03-07T06:53:26.556561+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:08:50.099907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555759+0000","last_change":"2024-03-07T06:53:25.177187+0000","last_active":"2024-03-07T06:53:26.555759+0000","last_peered":"2024-03-07T06:53:26.555759+0000","last_clean":"2024-03-07T06:53:26.555759+0000","last_became_active":"2024-03-07T06:53:25.176924+0000","last_became_peered":"2024-03-07T06:53:25.176924+0000","last_unstale":"2024-03-07T06:53:26.555759+0000","last_undegraded":"2024-03-07T06:53:26.555759+0000","last_fullsized":"2024-03-07T06:53:26.555759+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:04.729008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193384+0000","last_change":"2024-03-07T06:53:25.197797+0000","last_active":"2024-03-07T06:53:26.193384+0000","last_peered":"2024-03-07T06:53:26.193384+0000","last_clean":"2024-03-07T06:53:26.193384+0000","last_became_active":"2024-03-07T06:53:25.197396+0000","last_became_peered":"2024-03-07T06:53:25.197396+0000","last_unstale":"2024-03-07T06:53:26.193384+0000","last_undegraded":"2024-03-07T06:53:26.193384+0000","last_fullsized":"2024-03-07T06:53:26.193384+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:49:12.962971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797908+0000","last_change":"2024-03-07T06:53:25.198602+0000","last_active":"2024-03-07T06:53:26.797908+0000","last_peered":"2024-03-07T06:53:26.797908+0000","last_clean":"2024-03-07T06:53:26.797908+0000","last_became_active":"2024-03-07T06:53:25.197653+0000","last_became_peered":"2024-03-07T06:53:25.197653+0000","last_unstale":"2024-03-07T06:53:26.797908+0000","last_undegraded":"2024-03-07T06:53:26.797908+0000","last_fullsized":"2024-03-07T06:53:26.797908+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:03:09.371531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555920+0000","last_change":"2024-03-07T06:53:25.180030+0000","last_active":"2024-03-07T06:53:26.555920+0000","last_peered":"2024-03-07T06:53:26.555920+0000","last_clean":"2024-03-07T06:53:26.555920+0000","last_became_active":"2024-03-07T06:53:25.178772+0000","last_became_peered":"2024-03-07T06:53:25.178772+0000","last_unstale":"2024-03-07T06:53:26.555920+0000","last_undegraded":"2024-03-07T06:53:26.555920+0000","last_fullsized":"2024-03-07T06:53:26.555920+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:46:45.216443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568373+0000","last_change":"2024-03-07T06:53:25.197311+0000","last_active":"2024-03-07T06:53:26.568373+0000","last_peered":"2024-03-07T06:53:26.568373+0000","last_clean":"2024-03-07T06:53:26.568373+0000","last_became_active":"2024-03-07T06:53:25.196985+0000","last_became_peered":"2024-03-07T06:53:25.196985+0000","last_unstale":"2024-03-07T06:53:26.568373+0000","last_undegraded":"2024-03-07T06:53:26.568373+0000","last_fullsized":"2024-03-07T06:53:26.568373+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:05:54.805828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192024+0000","last_change":"2024-03-07T06:53:25.204127+0000","last_active":"2024-03-07T06:53:26.192024+0000","last_peered":"2024-03-07T06:53:26.192024+0000","last_clean":"2024-03-07T06:53:26.192024+0000","last_became_active":"2024-03-07T06:53:25.203266+0000","last_became_peered":"2024-03-07T06:53:25.203266+0000","last_unstale":"2024-03-07T06:53:26.192024+0000","last_undegraded":"2024-03-07T06:53:26.192024+0000","last_fullsized":"2024-03-07T06:53:26.192024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:13:15.845709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797626+0000","last_change":"2024-03-07T06:53:25.194438+0000","last_active":"2024-03-07T06:53:26.797626+0000","last_peered":"2024-03-07T06:53:26.797626+0000","last_clean":"2024-03-07T06:53:26.797626+0000","last_became_active":"2024-03-07T06:53:25.194130+0000","last_became_peered":"2024-03-07T06:53:25.194130+0000","last_unstale":"2024-03-07T06:53:26.797626+0000","last_undegraded":"2024-03-07T06:53:26.797626+0000","last_fullsized":"2024-03-07T06:53:26.797626+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:19:59.756501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797261+0000","last_change":"2024-03-07T06:53:25.206131+0000","last_active":"2024-03-07T06:53:26.797261+0000","last_peered":"2024-03-07T06:53:26.797261+0000","last_clean":"2024-03-07T06:53:26.797261+0000","last_became_active":"2024-03-07T06:53:25.205578+0000","last_became_peered":"2024-03-07T06:53:25.205578+0000","last_unstale":"2024-03-07T06:53:26.797261+0000","last_undegraded":"2024-03-07T06:53:26.797261+0000","last_fullsized":"2024-03-07T06:53:26.797261+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:44.090672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195955+0000","last_change":"2024-03-07T06:53:25.188986+0000","last_active":"2024-03-07T06:53:26.195955+0000","last_peered":"2024-03-07T06:53:26.195955+0000","last_clean":"2024-03-07T06:53:26.195955+0000","last_became_active":"2024-03-07T06:53:25.188678+0000","last_became_peered":"2024-03-07T06:53:25.188678+0000","last_unstale":"2024-03-07T06:53:26.195955+0000","last_undegraded":"2024-03-07T06:53:26.195955+0000","last_fullsized":"2024-03-07T06:53:26.195955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:32.048437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798581+0000","last_change":"2024-03-07T06:53:25.204686+0000","last_active":"2024-03-07T06:53:26.798581+0000","last_peered":"2024-03-07T06:53:26.798581+0000","last_clean":"2024-03-07T06:53:26.798581+0000","last_became_active":"2024-03-07T06:53:25.203607+0000","last_became_peered":"2024-03-07T06:53:25.203607+0000","last_unstale":"2024-03-07T06:53:26.798581+0000","last_undegraded":"2024-03-07T06:53:26.798581+0000","last_fullsized":"2024-03-07T06:53:26.798581+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:27:56.445897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568794+0000","last_change":"2024-03-07T06:53:25.190607+0000","last_active":"2024-03-07T06:53:26.568794+0000","last_peered":"2024-03-07T06:53:26.568794+0000","last_clean":"2024-03-07T06:53:26.568794+0000","last_became_active":"2024-03-07T06:53:25.190083+0000","last_became_peered":"2024-03-07T06:53:25.190083+0000","last_unstale":"2024-03-07T06:53:26.568794+0000","last_undegraded":"2024-03-07T06:53:26.568794+0000","last_fullsized":"2024-03-07T06:53:26.568794+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:47:02.991917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555545+0000","last_change":"2024-03-07T06:53:25.200334+0000","last_active":"2024-03-07T06:53:26.555545+0000","last_peered":"2024-03-07T06:53:26.555545+0000","last_clean":"2024-03-07T06:53:26.555545+0000","last_became_active":"2024-03-07T06:53:25.200096+0000","last_became_peered":"2024-03-07T06:53:25.200096+0000","last_unstale":"2024-03-07T06:53:26.555545+0000","last_undegraded":"2024-03-07T06:53:26.555545+0000","last_fullsized":"2024-03-07T06:53:26.555545+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:33:47.684117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193801+0000","last_change":"2024-03-07T06:53:25.202494+0000","last_active":"2024-03-07T06:53:26.193801+0000","last_peered":"2024-03-07T06:53:26.193801+0000","last_clean":"2024-03-07T06:53:26.193801+0000","last_became_active":"2024-03-07T06:53:25.202185+0000","last_became_peered":"2024-03-07T06:53:25.202185+0000","last_unstale":"2024-03-07T06:53:26.193801+0000","last_undegraded":"2024-03-07T06:53:26.193801+0000","last_fullsized":"2024-03-07T06:53:26.193801+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:36.565865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192462+0000","last_change":"2024-03-07T06:53:25.203558+0000","last_active":"2024-03-07T06:53:26.192462+0000","last_peered":"2024-03-07T06:53:26.192462+0000","last_clean":"2024-03-07T06:53:26.192462+0000","last_became_active":"2024-03-07T06:53:25.202854+0000","last_became_peered":"2024-03-07T06:53:25.202854+0000","last_unstale":"2024-03-07T06:53:26.192462+0000","last_undegraded":"2024-03-07T06:53:26.192462+0000","last_fullsized":"2024-03-07T06:53:26.192462+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:34:02.638312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195196+0000","last_change":"2024-03-07T06:53:25.193176+0000","last_active":"2024-03-07T06:53:26.195196+0000","last_peered":"2024-03-07T06:53:26.195196+0000","last_clean":"2024-03-07T06:53:26.195196+0000","last_became_active":"2024-03-07T06:53:25.192881+0000","last_became_peered":"2024-03-07T06:53:25.192881+0000","last_unstale":"2024-03-07T06:53:26.195196+0000","last_undegraded":"2024-03-07T06:53:26.195196+0000","last_fullsized":"2024-03-07T06:53:26.195196+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:07:11.833897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798298+0000","last_change":"2024-03-07T06:53:25.205870+0000","last_active":"2024-03-07T06:53:26.798298+0000","last_peered":"2024-03-07T06:53:26.798298+0000","last_clean":"2024-03-07T06:53:26.798298+0000","last_became_active":"2024-03-07T06:53:25.205595+0000","last_became_peered":"2024-03-07T06:53:25.205595+0000","last_unstale":"2024-03-07T06:53:26.798298+0000","last_undegraded":"2024-03-07T06:53:26.798298+0000","last_fullsized":"2024-03-07T06:53:26.798298+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:13:01.272121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797104+0000","last_change":"2024-03-07T06:53:25.187957+0000","last_active":"2024-03-07T06:53:26.797104+0000","last_peered":"2024-03-07T06:53:26.797104+0000","last_clean":"2024-03-07T06:53:26.797104+0000","last_became_active":"2024-03-07T06:53:25.184716+0000","last_became_peered":"2024-03-07T06:53:25.184716+0000","last_unstale":"2024-03-07T06:53:26.797104+0000","last_undegraded":"2024-03-07T06:53:26.797104+0000","last_fullsized":"2024-03-07T06:53:26.797104+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:14.520156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194874+0000","last_change":"2024-03-07T06:53:25.194725+0000","last_active":"2024-03-07T06:53:26.194874+0000","last_peered":"2024-03-07T06:53:26.194874+0000","last_clean":"2024-03-07T06:53:26.194874+0000","last_became_active":"2024-03-07T06:53:25.194434+0000","last_became_peered":"2024-03-07T06:53:25.194434+0000","last_unstale":"2024-03-07T06:53:26.194874+0000","last_undegraded":"2024-03-07T06:53:26.194874+0000","last_fullsized":"2024-03-07T06:53:26.194874+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:08:50.099907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195182+0000","last_change":"2024-03-07T06:53:25.188058+0000","last_active":"2024-03-07T06:53:26.195182+0000","last_peered":"2024-03-07T06:53:26.195182+0000","last_clean":"2024-03-07T06:53:26.195182+0000","last_became_active":"2024-03-07T06:53:25.187778+0000","last_became_peered":"2024-03-07T06:53:25.187778+0000","last_unstale":"2024-03-07T06:53:26.195182+0000","last_undegraded":"2024-03-07T06:53:26.195182+0000","last_fullsized":"2024-03-07T06:53:26.195182+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:41:07.796024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796976+0000","last_change":"2024-03-07T06:53:25.203241+0000","last_active":"2024-03-07T06:53:26.796976+0000","last_peered":"2024-03-07T06:53:26.796976+0000","last_clean":"2024-03-07T06:53:26.796976+0000","last_became_active":"2024-03-07T06:53:25.202804+0000","last_became_peered":"2024-03-07T06:53:25.202804+0000","last_unstale":"2024-03-07T06:53:26.796976+0000","last_undegraded":"2024-03-07T06:53:26.796976+0000","last_fullsized":"2024-03-07T06:53:26.796976+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:46:45.216443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568377+0000","last_change":"2024-03-07T06:53:25.193410+0000","last_active":"2024-03-07T06:53:26.568377+0000","last_peered":"2024-03-07T06:53:26.568377+0000","last_clean":"2024-03-07T06:53:26.568377+0000","last_became_active":"2024-03-07T06:53:25.193130+0000","last_became_peered":"2024-03-07T06:53:25.193130+0000","last_unstale":"2024-03-07T06:53:26.568377+0000","last_undegraded":"2024-03-07T06:53:26.568377+0000","last_fullsized":"2024-03-07T06:53:26.568377+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T11:17:12.387823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797108+0000","last_change":"2024-03-07T06:53:25.181460+0000","last_active":"2024-03-07T06:53:26.797108+0000","last_peered":"2024-03-07T06:53:26.797108+0000","last_clean":"2024-03-07T06:53:26.797108+0000","last_became_active":"2024-03-07T06:53:25.181176+0000","last_became_peered":"2024-03-07T06:53:25.181176+0000","last_unstale":"2024-03-07T06:53:26.797108+0000","last_undegraded":"2024-03-07T06:53:26.797108+0000","last_fullsized":"2024-03-07T06:53:26.797108+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:31:32.369129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797278+0000","last_change":"2024-03-07T06:53:25.185869+0000","last_active":"2024-03-07T06:53:26.797278+0000","last_peered":"2024-03-07T06:53:26.797278+0000","last_clean":"2024-03-07T06:53:26.797278+0000","last_became_active":"2024-03-07T06:53:25.185574+0000","last_became_peered":"2024-03-07T06:53:25.185574+0000","last_unstale":"2024-03-07T06:53:26.797278+0000","last_undegraded":"2024-03-07T06:53:26.797278+0000","last_fullsized":"2024-03-07T06:53:26.797278+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:32.048437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":18,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569830+0000","last_change":"2024-03-07T06:53:24.190584+0000","last_active":"2024-03-07T06:53:26.569830+0000","last_peered":"2024-03-07T06:53:26.569830+0000","last_clean":"2024-03-07T06:53:26.569830+0000","last_became_active":"2024-03-07T06:53:24.190262+0000","last_became_peered":"2024-03-07T06:53:24.190262+0000","last_unstale":"2024-03-07T06:53:26.569830+0000","last_undegraded":"2024-03-07T06:53:26.569830+0000","last_fullsized":"2024-03-07T06:53:26.569830+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:53:36.898081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.196313+0000","last_change":"2024-03-07T06:53:25.195388+0000","last_active":"2024-03-07T06:53:26.196313+0000","last_peered":"2024-03-07T06:53:26.196313+0000","last_clean":"2024-03-07T06:53:26.196313+0000","last_became_active":"2024-03-07T06:53:25.192645+0000","last_became_peered":"2024-03-07T06:53:25.192645+0000","last_unstale":"2024-03-07T06:53:26.196313+0000","last_undegraded":"2024-03-07T06:53:26.196313+0000","last_fullsized":"2024-03-07T06:53:26.196313+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:12:49.521040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797418+0000","last_change":"2024-03-07T06:53:24.190898+0000","last_active":"2024-03-07T06:53:26.797418+0000","last_peered":"2024-03-07T06:53:26.797418+0000","last_clean":"2024-03-07T06:53:26.797418+0000","last_became_active":"2024-03-07T06:53:24.190216+0000","last_became_peered":"2024-03-07T06:53:24.190216+0000","last_unstale":"2024-03-07T06:53:26.797418+0000","last_undegraded":"2024-03-07T06:53:26.797418+0000","last_fullsized":"2024-03-07T06:53:26.797418+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:14:52.563314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193786+0000","last_change":"2024-03-07T06:53:25.206962+0000","last_active":"2024-03-07T06:53:26.193786+0000","last_peered":"2024-03-07T06:53:26.193786+0000","last_clean":"2024-03-07T06:53:26.193786+0000","last_became_active":"2024-03-07T06:53:25.206428+0000","last_became_peered":"2024-03-07T06:53:25.206428+0000","last_unstale":"2024-03-07T06:53:26.193786+0000","last_undegraded":"2024-03-07T06:53:26.193786+0000","last_fullsized":"2024-03-07T06:53:26.193786+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:19:59.756501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798392+0000","last_change":"2024-03-07T06:53:24.189931+0000","last_active":"2024-03-07T06:53:26.798392+0000","last_peered":"2024-03-07T06:53:26.798392+0000","last_clean":"2024-03-07T06:53:26.798392+0000","last_became_active":"2024-03-07T06:53:24.189610+0000","last_became_peered":"2024-03-07T06:53:24.189610+0000","last_unstale":"2024-03-07T06:53:26.798392+0000","last_undegraded":"2024-03-07T06:53:26.798392+0000","last_fullsized":"2024-03-07T06:53:26.798392+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:36:40.695222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798250+0000","last_change":"2024-03-07T06:53:25.190984+0000","last_active":"2024-03-07T06:53:26.798250+0000","last_peered":"2024-03-07T06:53:26.798250+0000","last_clean":"2024-03-07T06:53:26.798250+0000","last_became_active":"2024-03-07T06:53:25.190624+0000","last_became_peered":"2024-03-07T06:53:25.190624+0000","last_unstale":"2024-03-07T06:53:26.798250+0000","last_undegraded":"2024-03-07T06:53:26.798250+0000","last_fullsized":"2024-03-07T06:53:26.798250+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:36:14.943170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.189900+0000","last_change":"2024-03-07T06:53:24.192196+0000","last_active":"2024-03-07T06:53:26.189900+0000","last_peered":"2024-03-07T06:53:26.189900+0000","last_clean":"2024-03-07T06:53:26.189900+0000","last_became_active":"2024-03-07T06:53:24.191698+0000","last_became_peered":"2024-03-07T06:53:24.191698+0000","last_unstale":"2024-03-07T06:53:26.189900+0000","last_undegraded":"2024-03-07T06:53:26.189900+0000","last_fullsized":"2024-03-07T06:53:26.189900+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:45:09.141933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796861+0000","last_change":"2024-03-07T06:53:25.193044+0000","last_active":"2024-03-07T06:53:26.796861+0000","last_peered":"2024-03-07T06:53:26.796861+0000","last_clean":"2024-03-07T06:53:26.796861+0000","last_became_active":"2024-03-07T06:53:25.192696+0000","last_became_peered":"2024-03-07T06:53:25.192696+0000","last_unstale":"2024-03-07T06:53:26.796861+0000","last_undegraded":"2024-03-07T06:53:26.796861+0000","last_fullsized":"2024-03-07T06:53:26.796861+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:45:10.156912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569421+0000","last_change":"2024-03-07T06:53:24.180332+0000","last_active":"2024-03-07T06:53:26.569421+0000","last_peered":"2024-03-07T06:53:26.569421+0000","last_clean":"2024-03-07T06:53:26.569421+0000","last_became_active":"2024-03-07T06:53:24.179783+0000","last_became_peered":"2024-03-07T06:53:24.179783+0000","last_unstale":"2024-03-07T06:53:26.569421+0000","last_undegraded":"2024-03-07T06:53:26.569421+0000","last_fullsized":"2024-03-07T06:53:26.569421+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:11:03.714029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556363+0000","last_change":"2024-03-07T06:53:25.193219+0000","last_active":"2024-03-07T06:53:26.556363+0000","last_peered":"2024-03-07T06:53:26.556363+0000","last_clean":"2024-03-07T06:53:26.556363+0000","last_became_active":"2024-03-07T06:53:25.192945+0000","last_became_peered":"2024-03-07T06:53:25.192945+0000","last_unstale":"2024-03-07T06:53:26.556363+0000","last_undegraded":"2024-03-07T06:53:26.556363+0000","last_fullsized":"2024-03-07T06:53:26.556363+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:54:44.090672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797025+0000","last_change":"2024-03-07T06:53:24.186564+0000","last_active":"2024-03-07T06:53:26.797025+0000","last_peered":"2024-03-07T06:53:26.797025+0000","last_clean":"2024-03-07T06:53:26.797025+0000","last_became_active":"2024-03-07T06:53:24.184717+0000","last_became_peered":"2024-03-07T06:53:24.184717+0000","last_unstale":"2024-03-07T06:53:26.797025+0000","last_undegraded":"2024-03-07T06:53:26.797025+0000","last_fullsized":"2024-03-07T06:53:26.797025+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:07:10.818918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555572+0000","last_change":"2024-03-07T06:53:25.196813+0000","last_active":"2024-03-07T06:53:26.555572+0000","last_peered":"2024-03-07T06:53:26.555572+0000","last_clean":"2024-03-07T06:53:26.555572+0000","last_became_active":"2024-03-07T06:53:25.195979+0000","last_became_peered":"2024-03-07T06:53:25.195979+0000","last_unstale":"2024-03-07T06:53:26.555572+0000","last_undegraded":"2024-03-07T06:53:26.555572+0000","last_fullsized":"2024-03-07T06:53:26.555572+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:53:37.913060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193612+0000","last_change":"2024-03-07T06:53:24.187243+0000","last_active":"2024-03-07T06:53:26.193612+0000","last_peered":"2024-03-07T06:53:26.193612+0000","last_clean":"2024-03-07T06:53:26.193612+0000","last_became_active":"2024-03-07T06:53:24.186379+0000","last_became_peered":"2024-03-07T06:53:24.186379+0000","last_unstale":"2024-03-07T06:53:26.193612+0000","last_undegraded":"2024-03-07T06:53:26.193612+0000","last_fullsized":"2024-03-07T06:53:26.193612+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:12:48.506061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195332+0000","last_change":"2024-03-07T06:53:25.203867+0000","last_active":"2024-03-07T06:53:26.195332+0000","last_peered":"2024-03-07T06:53:26.195332+0000","last_clean":"2024-03-07T06:53:26.195332+0000","last_became_active":"2024-03-07T06:53:25.203567+0000","last_became_peered":"2024-03-07T06:53:25.203567+0000","last_unstale":"2024-03-07T06:53:26.195332+0000","last_undegraded":"2024-03-07T06:53:26.195332+0000","last_fullsized":"2024-03-07T06:53:26.195332+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:43:40.476283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555610+0000","last_change":"2024-03-07T06:53:24.176450+0000","last_active":"2024-03-07T06:53:26.555610+0000","last_peered":"2024-03-07T06:53:26.555610+0000","last_clean":"2024-03-07T06:53:26.555610+0000","last_became_active":"2024-03-07T06:53:24.175876+0000","last_became_peered":"2024-03-07T06:53:24.175876+0000","last_unstale":"2024-03-07T06:53:26.555610+0000","last_undegraded":"2024-03-07T06:53:26.555610+0000","last_fullsized":"2024-03-07T06:53:26.555610+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:20:37.347168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797243+0000","last_change":"2024-03-07T06:53:25.205883+0000","last_active":"2024-03-07T06:53:26.797243+0000","last_peered":"2024-03-07T06:53:26.797243+0000","last_clean":"2024-03-07T06:53:26.797243+0000","last_became_active":"2024-03-07T06:53:25.205193+0000","last_became_peered":"2024-03-07T06:53:25.205193+0000","last_unstale":"2024-03-07T06:53:26.797243+0000","last_undegraded":"2024-03-07T06:53:26.797243+0000","last_fullsized":"2024-03-07T06:53:26.797243+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:49:12.962971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797587+0000","last_change":"2024-03-07T06:53:24.184752+0000","last_active":"2024-03-07T06:53:26.797587+0000","last_peered":"2024-03-07T06:53:26.797587+0000","last_clean":"2024-03-07T06:53:26.797587+0000","last_became_active":"2024-03-07T06:53:24.184338+0000","last_became_peered":"2024-03-07T06:53:24.184338+0000","last_unstale":"2024-03-07T06:53:26.797587+0000","last_undegraded":"2024-03-07T06:53:26.797587+0000","last_fullsized":"2024-03-07T06:53:26.797587+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:41:06.781045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194726+0000","last_change":"2024-03-07T06:53:25.203839+0000","last_active":"2024-03-07T06:53:26.194726+0000","last_peered":"2024-03-07T06:53:26.194726+0000","last_clean":"2024-03-07T06:53:26.194726+0000","last_became_active":"2024-03-07T06:53:25.203570+0000","last_became_peered":"2024-03-07T06:53:25.203570+0000","last_unstale":"2024-03-07T06:53:26.194726+0000","last_undegraded":"2024-03-07T06:53:26.194726+0000","last_fullsized":"2024-03-07T06:53:26.194726+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:25:16.302702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556704+0000","last_change":"2024-03-07T06:53:24.180862+0000","last_active":"2024-03-07T06:53:26.556704+0000","last_peered":"2024-03-07T06:53:26.556704+0000","last_clean":"2024-03-07T06:53:26.556704+0000","last_became_active":"2024-03-07T06:53:24.180558+0000","last_became_peered":"2024-03-07T06:53:24.180558+0000","last_unstale":"2024-03-07T06:53:26.556704+0000","last_undegraded":"2024-03-07T06:53:26.556704+0000","last_fullsized":"2024-03-07T06:53:26.556704+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:23:18.942189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569862+0000","last_change":"2024-03-07T06:53:25.193872+0000","last_active":"2024-03-07T06:53:26.569862+0000","last_peered":"2024-03-07T06:53:26.569862+0000","last_clean":"2024-03-07T06:53:26.569862+0000","last_became_active":"2024-03-07T06:53:25.193677+0000","last_became_peered":"2024-03-07T06:53:25.193677+0000","last_unstale":"2024-03-07T06:53:26.569862+0000","last_undegraded":"2024-03-07T06:53:26.569862+0000","last_fullsized":"2024-03-07T06:53:26.569862+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:40:13.617392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192612+0000","last_change":"2024-03-07T06:53:24.193883+0000","last_active":"2024-03-07T06:53:26.192612+0000","last_peered":"2024-03-07T06:53:26.192612+0000","last_clean":"2024-03-07T06:53:26.192612+0000","last_became_active":"2024-03-07T06:53:24.193619+0000","last_became_peered":"2024-03-07T06:53:24.193619+0000","last_unstale":"2024-03-07T06:53:26.192612+0000","last_undegraded":"2024-03-07T06:53:26.192612+0000","last_fullsized":"2024-03-07T06:53:26.192612+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:33:46.669137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.191713+0000","last_change":"2024-03-07T06:53:25.208536+0000","last_active":"2024-03-07T06:53:26.191713+0000","last_peered":"2024-03-07T06:53:26.191713+0000","last_clean":"2024-03-07T06:53:26.191713+0000","last_became_active":"2024-03-07T06:53:25.208127+0000","last_became_peered":"2024-03-07T06:53:25.208127+0000","last_unstale":"2024-03-07T06:53:26.191713+0000","last_undegraded":"2024-03-07T06:53:26.191713+0000","last_fullsized":"2024-03-07T06:53:26.191713+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:54:32.067212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":20,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798314+0000","last_change":"2024-03-07T06:53:24.182955+0000","last_active":"2024-03-07T06:53:26.798314+0000","last_peered":"2024-03-07T06:53:26.798314+0000","last_clean":"2024-03-07T06:53:26.798314+0000","last_became_active":"2024-03-07T06:53:24.182685+0000","last_became_peered":"2024-03-07T06:53:24.182685+0000","last_unstale":"2024-03-07T06:53:26.798314+0000","last_undegraded":"2024-03-07T06:53:26.798314+0000","last_fullsized":"2024-03-07T06:53:26.798314+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:35:03.451977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194621+0000","last_change":"2024-03-07T06:53:25.190435+0000","last_active":"2024-03-07T06:53:26.194621+0000","last_peered":"2024-03-07T06:53:26.194621+0000","last_clean":"2024-03-07T06:53:26.194621+0000","last_became_active":"2024-03-07T06:53:25.189951+0000","last_became_peered":"2024-03-07T06:53:25.189951+0000","last_unstale":"2024-03-07T06:53:26.194621+0000","last_undegraded":"2024-03-07T06:53:26.194621+0000","last_fullsized":"2024-03-07T06:53:26.194621+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:14:53.578293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569710+0000","last_change":"2024-03-07T06:53:24.177161+0000","last_active":"2024-03-07T06:53:26.569710+0000","last_peered":"2024-03-07T06:53:26.569710+0000","last_clean":"2024-03-07T06:53:26.569710+0000","last_became_active":"2024-03-07T06:53:24.176716+0000","last_became_peered":"2024-03-07T06:53:24.176716+0000","last_unstale":"2024-03-07T06:53:26.569710+0000","last_undegraded":"2024-03-07T06:53:26.569710+0000","last_fullsized":"2024-03-07T06:53:26.569710+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:46:44.201464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556482+0000","last_change":"2024-03-07T06:53:25.203128+0000","last_active":"2024-03-07T06:53:26.556482+0000","last_peered":"2024-03-07T06:53:26.556482+0000","last_clean":"2024-03-07T06:53:26.556482+0000","last_became_active":"2024-03-07T06:53:25.202924+0000","last_became_peered":"2024-03-07T06:53:25.202924+0000","last_unstale":"2024-03-07T06:53:26.556482+0000","last_undegraded":"2024-03-07T06:53:26.556482+0000","last_fullsized":"2024-03-07T06:53:26.556482+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:19:59.756501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.190349+0000","last_change":"2024-03-07T06:53:24.188380+0000","last_active":"2024-03-07T06:53:26.190349+0000","last_peered":"2024-03-07T06:53:26.190349+0000","last_clean":"2024-03-07T06:53:26.190349+0000","last_became_active":"2024-03-07T06:53:24.187005+0000","last_became_peered":"2024-03-07T06:53:24.187005+0000","last_unstale":"2024-03-07T06:53:26.190349+0000","last_undegraded":"2024-03-07T06:53:26.190349+0000","last_fullsized":"2024-03-07T06:53:26.190349+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:43:50.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568329+0000","last_change":"2024-03-07T06:53:25.185049+0000","last_active":"2024-03-07T06:53:26.568329+0000","last_peered":"2024-03-07T06:53:26.568329+0000","last_clean":"2024-03-07T06:53:26.568329+0000","last_became_active":"2024-03-07T06:53:25.184192+0000","last_became_peered":"2024-03-07T06:53:25.184192+0000","last_unstale":"2024-03-07T06:53:26.568329+0000","last_undegraded":"2024-03-07T06:53:26.568329+0000","last_fullsized":"2024-03-07T06:53:26.568329+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:11:45.850119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568464+0000","last_change":"2024-03-07T06:53:24.177157+0000","last_active":"2024-03-07T06:53:26.568464+0000","last_peered":"2024-03-07T06:53:26.568464+0000","last_clean":"2024-03-07T06:53:26.568464+0000","last_became_active":"2024-03-07T06:53:24.176724+0000","last_became_peered":"2024-03-07T06:53:24.176724+0000","last_unstale":"2024-03-07T06:53:26.568464+0000","last_undegraded":"2024-03-07T06:53:26.568464+0000","last_fullsized":"2024-03-07T06:53:26.568464+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:54.259961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.190647+0000","last_change":"2024-03-07T06:53:25.194921+0000","last_active":"2024-03-07T06:53:26.190647+0000","last_peered":"2024-03-07T06:53:26.190647+0000","last_clean":"2024-03-07T06:53:26.190647+0000","last_became_active":"2024-03-07T06:53:25.193907+0000","last_became_peered":"2024-03-07T06:53:25.193907+0000","last_unstale":"2024-03-07T06:53:26.190647+0000","last_undegraded":"2024-03-07T06:53:26.190647+0000","last_fullsized":"2024-03-07T06:53:26.190647+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:21:58.317633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":22,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555716+0000","last_change":"2024-03-07T06:53:24.181056+0000","last_active":"2024-03-07T06:53:26.555716+0000","last_peered":"2024-03-07T06:53:26.555716+0000","last_clean":"2024-03-07T06:53:26.555716+0000","last_became_active":"2024-03-07T06:53:24.180668+0000","last_became_peered":"2024-03-07T06:53:24.180668+0000","last_unstale":"2024-03-07T06:53:26.555716+0000","last_undegraded":"2024-03-07T06:53:26.555716+0000","last_fullsized":"2024-03-07T06:53:26.555716+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:27:55.430918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798458+0000","last_change":"2024-03-07T06:53:25.196095+0000","last_active":"2024-03-07T06:53:26.798458+0000","last_peered":"2024-03-07T06:53:26.798458+0000","last_clean":"2024-03-07T06:53:26.798458+0000","last_became_active":"2024-03-07T06:53:25.195557+0000","last_became_peered":"2024-03-07T06:53:25.195557+0000","last_unstale":"2024-03-07T06:53:26.798458+0000","last_undegraded":"2024-03-07T06:53:26.798458+0000","last_fullsized":"2024-03-07T06:53:26.798458+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:20:38.362147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.798600+0000","last_change":"2024-03-07T06:53:24.179843+0000","last_active":"2024-03-07T06:53:26.798600+0000","last_peered":"2024-03-07T06:53:26.798600+0000","last_clean":"2024-03-07T06:53:26.798600+0000","last_became_active":"2024-03-07T06:53:24.179393+0000","last_became_peered":"2024-03-07T06:53:24.179393+0000","last_unstale":"2024-03-07T06:53:26.798600+0000","last_undegraded":"2024-03-07T06:53:26.798600+0000","last_fullsized":"2024-03-07T06:53:26.798600+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:51:11.895335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556574+0000","last_change":"2024-03-07T06:53:25.205258+0000","last_active":"2024-03-07T06:53:26.556574+0000","last_peered":"2024-03-07T06:53:26.556574+0000","last_clean":"2024-03-07T06:53:26.556574+0000","last_became_active":"2024-03-07T06:53:25.204991+0000","last_became_peered":"2024-03-07T06:53:25.204991+0000","last_unstale":"2024-03-07T06:53:26.556574+0000","last_undegraded":"2024-03-07T06:53:26.556574+0000","last_fullsized":"2024-03-07T06:53:26.556574+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T18:21:58.317633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797419+0000","last_change":"2024-03-07T06:53:24.185672+0000","last_active":"2024-03-07T06:53:26.797419+0000","last_peered":"2024-03-07T06:53:26.797419+0000","last_clean":"2024-03-07T06:53:26.797419+0000","last_became_active":"2024-03-07T06:53:24.185368+0000","last_became_peered":"2024-03-07T06:53:24.185368+0000","last_unstale":"2024-03-07T06:53:26.797419+0000","last_undegraded":"2024-03-07T06:53:26.797419+0000","last_fullsized":"2024-03-07T06:53:26.797419+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:36:40.695222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195109+0000","last_change":"2024-03-07T06:53:25.186795+0000","last_active":"2024-03-07T06:53:26.195109+0000","last_peered":"2024-03-07T06:53:26.195109+0000","last_clean":"2024-03-07T06:53:26.195109+0000","last_became_active":"2024-03-07T06:53:25.186502+0000","last_became_peered":"2024-03-07T06:53:25.186502+0000","last_unstale":"2024-03-07T06:53:26.195109+0000","last_undegraded":"2024-03-07T06:53:26.195109+0000","last_fullsized":"2024-03-07T06:53:26.195109+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:36:41.710201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193159+0000","last_change":"2024-03-07T06:53:24.186577+0000","last_active":"2024-03-07T06:53:26.193159+0000","last_peered":"2024-03-07T06:53:26.193159+0000","last_clean":"2024-03-07T06:53:26.193159+0000","last_became_active":"2024-03-07T06:53:24.186258+0000","last_became_peered":"2024-03-07T06:53:24.186258+0000","last_unstale":"2024-03-07T06:53:26.193159+0000","last_undegraded":"2024-03-07T06:53:26.193159+0000","last_fullsized":"2024-03-07T06:53:26.193159+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:51:31.033458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.195414+0000","last_change":"2024-03-07T06:53:25.189444+0000","last_active":"2024-03-07T06:53:26.195414+0000","last_peered":"2024-03-07T06:53:26.195414+0000","last_clean":"2024-03-07T06:53:26.195414+0000","last_became_active":"2024-03-07T06:53:25.187460+0000","last_became_peered":"2024-03-07T06:53:25.187460+0000","last_unstale":"2024-03-07T06:53:26.195414+0000","last_undegraded":"2024-03-07T06:53:26.195414+0000","last_fullsized":"2024-03-07T06:53:26.195414+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:28:41.703936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569599+0000","last_change":"2024-03-07T06:53:24.189372+0000","last_active":"2024-03-07T06:53:26.569599+0000","last_peered":"2024-03-07T06:53:26.569599+0000","last_clean":"2024-03-07T06:53:26.569599+0000","last_became_active":"2024-03-07T06:53:24.189061+0000","last_became_peered":"2024-03-07T06:53:24.189061+0000","last_unstale":"2024-03-07T06:53:26.569599+0000","last_undegraded":"2024-03-07T06:53:26.569599+0000","last_fullsized":"2024-03-07T06:53:26.569599+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:45:09.141933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568500+0000","last_change":"2024-03-07T06:53:25.193713+0000","last_active":"2024-03-07T06:53:26.568500+0000","last_peered":"2024-03-07T06:53:26.568500+0000","last_clean":"2024-03-07T06:53:26.568500+0000","last_became_active":"2024-03-07T06:53:25.186210+0000","last_became_peered":"2024-03-07T06:53:25.186210+0000","last_unstale":"2024-03-07T06:53:26.568500+0000","last_undegraded":"2024-03-07T06:53:26.568500+0000","last_fullsized":"2024-03-07T06:53:26.568500+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:33:15.293206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.570025+0000","last_change":"2024-03-07T06:53:24.180089+0000","last_active":"2024-03-07T06:53:26.570025+0000","last_peered":"2024-03-07T06:53:26.570025+0000","last_clean":"2024-03-07T06:53:26.570025+0000","last_became_active":"2024-03-07T06:53:24.179418+0000","last_became_peered":"2024-03-07T06:53:24.179418+0000","last_unstale":"2024-03-07T06:53:26.570025+0000","last_undegraded":"2024-03-07T06:53:26.570025+0000","last_fullsized":"2024-03-07T06:53:26.570025+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:13.505176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797763+0000","last_change":"2024-03-07T06:53:25.196193+0000","last_active":"2024-03-07T06:53:26.797763+0000","last_peered":"2024-03-07T06:53:26.797763+0000","last_clean":"2024-03-07T06:53:26.797763+0000","last_became_active":"2024-03-07T06:53:25.195898+0000","last_became_peered":"2024-03-07T06:53:25.195898+0000","last_unstale":"2024-03-07T06:53:26.797763+0000","last_undegraded":"2024-03-07T06:53:26.797763+0000","last_fullsized":"2024-03-07T06:53:26.797763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:12:49.521040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193010+0000","last_change":"2024-03-07T06:53:24.191253+0000","last_active":"2024-03-07T06:53:26.193010+0000","last_peered":"2024-03-07T06:53:26.193010+0000","last_clean":"2024-03-07T06:53:26.193010+0000","last_became_active":"2024-03-07T06:53:24.190919+0000","last_became_peered":"2024-03-07T06:53:24.190919+0000","last_unstale":"2024-03-07T06:53:26.193010+0000","last_undegraded":"2024-03-07T06:53:26.193010+0000","last_fullsized":"2024-03-07T06:53:26.193010+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:43:50.379428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797624+0000","last_change":"2024-03-07T06:53:25.202509+0000","last_active":"2024-03-07T06:53:26.797624+0000","last_peered":"2024-03-07T06:53:26.797624+0000","last_clean":"2024-03-07T06:53:26.797624+0000","last_became_active":"2024-03-07T06:53:25.202242+0000","last_became_peered":"2024-03-07T06:53:25.202242+0000","last_unstale":"2024-03-07T06:53:26.797624+0000","last_undegraded":"2024-03-07T06:53:26.797624+0000","last_fullsized":"2024-03-07T06:53:26.797624+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:01:23.009414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555290+0000","last_change":"2024-03-07T06:53:24.181121+0000","last_active":"2024-03-07T06:53:26.555290+0000","last_peered":"2024-03-07T06:53:26.555290+0000","last_clean":"2024-03-07T06:53:26.555290+0000","last_became_active":"2024-03-07T06:53:24.180669+0000","last_became_peered":"2024-03-07T06:53:24.180669+0000","last_unstale":"2024-03-07T06:53:26.555290+0000","last_undegraded":"2024-03-07T06:53:26.555290+0000","last_fullsized":"2024-03-07T06:53:26.555290+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:24:07.355696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555633+0000","last_change":"2024-03-07T06:53:25.177613+0000","last_active":"2024-03-07T06:53:26.555633+0000","last_peered":"2024-03-07T06:53:26.555633+0000","last_clean":"2024-03-07T06:53:26.555633+0000","last_became_active":"2024-03-07T06:53:25.177320+0000","last_became_peered":"2024-03-07T06:53:25.177320+0000","last_unstale":"2024-03-07T06:53:26.555633+0000","last_undegraded":"2024-03-07T06:53:26.555633+0000","last_fullsized":"2024-03-07T06:53:26.555633+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:45:10.156912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.189619+0000","last_change":"2024-03-07T06:53:24.184705+0000","last_active":"2024-03-07T06:53:26.189619+0000","last_peered":"2024-03-07T06:53:26.189619+0000","last_clean":"2024-03-07T06:53:26.189619+0000","last_became_active":"2024-03-07T06:53:24.184354+0000","last_became_peered":"2024-03-07T06:53:24.184354+0000","last_unstale":"2024-03-07T06:53:26.189619+0000","last_undegraded":"2024-03-07T06:53:26.189619+0000","last_fullsized":"2024-03-07T06:53:26.189619+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:10:37.281189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.196424+0000","last_change":"2024-03-07T06:53:25.204962+0000","last_active":"2024-03-07T06:53:26.196424+0000","last_peered":"2024-03-07T06:53:26.196424+0000","last_clean":"2024-03-07T06:53:26.196424+0000","last_became_active":"2024-03-07T06:53:25.203898+0000","last_became_peered":"2024-03-07T06:53:25.203898+0000","last_unstale":"2024-03-07T06:53:26.196424+0000","last_undegraded":"2024-03-07T06:53:26.196424+0000","last_fullsized":"2024-03-07T06:53:26.196424+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:36:14.943170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"46'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569903+0000","last_change":"2024-03-07T06:53:24.182347+0000","last_active":"2024-03-07T06:53:26.569903+0000","last_peered":"2024-03-07T06:53:26.569903+0000","last_clean":"2024-03-07T06:53:26.569903+0000","last_became_active":"2024-03-07T06:53:24.182031+0000","last_became_peered":"2024-03-07T06:53:24.182031+0000","last_unstale":"2024-03-07T06:53:26.569903+0000","last_undegraded":"2024-03-07T06:53:26.569903+0000","last_fullsized":"2024-03-07T06:53:26.569903+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T16:33:46.669137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556121+0000","last_change":"2024-03-07T06:53:25.184985+0000","last_active":"2024-03-07T06:53:26.556121+0000","last_peered":"2024-03-07T06:53:26.556121+0000","last_clean":"2024-03-07T06:53:26.556121+0000","last_became_active":"2024-03-07T06:53:25.184052+0000","last_became_peered":"2024-03-07T06:53:25.184052+0000","last_unstale":"2024-03-07T06:53:26.556121+0000","last_undegraded":"2024-03-07T06:53:26.556121+0000","last_fullsized":"2024-03-07T06:53:26.556121+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T12:49:12.962971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192906+0000","last_change":"2024-03-07T06:53:24.194031+0000","last_active":"2024-03-07T06:53:26.192906+0000","last_peered":"2024-03-07T06:53:26.192906+0000","last_clean":"2024-03-07T06:53:26.192906+0000","last_became_active":"2024-03-07T06:53:24.193718+0000","last_became_peered":"2024-03-07T06:53:24.193718+0000","last_unstale":"2024-03-07T06:53:26.192906+0000","last_undegraded":"2024-03-07T06:53:26.192906+0000","last_fullsized":"2024-03-07T06:53:26.192906+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:53:36.898081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.193172+0000","last_change":"2024-03-07T06:53:25.202526+0000","last_active":"2024-03-07T06:53:26.193172+0000","last_peered":"2024-03-07T06:53:26.193172+0000","last_clean":"2024-03-07T06:53:26.193172+0000","last_became_active":"2024-03-07T06:53:25.202243+0000","last_became_peered":"2024-03-07T06:53:25.202243+0000","last_unstale":"2024-03-07T06:53:26.193172+0000","last_undegraded":"2024-03-07T06:53:26.193172+0000","last_fullsized":"2024-03-07T06:53:26.193172+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:55.587418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796596+0000","last_change":"2024-03-07T06:53:24.183011+0000","last_active":"2024-03-07T06:53:26.796596+0000","last_peered":"2024-03-07T06:53:26.796596+0000","last_clean":"2024-03-07T06:53:26.796596+0000","last_became_active":"2024-03-07T06:53:24.182772+0000","last_became_peered":"2024-03-07T06:53:24.182772+0000","last_unstale":"2024-03-07T06:53:26.796596+0000","last_undegraded":"2024-03-07T06:53:26.796596+0000","last_fullsized":"2024-03-07T06:53:26.796596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T13:14:52.563314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.568643+0000","last_change":"2024-03-07T06:53:25.173467+0000","last_active":"2024-03-07T06:53:26.568643+0000","last_peered":"2024-03-07T06:53:26.568643+0000","last_clean":"2024-03-07T06:53:26.568643+0000","last_became_active":"2024-03-07T06:53:25.173157+0000","last_became_peered":"2024-03-07T06:53:25.173157+0000","last_unstale":"2024-03-07T06:53:26.568643+0000","last_undegraded":"2024-03-07T06:53:26.568643+0000","last_fullsized":"2024-03-07T06:53:26.568643+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:04:56.048641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.191151+0000","last_change":"2024-03-07T06:53:24.196702+0000","last_active":"2024-03-07T06:53:26.191151+0000","last_peered":"2024-03-07T06:53:26.191151+0000","last_clean":"2024-03-07T06:53:26.191151+0000","last_became_active":"2024-03-07T06:53:24.196356+0000","last_became_peered":"2024-03-07T06:53:24.196356+0000","last_unstale":"2024-03-07T06:53:26.191151+0000","last_undegraded":"2024-03-07T06:53:26.191151+0000","last_fullsized":"2024-03-07T06:53:26.191151+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T09:17:54.259961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.569797+0000","last_change":"2024-03-07T06:53:25.190375+0000","last_active":"2024-03-07T06:53:26.569797+0000","last_peered":"2024-03-07T06:53:26.569797+0000","last_clean":"2024-03-07T06:53:26.569797+0000","last_became_active":"2024-03-07T06:53:25.189627+0000","last_became_peered":"2024-03-07T06:53:25.189627+0000","last_unstale":"2024-03-07T06:53:26.569797+0000","last_undegraded":"2024-03-07T06:53:26.569797+0000","last_fullsized":"2024-03-07T06:53:26.569797+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:39:28.721920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.797059+0000","last_change":"2024-03-07T06:53:24.180383+0000","last_active":"2024-03-07T06:53:26.797059+0000","last_peered":"2024-03-07T06:53:26.797059+0000","last_clean":"2024-03-07T06:53:26.797059+0000","last_became_active":"2024-03-07T06:53:24.180057+0000","last_became_peered":"2024-03-07T06:53:24.180057+0000","last_unstale":"2024-03-07T06:53:26.797059+0000","last_undegraded":"2024-03-07T06:53:26.797059+0000","last_fullsized":"2024-03-07T06:53:26.797059+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:43:39.461304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.192907+0000","last_change":"2024-03-07T06:53:25.195366+0000","last_active":"2024-03-07T06:53:26.192907+0000","last_peered":"2024-03-07T06:53:26.192907+0000","last_clean":"2024-03-07T06:53:26.192907+0000","last_became_active":"2024-03-07T06:53:25.194848+0000","last_became_peered":"2024-03-07T06:53:25.194848+0000","last_unstale":"2024-03-07T06:53:26.192907+0000","last_undegraded":"2024-03-07T06:53:26.192907+0000","last_fullsized":"2024-03-07T06:53:26.192907+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:46:45.216443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796937+0000","last_change":"2024-03-07T06:53:24.185042+0000","last_active":"2024-03-07T06:53:26.796937+0000","last_peered":"2024-03-07T06:53:26.796937+0000","last_clean":"2024-03-07T06:53:26.796937+0000","last_became_active":"2024-03-07T06:53:24.184633+0000","last_became_peered":"2024-03-07T06:53:24.184633+0000","last_unstale":"2024-03-07T06:53:26.796937+0000","last_undegraded":"2024-03-07T06:53:26.796937+0000","last_fullsized":"2024-03-07T06:53:26.796937+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T10:09:35.550886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.796752+0000","last_change":"2024-03-07T06:53:25.186793+0000","last_active":"2024-03-07T06:53:26.796752+0000","last_peered":"2024-03-07T06:53:26.796752+0000","last_clean":"2024-03-07T06:53:26.796752+0000","last_became_active":"2024-03-07T06:53:25.182526+0000","last_became_peered":"2024-03-07T06:53:25.182526+0000","last_unstale":"2024-03-07T06:53:26.796752+0000","last_undegraded":"2024-03-07T06:53:26.796752+0000","last_fullsized":"2024-03-07T06:53:26.796752+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:53:37.913060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.555749+0000","last_change":"2024-03-07T06:53:24.176113+0000","last_active":"2024-03-07T06:53:26.555749+0000","last_peered":"2024-03-07T06:53:26.555749+0000","last_clean":"2024-03-07T06:53:26.555749+0000","last_became_active":"2024-03-07T06:53:24.175506+0000","last_became_peered":"2024-03-07T06:53:24.175506+0000","last_unstale":"2024-03-07T06:53:26.555749+0000","last_undegraded":"2024-03-07T06:53:26.555749+0000","last_fullsized":"2024-03-07T06:53:26.555749+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T08:20:28.607718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.556710+0000","last_change":"2024-03-07T06:53:25.203466+0000","last_active":"2024-03-07T06:53:26.556710+0000","last_peered":"2024-03-07T06:53:26.556710+0000","last_clean":"2024-03-07T06:53:26.556710+0000","last_became_active":"2024-03-07T06:53:25.203113+0000","last_became_peered":"2024-03-07T06:53:25.203113+0000","last_unstale":"2024-03-07T06:53:26.556710+0000","last_undegraded":"2024-03-07T06:53:26.556710+0000","last_fullsized":"2024-03-07T06:53:26.556710+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:24.135186+0000","last_clean_scrub_stamp":"2024-03-07T06:53:24.135186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T15:35:55.587418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.194351+0000","last_change":"2024-03-07T06:53:24.186782+0000","last_active":"2024-03-07T06:53:26.194351+0000","last_peered":"2024-03-07T06:53:26.194351+0000","last_clean":"2024-03-07T06:53:26.194351+0000","last_became_active":"2024-03-07T06:53:24.186396+0000","last_became_peered":"2024-03-07T06:53:24.186396+0000","last_unstale":"2024-03-07T06:53:26.194351+0000","last_undegraded":"2024-03-07T06:53:26.194351+0000","last_fullsized":"2024-03-07T06:53:26.194351+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:53:23.120207+0000","last_clean_scrub_stamp":"2024-03-07T06:53:23.120207+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T07:36:13.928191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-07T06:53:26.189635+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:53:26.189635+0000","last_peered":"2024-03-07T06:53:26.189635+0000","last_clean":"2024-03-07T06:53:26.189635+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:53:26.189635+0000","last_undegraded":"2024-03-07T06:53:26.189635+0000","last_fullsized":"2024-03-07T06:53:26.189635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":810252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31720,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":548864,"data_stored":220466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":219864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":552960,"data_stored":221406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31724,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":552960,"data_stored":220432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":557056,"data_stored":221428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31716,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340124,"statfs":{"total":96636764160,"available":96604286976,"internally_reserved":0,"allocated":544768,"data_stored":219864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1138688,"data_stored":810196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:53:35.501 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-07T06:53:35.502 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-07T06:53:35.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.750+0000 7faaf2e00640 1 Processor -- start 2024-03-07T06:53:35.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.751+0000 7faaf2e00640 1 -- start start 2024-03-07T06:53:35.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.753+0000 7faaf2e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faaec0b6600 0x7faaec0d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:35.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.753+0000 7faaf2e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaec0cf1f0 con 0x7faaec0b6150 2024-03-07T06:53:35.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.753+0000 7faaf2e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaec0cfa50 con 0x7faaec158800 2024-03-07T06:53:35.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.753+0000 7faaf2e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaec0d2870 con 0x7faaec0b6600 2024-03-07T06:53:35.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.753+0000 7faae3e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7faaec0b6150 0x7faaec0d2d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47468/0 (socket says 172.21.15.133:47468) 2024-03-07T06:53:35.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.753+0000 7faae3e00640 1 -- 172.21.15.133:0/313761244 learned_addr learned my addr 172.21.15.133:0/313761244 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:35.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.753+0000 7faaf0e00640 1 --2- 172.21.15.133:0/313761244 >> v2:172.21.15.133:3300/0 conn(0x7faaec0b6600 0x7faaec0d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:35.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.754+0000 7faaf0e00640 1 -- 172.21.15.133:0/313761244 >> v1:172.21.15.161:6789/0 conn(0x7faaec158800 legacy=0x7faaec15f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:35.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.754+0000 7faaf0e00640 1 -- 172.21.15.133:0/313761244 >> v1:172.21.15.133:6789/0 conn(0x7faaec0b6150 legacy=0x7faaec0d2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:35.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.754+0000 7faaeb400640 1 -- 172.21.15.133:0/313761244 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195158447 0 0) 0x7faaec0cf1f0 con 0x7faaec0b6150 2024-03-07T06:53:35.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.754+0000 7faaf0e00640 1 -- 172.21.15.133:0/313761244 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec0b6a50 con 0x7faaec0b6600 2024-03-07T06:53:35.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.755+0000 7faaf0e00640 1 --2- 172.21.15.133:0/313761244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaec0b6600 0x7faaec0d3410 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7faaec0d27e0 tx=0x7faae402be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66328774e5ed763f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:35.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.755+0000 7faaeb400640 1 -- 172.21.15.133:0/313761244 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae4039070 con 0x7faaec0b6600 2024-03-07T06:53:35.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.755+0000 7faaeb400640 1 -- 172.21.15.133:0/313761244 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae402cc30 con 0x7faaec0b6600 2024-03-07T06:53:35.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.755+0000 7faaeb400640 1 -- 172.21.15.133:0/313761244 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae4034440 con 0x7faaec0b6600 2024-03-07T06:53:35.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.756+0000 7faaf2e00640 1 -- 172.21.15.133:0/313761244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaec0b6600 msgr2=0x7faaec0d3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:35.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.756+0000 7faaf2e00640 1 --2- 172.21.15.133:0/313761244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaec0b6600 0x7faaec0d3410 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7faaec0d27e0 tx=0x7faae402be80 comp rx=0 tx=0).stop 2024-03-07T06:53:35.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.756+0000 7faaf2e00640 1 -- 172.21.15.133:0/313761244 shutdown_connections 2024-03-07T06:53:35.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.756+0000 7faaf2e00640 1 --2- 172.21.15.133:0/313761244 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaec0b6600 0x7faaec0d3410 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:35.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.756+0000 7faaf2e00640 1 -- 172.21.15.133:0/313761244 >> 172.21.15.133:0/313761244 conn(0x7faaec0d1060 msgr2=0x7faaec0d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:35.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.756+0000 7faaf2e00640 1 -- 172.21.15.133:0/313761244 shutdown_connections 2024-03-07T06:53:35.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.756+0000 7faaf2e00640 1 -- 172.21.15.133:0/313761244 wait complete. 2024-03-07T06:53:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.757+0000 7faaf2e00640 1 Processor -- start 2024-03-07T06:53:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.758+0000 7faaf2e00640 1 -- start start 2024-03-07T06:53:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.758+0000 7faaf2e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaec0b6150 0x7faaec1548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.758+0000 7faaf2e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaec0b6600 0x7faaec158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.758+0000 7faaf2e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaec158800 0x7faaec14d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.758+0000 7faaf2e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec0d2500 con 0x7faaec158800 2024-03-07T06:53:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.758+0000 7faaf2e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaec0cf770 con 0x7faaec0b6600 2024-03-07T06:53:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.758+0000 7faaf2e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaec158be0 con 0x7faaec0b6150 2024-03-07T06:53:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.759+0000 7faaf0e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaec158800 0x7faaec14d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.759+0000 7faaf0e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaec158800 0x7faaec14d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35834/0 (socket says 172.21.15.133:35834) 2024-03-07T06:53:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.759+0000 7faaf0e00640 1 -- 172.21.15.133:0/3335634593 learned_addr learned my addr 172.21.15.133:0/3335634593 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.759+0000 7faaebe00640 1 --2- 172.21.15.133:0/3335634593 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaec0b6150 0x7faaec1548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:35.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.759+0000 7faae3e00640 1 --2- 172.21.15.133:0/3335634593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaec0b6600 0x7faaec158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.759+0000 7faaf0e00640 1 -- 172.21.15.133:0/3335634593 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaec0b6150 msgr2=0x7faaec1548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.759+0000 7faaf0e00640 1 --2- 172.21.15.133:0/3335634593 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaec0b6150 0x7faaec1548f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.759+0000 7faaf0e00640 1 -- 172.21.15.133:0/3335634593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaec0b6600 msgr2=0x7faaec158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.759+0000 7faaf0e00640 1 --2- 172.21.15.133:0/3335634593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaec0b6600 0x7faaec158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.759+0000 7faaf0e00640 1 -- 172.21.15.133:0/3335634593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec159300 con 0x7faaec158800 2024-03-07T06:53:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.759+0000 7faaf0e00640 1 --2- 172.21.15.133:0/3335634593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaec158800 0x7faaec14d370 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7faae402c490 tx=0x7faae4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.760+0000 7faae8c00640 1 -- 172.21.15.133:0/3335634593 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae4044030 con 0x7faaec158800 2024-03-07T06:53:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.760+0000 7faae8c00640 1 -- 172.21.15.133:0/3335634593 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae402ca30 con 0x7faaec158800 2024-03-07T06:53:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.760+0000 7faae8c00640 1 -- 172.21.15.133:0/3335634593 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae4039040 con 0x7faaec158800 2024-03-07T06:53:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.760+0000 7faaf2e00640 1 -- 172.21.15.133:0/3335634593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaec159580 con 0x7faaec158800 2024-03-07T06:53:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.760+0000 7faaf2e00640 1 -- 172.21.15.133:0/3335634593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaec159b40 con 0x7faaec158800 2024-03-07T06:53:35.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.761+0000 7faaf2e00640 1 -- 172.21.15.133:0/3335634593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaec062c90 con 0x7faaec158800 2024-03-07T06:53:35.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.762+0000 7faae8c00640 1 -- 172.21.15.133:0/3335634593 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7faae4047070 con 0x7faaec158800 2024-03-07T06:53:35.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.763+0000 7faae8c00640 1 --2- 172.21.15.133:0/3335634593 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7faac806b4d0 0x7faac806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:35.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.763+0000 7faae8c00640 1 -- 172.21.15.133:0/3335634593 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faae40acc00 con 0x7faaec158800 2024-03-07T06:53:35.763 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.764+0000 7faaebe00640 1 --2- 172.21.15.133:0/3335634593 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7faac806b4d0 0x7faac806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:35.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.765+0000 7faaebe00640 1 --2- 172.21.15.133:0/3335634593 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7faac806b4d0 0x7faac806d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faadc006fd0 tx=0x7faadc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:35.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:35.767+0000 7faae8c00640 1 -- 172.21.15.133:0/3335634593 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faae404e070 con 0x7faaec158800 2024-03-07T06:53:36.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.014+0000 7faaf2e00640 1 -- 172.21.15.133:0/3335634593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7faaec068180 con 0x7faaec158800 2024-03-07T06:53:36.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.019+0000 7faae8c00640 1 -- 172.21.15.133:0/3335634593 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7faaec159300 con 0x7faaec158800 2024-03-07T06:53:36.019 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-03-07T06:53:36.019 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQCbY+ll6xWENRAArKbw8ID7PLMFVCO3UlKDOg== 2024-03-07T06:53:36.019 INFO:teuthology.orchestra.run.smithi133.stdout: caps mds = "allow" 2024-03-07T06:53:36.019 INFO:teuthology.orchestra.run.smithi133.stdout: caps mon = "allow r" 2024-03-07T06:53:36.019 INFO:teuthology.orchestra.run.smithi133.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-07T06:53:36.019 INFO:teuthology.orchestra.run.smithi133.stderr:updated caps for client.0 2024-03-07T06:53:36.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.021+0000 7faaf2e00640 1 -- 172.21.15.133:0/3335634593 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7faac806b4d0 msgr2=0x7faac806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:36.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.021+0000 7faaf2e00640 1 --2- 172.21.15.133:0/3335634593 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7faac806b4d0 0x7faac806d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faadc006fd0 tx=0x7faadc008040 comp rx=0 tx=0).stop 2024-03-07T06:53:36.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.021+0000 7faaf2e00640 1 -- 172.21.15.133:0/3335634593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaec158800 msgr2=0x7faaec14d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:36.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.021+0000 7faaf2e00640 1 --2- 172.21.15.133:0/3335634593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaec158800 0x7faaec14d370 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7faae402c490 tx=0x7faae4004a40 comp rx=0 tx=0).stop 2024-03-07T06:53:36.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.021+0000 7faaf2e00640 1 -- 172.21.15.133:0/3335634593 shutdown_connections 2024-03-07T06:53:36.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.021+0000 7faaf2e00640 1 --2- 172.21.15.133:0/3335634593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faaec158800 0x7faaec14d370 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:36.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.021+0000 7faaf2e00640 1 --2- 172.21.15.133:0/3335634593 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7faac806b4d0 0x7faac806d990 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:36.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.021+0000 7faaf2e00640 1 --2- 172.21.15.133:0/3335634593 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faaec0b6600 0x7faaec158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:36.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.021+0000 7faaf2e00640 1 --2- 172.21.15.133:0/3335634593 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faaec0b6150 0x7faaec1548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:36.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.021+0000 7faaf2e00640 1 -- 172.21.15.133:0/3335634593 >> 172.21.15.133:0/3335634593 conn(0x7faaec0d1060 msgr2=0x7faaec0b8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:36.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.022+0000 7faaf2e00640 1 -- 172.21.15.133:0/3335634593 shutdown_connections 2024-03-07T06:53:36.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.022+0000 7faaf2e00640 1 -- 172.21.15.133:0/3335634593 wait complete. 2024-03-07T06:53:36.041 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:53:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.251+0000 7f2124200640 1 Processor -- start 2024-03-07T06:53:36.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.251+0000 7f2124200640 1 -- start start 2024-03-07T06:53:36.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.253+0000 7f2124200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f211c0b5090 0x7f211c072910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:36.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.253+0000 7f2124200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f211c0d1260 0x7f211c072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:36.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.253+0000 7f2124200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f211c15fc30 con 0x7f211c147bb0 2024-03-07T06:53:36.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.253+0000 7f2124200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211c161ce0 con 0x7f211c0d1260 2024-03-07T06:53:36.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.253+0000 7f2124200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c15ff10 con 0x7f211c0b5090 2024-03-07T06:53:36.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.253+0000 7f2122200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f211c0d1260 0x7f211c072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:36.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.254+0000 7f2122200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f211c0d1260 0x7f211c072e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35848/0 (socket says 172.21.15.133:35848) 2024-03-07T06:53:36.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.254+0000 7f2122200640 1 -- 172.21.15.133:0/1640435340 learned_addr learned my addr 172.21.15.133:0/1640435340 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:36.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.254+0000 7f2120e00640 1 --2- 172.21.15.133:0/1640435340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f211c0b5090 0x7f211c072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:36.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.254+0000 7f2122200640 1 -- 172.21.15.133:0/1640435340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f211c0b5090 msgr2=0x7f211c072910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:36.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.254+0000 7f2122200640 1 --2- 172.21.15.133:0/1640435340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f211c0b5090 0x7f211c072910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:36.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.254+0000 7f2122200640 1 -- 172.21.15.133:0/1640435340 >> v1:172.21.15.161:6789/0 conn(0x7f211c147bb0 legacy=0x7f211c0d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:36.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.254+0000 7f2122200640 1 -- 172.21.15.133:0/1640435340 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c1601f0 con 0x7f211c0d1260 2024-03-07T06:53:36.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.256+0000 7f2122200640 1 --2- 172.21.15.133:0/1640435340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211c0d1260 0x7f211c072e50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2114009e50 tx=0x7f211402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0566bd11229cf12 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:36.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.256+0000 7f211be00640 1 -- 172.21.15.133:0/1640435340 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211402cb00 con 0x7f211c0d1260 2024-03-07T06:53:36.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.256+0000 7f211be00640 1 -- 172.21.15.133:0/1640435340 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211402cca0 con 0x7f211c0d1260 2024-03-07T06:53:36.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.256+0000 7f211be00640 1 -- 172.21.15.133:0/1640435340 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114034610 con 0x7f211c0d1260 2024-03-07T06:53:36.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.257+0000 7f2124200640 1 -- 172.21.15.133:0/1640435340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211c0d1260 msgr2=0x7f211c072e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:36.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.257+0000 7f2124200640 1 --2- 172.21.15.133:0/1640435340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211c0d1260 0x7f211c072e50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2114009e50 tx=0x7f211402bf60 comp rx=0 tx=0).stop 2024-03-07T06:53:36.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.257+0000 7f2124200640 1 -- 172.21.15.133:0/1640435340 shutdown_connections 2024-03-07T06:53:36.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.257+0000 7f2124200640 1 --2- 172.21.15.133:0/1640435340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211c0d1260 0x7f211c072e50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:36.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.257+0000 7f2124200640 1 --2- 172.21.15.133:0/1640435340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f211c0b5090 0x7f211c072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:36.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.257+0000 7f2124200640 1 -- 172.21.15.133:0/1640435340 >> 172.21.15.133:0/1640435340 conn(0x7f211c06b480 msgr2=0x7f211c06b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:36.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.257+0000 7f2124200640 1 -- 172.21.15.133:0/1640435340 shutdown_connections 2024-03-07T06:53:36.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.257+0000 7f2124200640 1 -- 172.21.15.133:0/1640435340 wait complete. 2024-03-07T06:53:36.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.259+0000 7f2124200640 1 Processor -- start 2024-03-07T06:53:36.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.259+0000 7f2124200640 1 -- start start 2024-03-07T06:53:36.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.260+0000 7f2124200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211c0b5090 0x7f211c15a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:36.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.260+0000 7f2124200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f211c0d1260 0x7f211c0b2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:36.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.260+0000 7f2124200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f211c147bb0 0x7f211c0b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:36.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.260+0000 7f2124200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c147f90 con 0x7f211c0b5090 2024-03-07T06:53:36.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.260+0000 7f2124200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c1607b0 con 0x7f211c0d1260 2024-03-07T06:53:36.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.260+0000 7f2124200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c160cd0 con 0x7f211c147bb0 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.260+0000 7f2121800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211c0b5090 0x7f211c15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.260+0000 7f2121800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211c0b5090 0x7f211c15a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35862/0 (socket says 172.21.15.133:35862) 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.260+0000 7f2121800640 1 -- 172.21.15.133:0/3909216920 learned_addr learned my addr 172.21.15.133:0/3909216920 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.261+0000 7f2120e00640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f211c0d1260 0x7f211c0b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.261+0000 7f2122200640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f211c147bb0 0x7f211c0b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.261+0000 7f2121800640 1 -- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f211c147bb0 msgr2=0x7f211c0b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.261+0000 7f2121800640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f211c147bb0 0x7f211c0b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.261+0000 7f2121800640 1 -- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f211c0d1260 msgr2=0x7f211c0b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.261+0000 7f2121800640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f211c0d1260 0x7f211c0b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.261+0000 7f2121800640 1 -- 172.21.15.133:0/3909216920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c13fbb0 con 0x7f211c0b5090 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.261+0000 7f2122200640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f211c147bb0 0x7f211c0b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.261+0000 7f2121800640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211c0b5090 0x7f211c15a860 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f211000c3c0 tx=0x7f211000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.262+0000 7f2119600640 1 -- 172.21.15.133:0/3909216920 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21100052d0 con 0x7f211c0b5090 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.262+0000 7f2119600640 1 -- 172.21.15.133:0/3909216920 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2110005490 con 0x7f211c0b5090 2024-03-07T06:53:36.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.262+0000 7f2124200640 1 -- 172.21.15.133:0/3909216920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c13fe20 con 0x7f211c0b5090 2024-03-07T06:53:36.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.262+0000 7f2119600640 1 -- 172.21.15.133:0/3909216920 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110007260 con 0x7f211c0b5090 2024-03-07T06:53:36.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.262+0000 7f2124200640 1 -- 172.21.15.133:0/3909216920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c1403e0 con 0x7f211c0b5090 2024-03-07T06:53:36.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.263+0000 7f2124200640 1 -- 172.21.15.133:0/3909216920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211c062c90 con 0x7f211c0b5090 2024-03-07T06:53:36.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.264+0000 7f2119600640 1 -- 172.21.15.133:0/3909216920 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2110007400 con 0x7f211c0b5090 2024-03-07T06:53:36.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.265+0000 7f2119600640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f20fc06b4d0 0x7f20fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:36.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.266+0000 7f2119600640 1 -- 172.21.15.133:0/3909216920 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2110089770 con 0x7f211c0b5090 2024-03-07T06:53:36.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.266+0000 7f2120e00640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f20fc06b4d0 0x7f20fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:36.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.267+0000 7f2120e00640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f20fc06b4d0 0x7f20fc06d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f211c0b7740 tx=0x7f2108008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:36.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.269+0000 7f2119600640 1 -- 172.21.15.133:0/3909216920 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2110058560 con 0x7f211c0b5090 2024-03-07T06:53:36.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.513+0000 7f2124200640 1 -- 172.21.15.133:0/3909216920 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f211c0b1a60 con 0x7f211c0b5090 2024-03-07T06:53:36.514 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.514+0000 7f2119600640 1 -- 172.21.15.133:0/3909216920 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4980 (secure 0 0 0) 0x7f2110057e60 con 0x7f211c0b5090 2024-03-07T06:53:36.514 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:36.514 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/44975892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":44975892},{"type":"v1","addr":"172.21.15.161:6835","nonce":44975892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4972,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/906115542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":906115542},{"type":"v1","addr":"172.21.15.133:6833","nonce":906115542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1998424684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1998424684},{"type":"v1","addr":"172.21.15.161:6837","nonce":1998424684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:53:24.627534+0000","modified":"2024-03-07T06:53:27.698471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/2951618124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2951618124},{"type":"v1","addr":"172.21.15.133:6835","nonce":2951618124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":3}]} 2024-03-07T06:53:36.514 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 27 2024-03-07T06:53:36.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.517+0000 7f2124200640 1 -- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f20fc06b4d0 msgr2=0x7f20fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:36.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.517+0000 7f2124200640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f20fc06b4d0 0x7f20fc06d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f211c0b7740 tx=0x7f2108008040 comp rx=0 tx=0).stop 2024-03-07T06:53:36.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.517+0000 7f2124200640 1 -- 172.21.15.133:0/3909216920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211c0b5090 msgr2=0x7f211c15a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:36.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.517+0000 7f2124200640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211c0b5090 0x7f211c15a860 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f211000c3c0 tx=0x7f211000c880 comp rx=0 tx=0).stop 2024-03-07T06:53:36.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.517+0000 7f2124200640 1 -- 172.21.15.133:0/3909216920 shutdown_connections 2024-03-07T06:53:36.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.517+0000 7f2124200640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f211c147bb0 0x7f211c0b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:36.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.517+0000 7f2124200640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f20fc06b4d0 0x7f20fc06d990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:36.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.517+0000 7f2124200640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f211c0d1260 0x7f211c0b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:36.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.517+0000 7f2124200640 1 --2- 172.21.15.133:0/3909216920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f211c0b5090 0x7f211c15a860 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:36.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.517+0000 7f2124200640 1 -- 172.21.15.133:0/3909216920 >> 172.21.15.133:0/3909216920 conn(0x7f211c06b480 msgr2=0x7f211c1610e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:36.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.517+0000 7f2124200640 1 -- 172.21.15.133:0/3909216920 shutdown_connections 2024-03-07T06:53:36.517 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:36.517+0000 7f2124200640 1 -- 172.21.15.133:0/3909216920 wait complete. 2024-03-07T06:53:36.938 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-07T06:53:24.627534+0000', 'modified': '2024-03-07T06:53:27.698471+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5580}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5580': {'gid': 5580, 'name': 'a', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6835/2951618124', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 2951618124}, {'type': 'v1', 'addr': '172.21.15.133:6835', 'nonce': 2951618124}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5580, 'qdb_cluster': [5580]} 2024-03-07T06:53:36.938 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-07T06:53:36.938 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-07T06:53:37.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.147+0000 7f4755600640 1 Processor -- start 2024-03-07T06:53:37.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.147+0000 7f4755600640 1 -- start start 2024-03-07T06:53:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.149+0000 7f4755600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f475015fbb0 0x7f47500cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.149+0000 7f4755600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4750072950 0x7f47500cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.149+0000 7f4755600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47500731b0 con 0x7f475015ff90 2024-03-07T06:53:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.149+0000 7f4755600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4750072f60 con 0x7f4750072950 2024-03-07T06:53:37.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.149+0000 7f4755600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4750073490 con 0x7f475015fbb0 2024-03-07T06:53:37.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.149+0000 7f474e000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f4750072950 0x7f47500cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.149+0000 7f474f400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f475015ff90 0x7f47500acd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47474/0 (socket says 172.21.15.133:47474) 2024-03-07T06:53:37.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.149+0000 7f474f400640 1 -- 172.21.15.133:0/500410627 learned_addr learned my addr 172.21.15.133:0/500410627 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:37.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.149+0000 7f474ea00640 1 --2- 172.21.15.133:0/500410627 >> v2:172.21.15.161:3300/0 conn(0x7f475015fbb0 0x7f47500cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.150+0000 7f474e000640 1 -- 172.21.15.133:0/500410627 >> v2:172.21.15.161:3300/0 conn(0x7f475015fbb0 msgr2=0x7f47500cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:37.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.150+0000 7f474e000640 1 --2- 172.21.15.133:0/500410627 >> v2:172.21.15.161:3300/0 conn(0x7f475015fbb0 0x7f47500cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.150+0000 7f474e000640 1 -- 172.21.15.133:0/500410627 >> v1:172.21.15.133:6789/0 conn(0x7f475015ff90 legacy=0x7f47500acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.150+0000 7f474e000640 1 -- 172.21.15.133:0/500410627 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4750072150 con 0x7f4750072950 2024-03-07T06:53:37.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.151+0000 7f474e000640 1 --2- 172.21.15.133:0/500410627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4750072950 0x7f47500cdee0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f475006ffa0 tx=0x7f473c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadc9fd1099185fd server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:37.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.151+0000 7f474d600640 1 -- 172.21.15.133:0/500410627 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c0315a0 con 0x7f4750072950 2024-03-07T06:53:37.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.151+0000 7f474d600640 1 -- 172.21.15.133:0/500410627 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473c037070 con 0x7f4750072950 2024-03-07T06:53:37.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.151+0000 7f474d600640 1 -- 172.21.15.133:0/500410627 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c031960 con 0x7f4750072950 2024-03-07T06:53:37.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.152+0000 7f4755600640 1 -- 172.21.15.133:0/500410627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4750072950 msgr2=0x7f47500cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.152+0000 7f4755600640 1 --2- 172.21.15.133:0/500410627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4750072950 0x7f47500cdee0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f475006ffa0 tx=0x7f473c02b840 comp rx=0 tx=0).stop 2024-03-07T06:53:37.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.152+0000 7f4755600640 1 -- 172.21.15.133:0/500410627 shutdown_connections 2024-03-07T06:53:37.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.152+0000 7f4755600640 1 --2- 172.21.15.133:0/500410627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4750072950 0x7f47500cdee0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.152+0000 7f4755600640 1 --2- 172.21.15.133:0/500410627 >> v2:172.21.15.161:3300/0 conn(0x7f475015fbb0 0x7f47500cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.152+0000 7f4755600640 1 -- 172.21.15.133:0/500410627 >> 172.21.15.133:0/500410627 conn(0x7f47500b3120 msgr2=0x7f47500b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:37.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.152+0000 7f4755600640 1 -- 172.21.15.133:0/500410627 shutdown_connections 2024-03-07T06:53:37.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.152+0000 7f4755600640 1 -- 172.21.15.133:0/500410627 wait complete. 2024-03-07T06:53:37.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.154+0000 7f4755600640 1 Processor -- start 2024-03-07T06:53:37.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.154+0000 7f4755600640 1 -- start start 2024-03-07T06:53:37.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.155+0000 7f4755600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f475015fbb0 0x7f47501538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.155+0000 7f4755600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f475015ff90 0x7f475006de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.155+0000 7f4755600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f475006edd0 0x7f47500b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.155+0000 7f4755600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47500b1a00 con 0x7f475015fbb0 2024-03-07T06:53:37.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.155+0000 7f4755600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4750160980 con 0x7f475015ff90 2024-03-07T06:53:37.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.155+0000 7f4755600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47501603b0 con 0x7f475006edd0 2024-03-07T06:53:37.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.155+0000 7f474ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f475015fbb0 0x7f47501538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.155+0000 7f474ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f475015fbb0 0x7f47501538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35870/0 (socket says 172.21.15.133:35870) 2024-03-07T06:53:37.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.155+0000 7f474ea00640 1 -- 172.21.15.133:0/4275229962 learned_addr learned my addr 172.21.15.133:0/4275229962 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:37.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.155+0000 7f474ea00640 1 -- 172.21.15.133:0/4275229962 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f475006edd0 msgr2=0x7f47500b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.155+0000 7f474ea00640 1 --2- 172.21.15.133:0/4275229962 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f475006edd0 0x7f47500b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.155+0000 7f474ea00640 1 -- 172.21.15.133:0/4275229962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f475015ff90 msgr2=0x7f475006de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.155+0000 7f474ea00640 1 --2- 172.21.15.133:0/4275229962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f475015ff90 0x7f475006de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.155+0000 7f474ea00640 1 -- 172.21.15.133:0/4275229962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4750159de0 con 0x7f475015fbb0 2024-03-07T06:53:37.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.156+0000 7f474ea00640 1 --2- 172.21.15.133:0/4275229962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f475015fbb0 0x7f47501538c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f474000d830 tx=0x7f474000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:37.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.156+0000 7f4746a00640 1 -- 172.21.15.133:0/4275229962 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474000b6c0 con 0x7f475015fbb0 2024-03-07T06:53:37.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.156+0000 7f4746a00640 1 -- 172.21.15.133:0/4275229962 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4740019070 con 0x7f475015fbb0 2024-03-07T06:53:37.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.156+0000 7f4746a00640 1 -- 172.21.15.133:0/4275229962 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474000ba80 con 0x7f475015fbb0 2024-03-07T06:53:37.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.156+0000 7f4755600640 1 -- 172.21.15.133:0/4275229962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4750159b00 con 0x7f475015fbb0 2024-03-07T06:53:37.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.156+0000 7f4755600640 1 -- 172.21.15.133:0/4275229962 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475006d620 con 0x7f475015fbb0 2024-03-07T06:53:37.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.158+0000 7f4746a00640 1 -- 172.21.15.133:0/4275229962 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f474001e460 con 0x7f475015fbb0 2024-03-07T06:53:37.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.159+0000 7f4746a00640 1 --2- 172.21.15.133:0/4275229962 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f473006b4d0 0x7f473006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.159+0000 7f4746a00640 1 -- 172.21.15.133:0/4275229962 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f474008a820 con 0x7f475015fbb0 2024-03-07T06:53:37.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.160+0000 7f474e000640 1 --2- 172.21.15.133:0/4275229962 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f473006b4d0 0x7f473006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.160+0000 7f4758626740 1 Processor -- start 2024-03-07T06:53:37.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.160+0000 7f4758626740 1 -- start start 2024-03-07T06:53:37.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.161+0000 7f474e000640 1 --2- 172.21.15.133:0/4275229962 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f473006b4d0 0x7f473006d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f473c036000 tx=0x7f473c030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:37.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.161+0000 7f4758626740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a610 0x55a6bff6d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.161+0000 7f4758626740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a9f0 0x55a6bff9d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.161+0000 7f4758626740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff6daf0 0x55a6bff9d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.161+0000 7f4758626740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bfed76c0 con 0x55a6bff6daf0 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.161+0000 7f4758626740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bfe809b0 con 0x55a6bff9a610 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.161+0000 7f4758626740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6bfed3760 con 0x55a6bff9a9f0 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.162+0000 7f474f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff6daf0 0x55a6bff9d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.162+0000 7f474f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff6daf0 0x55a6bff9d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35882/0 (socket says 172.21.15.133:35882) 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.162+0000 7f474f400640 1 -- 172.21.15.133:0/471923422 learned_addr learned my addr 172.21.15.133:0/471923422 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.162+0000 7f474ea00640 1 --2- 172.21.15.133:0/471923422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a610 0x55a6bff6d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.162+0000 7f474e000640 1 --2- 172.21.15.133:0/471923422 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a9f0 0x55a6bff9d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.162+0000 7f474f400640 1 -- 172.21.15.133:0/471923422 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a9f0 msgr2=0x55a6bff9d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.162+0000 7f474f400640 1 --2- 172.21.15.133:0/471923422 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a9f0 0x55a6bff9d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.162+0000 7f474f400640 1 -- 172.21.15.133:0/471923422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a610 msgr2=0x55a6bff6d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.162+0000 7f474f400640 1 --2- 172.21.15.133:0/471923422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a610 0x55a6bff6d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.162+0000 7f474f400640 1 -- 172.21.15.133:0/471923422 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6bfe746f0 con 0x55a6bff6daf0 2024-03-07T06:53:37.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.162+0000 7f474f400640 1 --2- 172.21.15.133:0/471923422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff6daf0 0x55a6bff9d8e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4748002120 tx=0x7f474800c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1faab49f8256d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:37.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.163+0000 7f473b000640 1 -- 172.21.15.133:0/471923422 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748017070 con 0x55a6bff6daf0 2024-03-07T06:53:37.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.163+0000 7f473b000640 1 -- 172.21.15.133:0/471923422 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47480072d0 con 0x55a6bff6daf0 2024-03-07T06:53:37.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.163+0000 7f473b000640 1 -- 172.21.15.133:0/471923422 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4748007680 con 0x55a6bff6daf0 2024-03-07T06:53:37.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.163+0000 7f4758626740 1 -- 172.21.15.133:0/471923422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff6daf0 msgr2=0x55a6bff9d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.163+0000 7f4758626740 1 --2- 172.21.15.133:0/471923422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff6daf0 0x55a6bff9d8e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4748002120 tx=0x7f474800c4b0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.163+0000 7f4758626740 1 -- 172.21.15.133:0/471923422 shutdown_connections 2024-03-07T06:53:37.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.163+0000 7f4758626740 1 --2- 172.21.15.133:0/471923422 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff6daf0 0x55a6bff9d8e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.163+0000 7f4758626740 1 --2- 172.21.15.133:0/471923422 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a9f0 0x55a6bff9d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.163+0000 7f4758626740 1 --2- 172.21.15.133:0/471923422 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a610 0x55a6bff6d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.163+0000 7f4758626740 1 -- 172.21.15.133:0/471923422 >> 172.21.15.133:0/471923422 conn(0x55a6bff6f820 msgr2=0x55a6bffb83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:37.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.163+0000 7f4758626740 1 -- 172.21.15.133:0/471923422 shutdown_connections 2024-03-07T06:53:37.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.163+0000 7f4758626740 1 -- 172.21.15.133:0/471923422 wait complete. 2024-03-07T06:53:37.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.164+0000 7f4758626740 1 -- start start 2024-03-07T06:53:37.165 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.164+0000 7f4758626740 1 Processor -- start 2024-03-07T06:53:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.165+0000 7f4758626740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a610 0x55a6bff84560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.165+0000 7f4758626740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff9a9f0 0x55a6bff96860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.165+0000 7f4758626740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff96da0 0x55a6bff86fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.165+0000 7f4758626740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bff71dc0 con 0x55a6bff9a9f0 2024-03-07T06:53:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.165+0000 7f4758626740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bfe93df0 con 0x55a6bff9a610 2024-03-07T06:53:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.165+0000 7f4758626740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6bff711b0 con 0x55a6bff96da0 2024-03-07T06:53:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.165+0000 7f4758626740 10 client.0 _ll_register_callbacks cb 0x55a6bfece8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T06:53:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.166+0000 7f474e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff9a9f0 0x55a6bff96860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.166+0000 7f474e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff9a9f0 0x55a6bff96860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35888/0 (socket says 172.21.15.133:35888) 2024-03-07T06:53:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.166+0000 7f474e000640 1 -- 172.21.15.133:0/3909455301 learned_addr learned my addr 172.21.15.133:0/3909455301 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.166+0000 7f474ea00640 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a610 0x55a6bff84560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.169 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.166+0000 7f474f400640 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff96da0 0x55a6bff86fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.166+0000 7f474e000640 1 -- 172.21.15.133:0/3909455301 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff96da0 msgr2=0x55a6bff86fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.166+0000 7f474e000640 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff96da0 0x55a6bff86fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.166+0000 7f474e000640 1 -- 172.21.15.133:0/3909455301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a610 msgr2=0x55a6bff84560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.166+0000 7f474e000640 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a610 0x55a6bff84560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.166+0000 7f474e000640 1 -- 172.21.15.133:0/3909455301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6bfbec2d0 con 0x55a6bff9a9f0 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.166+0000 7f474ea00640 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a610 0x55a6bff84560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.166+0000 7f474f400640 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff96da0 0x55a6bff86fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.166+0000 7f474e000640 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff9a9f0 0x55a6bff96860 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f473c009860 tx=0x7f473c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe83b382f708138 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.166+0000 7f472f400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.167+0000 7f472f400640 1 -- 172.21.15.133:0/3909455301 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c055060 con 0x55a6bff9a9f0 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.167+0000 7f472f400640 1 -- 172.21.15.133:0/3909455301 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f473c032840 con 0x55a6bff9a9f0 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.167+0000 7f4758626740 1 -- 172.21.15.133:0/3909455301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47480072d0 con 0x55a6bff9a9f0 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.167+0000 7f472f400640 1 -- 172.21.15.133:0/3909455301 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f473c05b050 con 0x55a6bff9a9f0 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.167+0000 7f4758626740 10 client.5742 fetch_fsmap learned FSMap version 27 2024-03-07T06:53:37.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.167+0000 7f4758626740 1 -- 172.21.15.133:0/3909455301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55a6bfcb66b0 con 0x55a6bff9a9f0 2024-03-07T06:53:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.167+0000 7f472f400640 1 -- 172.21.15.133:0/3909455301 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47480072d0 con 0x55a6bff9a9f0 2024-03-07T06:53:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.168+0000 7f472f400640 1 -- 172.21.15.133:0/3909455301 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f473c05e020 con 0x55a6bff9a9f0 2024-03-07T06:53:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.168+0000 7f4758626740 10 client.5742 fetch_fsmap finished waiting for FSMap version 27 2024-03-07T06:53:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.168+0000 7f4758626740 10 client.5742 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-07T06:53:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.168+0000 7f4758626740 20 client.5742 populate_metadata read hostname 'smithi133' 2024-03-07T06:53:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.168+0000 7f4758626740 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.133:6834/2951618124,v1:172.21.15.133:6835/2951618124] conn(0x55a6bffcbd90 0x55a6bffcc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.168+0000 7f4758626740 4 client.5742 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-07T06:53:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.168+0000 7f4758626740 1 -- 172.21.15.133:0/3909455301 --> [v2:172.21.15.133:6834/2951618124,v1:172.21.15.133:6835/2951618124] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6bfee4c00 con 0x55a6bffcbd90 2024-03-07T06:53:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.168+0000 7f474ea00640 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.133:6834/2951618124,v1:172.21.15.133:6835/2951618124] conn(0x55a6bffcbd90 0x55a6bffcc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.168+0000 7f474ea00640 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.133:6834/2951618124,v1:172.21.15.133:6835/2951618124] conn(0x55a6bffcbd90 0x55a6bffcc170 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11cad418ef2a6e21 server_cookie=50ac66cf8a76955 in_seq=0 out_seq=0 2024-03-07T06:53:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.168+0000 7f472f400640 10 client.5742 ms_handle_connect on v2:172.21.15.133:6834/2951618124 2024-03-07T06:53:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.169+0000 7f472f400640 1 -- 172.21.15.133:0/3909455301 <== mds.0 v2:172.21.15.133:6834/2951618124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a6bfee4c00 con 0x55a6bffcbd90 2024-03-07T06:53:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.169+0000 7f472f400640 10 client.5742 handle_command_reply: tid=0 2024-03-07T06:53:37.171 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.169+0000 7f4758626740 1 client.5742 shutdown 2024-03-07T06:53:37.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.170+0000 7f4758626740 1 -- 172.21.15.133:0/3909455301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff9a9f0 msgr2=0x55a6bff96860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.170+0000 7f4758626740 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff9a9f0 0x55a6bff96860 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f473c009860 tx=0x7f473c004730 comp rx=0 tx=0).stop 2024-03-07T06:53:37.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.170+0000 7f4758626740 1 -- 172.21.15.133:0/3909455301 shutdown_connections 2024-03-07T06:53:37.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.170+0000 7f4758626740 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.133:6834/2951618124,v1:172.21.15.133:6835/2951618124] conn(0x55a6bffcbd90 0x55a6bffcc170 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.170+0000 7f4758626740 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff96da0 0x55a6bff86fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.170+0000 7f4758626740 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff9a9f0 0x55a6bff96860 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.170+0000 7f4758626740 1 --2- 172.21.15.133:0/3909455301 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a610 0x55a6bff84560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.170+0000 7f472f400640 0 client.5742 ms_handle_reset on v2:172.21.15.133:6834/2951618124 2024-03-07T06:53:37.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.170+0000 7f4758626740 1 -- 172.21.15.133:0/3909455301 >> 172.21.15.133:0/3909455301 conn(0x55a6bff6daf0 msgr2=0x55a6bff9cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:37.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.170+0000 7f4758626740 1 -- 172.21.15.133:0/3909455301 shutdown_connections 2024-03-07T06:53:37.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.170+0000 7f4758626740 1 -- 172.21.15.133:0/3909455301 wait complete. 2024-03-07T06:53:37.173 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.170+0000 7f4758626740 20 client.5742 trim_cache size 0 max 16384 2024-03-07T06:53:37.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.188+0000 7f4758626740 1 Processor -- start 2024-03-07T06:53:37.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.189+0000 7f4758626740 1 -- start start 2024-03-07T06:53:37.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.189+0000 7f4758626740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff96da0 0x55a6bff9e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.189+0000 7f4758626740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff9a610 0x55a6bff9e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.189+0000 7f4758626740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a9f0 0x55a6bff9cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.189+0000 7f4758626740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4750158c40 con 0x55a6bff9a610 2024-03-07T06:53:37.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.189+0000 7f4758626740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4750158070 con 0x55a6bff96da0 2024-03-07T06:53:37.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.189+0000 7f4758626740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4750158640 con 0x55a6bff9a9f0 2024-03-07T06:53:37.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.190+0000 7f474e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff9a610 0x55a6bff9e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.190+0000 7f474e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff9a610 0x55a6bff9e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35894/0 (socket says 172.21.15.133:35894) 2024-03-07T06:53:37.190 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.190+0000 7f474e000640 1 -- 172.21.15.133:0/112462946 learned_addr learned my addr 172.21.15.133:0/112462946 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:37.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.190+0000 7f474f400640 1 --2- 172.21.15.133:0/112462946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a9f0 0x55a6bff9cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.190+0000 7f474ea00640 1 --2- 172.21.15.133:0/112462946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff96da0 0x55a6bff9e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.190+0000 7f474e000640 1 -- 172.21.15.133:0/112462946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a9f0 msgr2=0x55a6bff9cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.190+0000 7f474e000640 1 --2- 172.21.15.133:0/112462946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a9f0 0x55a6bff9cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.190+0000 7f474e000640 1 -- 172.21.15.133:0/112462946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff96da0 msgr2=0x55a6bff9e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.190+0000 7f474e000640 1 --2- 172.21.15.133:0/112462946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff96da0 0x55a6bff9e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.190+0000 7f474e000640 1 -- 172.21.15.133:0/112462946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6bff83cd0 con 0x55a6bff9a610 2024-03-07T06:53:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.190+0000 7f474f400640 1 --2- 172.21.15.133:0/112462946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a9f0 0x55a6bff9cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.190+0000 7f474e000640 1 --2- 172.21.15.133:0/112462946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff9a610 0x55a6bff9e8d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f473c02be50 tx=0x7f473c04a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bde1cb4b71cef5 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.191+0000 7f473a600640 1 -- 172.21.15.133:0/112462946 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c041070 con 0x55a6bff9a610 2024-03-07T06:53:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.191+0000 7f473a600640 1 -- 172.21.15.133:0/112462946 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473c037040 con 0x55a6bff9a610 2024-03-07T06:53:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.191+0000 7f473a600640 1 -- 172.21.15.133:0/112462946 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c032900 con 0x55a6bff9a610 2024-03-07T06:53:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.191+0000 7f4758626740 1 -- 172.21.15.133:0/112462946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff9a610 msgr2=0x55a6bff9e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.192 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.191+0000 7f4758626740 1 --2- 172.21.15.133:0/112462946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff9a610 0x55a6bff9e8d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f473c02be50 tx=0x7f473c04a040 comp rx=0 tx=0).stop 2024-03-07T06:53:37.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.191+0000 7f4758626740 1 -- 172.21.15.133:0/112462946 shutdown_connections 2024-03-07T06:53:37.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.191+0000 7f4758626740 1 --2- 172.21.15.133:0/112462946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a9f0 0x55a6bff9cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.191+0000 7f4758626740 1 --2- 172.21.15.133:0/112462946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff9a610 0x55a6bff9e8d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.191+0000 7f4758626740 1 --2- 172.21.15.133:0/112462946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff96da0 0x55a6bff9e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.191+0000 7f4758626740 1 -- 172.21.15.133:0/112462946 >> 172.21.15.133:0/112462946 conn(0x55a6bff6daf0 msgr2=0x55a6bff96630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:37.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.191+0000 7f4758626740 1 -- 172.21.15.133:0/112462946 shutdown_connections 2024-03-07T06:53:37.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.191+0000 7f4758626740 1 -- 172.21.15.133:0/112462946 wait complete. 2024-03-07T06:53:37.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.192+0000 7f4758626740 1 -- start start 2024-03-07T06:53:37.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.192+0000 7f4758626740 1 Processor -- start 2024-03-07T06:53:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.193+0000 7f4758626740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff96da0 0x55a6bff6e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.193+0000 7f4758626740 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a610 0x55a6bff9e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.193+0000 7f4758626740 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a9f0 0x55a6bff9e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.193+0000 7f4758626740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bffa0680 con 0x55a6bff96da0 2024-03-07T06:53:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.193+0000 7f4758626740 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6bff99900 con 0x55a6bff9a9f0 2024-03-07T06:53:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.193+0000 7f4758626740 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6bff95e50 con 0x55a6bff9a610 2024-03-07T06:53:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.193+0000 7f4758626740 10 client.0 _ll_register_callbacks cb 0x7f473c009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-07T06:53:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.193+0000 7f474ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff96da0 0x55a6bff6e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.194+0000 7f474ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff96da0 0x55a6bff6e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35908/0 (socket says 172.21.15.133:35908) 2024-03-07T06:53:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.194+0000 7f474ea00640 1 -- 172.21.15.133:0/2225055017 learned_addr learned my addr 172.21.15.133:0/2225055017 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.194+0000 7f474e000640 1 --2- 172.21.15.133:0/2225055017 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a610 0x55a6bff9e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.194+0000 7f474ea00640 1 -- 172.21.15.133:0/2225055017 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a610 msgr2=0x55a6bff9e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.194+0000 7f474ea00640 1 --2- 172.21.15.133:0/2225055017 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a610 0x55a6bff9e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.194+0000 7f474ea00640 1 -- 172.21.15.133:0/2225055017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a9f0 msgr2=0x55a6bff9e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.194+0000 7f474ea00640 1 --2- 172.21.15.133:0/2225055017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a9f0 0x55a6bff9e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.194+0000 7f474ea00640 1 -- 172.21.15.133:0/2225055017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6bffa0370 con 0x55a6bff96da0 2024-03-07T06:53:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.194+0000 7f474ea00640 1 --2- 172.21.15.133:0/2225055017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff96da0 0x55a6bff6e880 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4740061210 tx=0x7f4740058a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e591bd356ff6cf95 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.194+0000 7f472f400640 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2024-03-07T06:53:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.194+0000 7f472f400640 1 -- 172.21.15.133:0/2225055017 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4740074020 con 0x55a6bff96da0 2024-03-07T06:53:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.194+0000 7f472f400640 1 -- 172.21.15.133:0/2225055017 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4740057330 con 0x55a6bff96da0 2024-03-07T06:53:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.194+0000 7f4758626740 1 -- 172.21.15.133:0/2225055017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f473c037040 con 0x55a6bff96da0 2024-03-07T06:53:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.195+0000 7f472f400640 1 -- 172.21.15.133:0/2225055017 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f474008a040 con 0x55a6bff96da0 2024-03-07T06:53:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.195+0000 7f4758626740 10 client.5748 fetch_fsmap learned FSMap version 27 2024-03-07T06:53:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.195+0000 7f4758626740 1 -- 172.21.15.133:0/2225055017 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55a6bffc6ef0 con 0x55a6bff96da0 2024-03-07T06:53:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.195+0000 7f472f400640 1 -- 172.21.15.133:0/2225055017 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473c037040 con 0x55a6bff96da0 2024-03-07T06:53:37.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.195+0000 7f472f400640 1 -- 172.21.15.133:0/2225055017 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) v1 ==== 3314+0+0 (secure 0 0 0) 0x7f4740013070 con 0x55a6bff96da0 2024-03-07T06:53:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.195+0000 7f4758626740 10 client.5748 fetch_fsmap finished waiting for FSMap version 27 2024-03-07T06:53:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.195+0000 7f4758626740 10 client.5748 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-07T06:53:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.195+0000 7f4758626740 20 client.5748 populate_metadata read hostname 'smithi133' 2024-03-07T06:53:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.195+0000 7f4758626740 1 --2- 172.21.15.133:0/2225055017 >> [v2:172.21.15.133:6834/2951618124,v1:172.21.15.133:6835/2951618124] conn(0x55a6bffd27e0 0x55a6bff9cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.195+0000 7f4758626740 4 client.5748 mds_command: new command op to 5580 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-07T06:53:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.195+0000 7f4758626740 1 -- 172.21.15.133:0/2225055017 --> [v2:172.21.15.133:6834/2951618124,v1:172.21.15.133:6835/2951618124] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a6bfedef50 con 0x55a6bffd27e0 2024-03-07T06:53:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.195+0000 7f474e000640 1 --2- 172.21.15.133:0/2225055017 >> [v2:172.21.15.133:6834/2951618124,v1:172.21.15.133:6835/2951618124] conn(0x55a6bffd27e0 0x55a6bff9cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f474e000640 1 --2- 172.21.15.133:0/2225055017 >> [v2:172.21.15.133:6834/2951618124,v1:172.21.15.133:6835/2951618124] conn(0x55a6bffd27e0 0x55a6bff9cde0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983361c8c9e686c4 server_cookie=ef8995d325740fd5 in_seq=0 out_seq=0 2024-03-07T06:53:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f472f400640 10 client.5748 ms_handle_connect on v2:172.21.15.133:6834/2951618124 2024-03-07T06:53:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f472f400640 1 -- 172.21.15.133:0/2225055017 <== mds.0 v2:172.21.15.133:6834/2951618124 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55a6bfedef50 con 0x55a6bffd27e0 2024-03-07T06:53:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f472f400640 10 client.5748 handle_command_reply: tid=0 2024-03-07T06:53:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f4758626740 1 client.5748 shutdown 2024-03-07T06:53:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f4758626740 1 -- 172.21.15.133:0/2225055017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff96da0 msgr2=0x55a6bff6e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f4758626740 1 --2- 172.21.15.133:0/2225055017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff96da0 0x55a6bff6e880 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4740061210 tx=0x7f4740058a50 comp rx=0 tx=0).stop 2024-03-07T06:53:37.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f4758626740 1 -- 172.21.15.133:0/2225055017 shutdown_connections 2024-03-07T06:53:37.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f4758626740 1 --2- 172.21.15.133:0/2225055017 >> [v2:172.21.15.133:6834/2951618124,v1:172.21.15.133:6835/2951618124] conn(0x55a6bffd27e0 0x55a6bff9cde0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f4758626740 1 --2- 172.21.15.133:0/2225055017 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55a6bff9a9f0 0x55a6bff9e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f472f400640 0 client.5748 ms_handle_reset on v2:172.21.15.133:6834/2951618124 2024-03-07T06:53:37.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f4758626740 1 --2- 172.21.15.133:0/2225055017 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55a6bff9a610 0x55a6bff9e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f4758626740 1 --2- 172.21.15.133:0/2225055017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a6bff96da0 0x55a6bff6e880 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f4758626740 1 -- 172.21.15.133:0/2225055017 >> 172.21.15.133:0/2225055017 conn(0x55a6bff6daf0 msgr2=0x55a6bff72750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:37.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f4758626740 1 -- 172.21.15.133:0/2225055017 shutdown_connections 2024-03-07T06:53:37.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.196+0000 7f4758626740 1 -- 172.21.15.133:0/2225055017 wait complete. 2024-03-07T06:53:37.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.197+0000 7f4758626740 20 client.5748 trim_cache size 0 max 16384 2024-03-07T06:53:37.200 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:53:37.201 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "1655279d-73c2-44d4-ba1b-3e8a15ae005a", 2024-03-07T06:53:37.201 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2024-03-07T06:53:37.201 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 5580, 2024-03-07T06:53:37.201 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2024-03-07T06:53:37.201 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2024-03-07T06:53:37.201 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2024-03-07T06:53:37.201 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 12.040126562999999, 2024-03-07T06:53:37.201 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 27, 2024-03-07T06:53:37.201 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 46, 2024-03-07T06:53:37.201 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 46, 2024-03-07T06:53:37.201 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 17.446782348999999 2024-03-07T06:53:37.201 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:53:37.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.197+0000 7f4755600640 1 -- 172.21.15.133:0/4275229962 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f473006b4d0 msgr2=0x7f473006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.197+0000 7f4755600640 1 --2- 172.21.15.133:0/4275229962 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f473006b4d0 0x7f473006d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f473c036000 tx=0x7f473c030000 comp rx=0 tx=0).stop 2024-03-07T06:53:37.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.197+0000 7f4755600640 1 -- 172.21.15.133:0/4275229962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f475015fbb0 msgr2=0x7f47501538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.197+0000 7f4755600640 1 --2- 172.21.15.133:0/4275229962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f475015fbb0 0x7f47501538c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f474000d830 tx=0x7f474000dcf0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.197+0000 7f4755600640 1 -- 172.21.15.133:0/4275229962 shutdown_connections 2024-03-07T06:53:37.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.197+0000 7f4755600640 1 --2- 172.21.15.133:0/4275229962 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f475006edd0 0x7f47500b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.197+0000 7f4755600640 1 --2- 172.21.15.133:0/4275229962 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f473006b4d0 0x7f473006d990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.197+0000 7f4755600640 1 --2- 172.21.15.133:0/4275229962 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f475015ff90 0x7f475006de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.197+0000 7f4755600640 1 --2- 172.21.15.133:0/4275229962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f475015fbb0 0x7f47501538c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.197+0000 7f4755600640 1 -- 172.21.15.133:0/4275229962 >> 172.21.15.133:0/4275229962 conn(0x7f47500b3120 msgr2=0x7f475015efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:37.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.197+0000 7f4755600640 1 -- 172.21.15.133:0/4275229962 shutdown_connections 2024-03-07T06:53:37.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.197+0000 7f4755600640 1 -- 172.21.15.133:0/4275229962 wait complete. 2024-03-07T06:53:37.214 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-07T06:53:37.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.409+0000 7fb151800640 1 Processor -- start 2024-03-07T06:53:37.409 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.410+0000 7fb151800640 1 -- start start 2024-03-07T06:53:37.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.411+0000 7fb151800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb14c072950 0x7fb14c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.411+0000 7fb151800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fb14c0cdee0 0x7fb14c15c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.411+0000 7fb151800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14c0731b0 con 0x7fb14c15fbb0 2024-03-07T06:53:37.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.411+0000 7fb151800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb14c072f60 con 0x7fb14c0cdee0 2024-03-07T06:53:37.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.411+0000 7fb151800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c073490 con 0x7fb14c072950 2024-03-07T06:53:37.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.411+0000 7fb14aa00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fb14c15fbb0 0x7fb14c15ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47482/0 (socket says 172.21.15.133:47482) 2024-03-07T06:53:37.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.411+0000 7fb14aa00640 1 -- 172.21.15.133:0/406372001 learned_addr learned my addr 172.21.15.133:0/406372001 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:37.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.412+0000 7fb14b400640 1 --2- 172.21.15.133:0/406372001 >> v2:172.21.15.133:3300/0 conn(0x7fb14c0cdee0 0x7fb14c15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.412+0000 7fb143e00640 1 --2- 172.21.15.133:0/406372001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb14c072950 0x7fb14c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.412+0000 7fb14a000640 1 -- 172.21.15.133:0/406372001 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269570743 0 0) 0x7fb14c0731b0 con 0x7fb14c15fbb0 2024-03-07T06:53:37.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.412+0000 7fb14a000640 1 -- 172.21.15.133:0/406372001 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb130003520 con 0x7fb14c15fbb0 2024-03-07T06:53:37.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.412+0000 7fb14b400640 1 -- 172.21.15.133:0/406372001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb14c072950 msgr2=0x7fb14c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:37.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.412+0000 7fb14b400640 1 --2- 172.21.15.133:0/406372001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb14c072950 0x7fb14c0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.412+0000 7fb14b400640 1 -- 172.21.15.133:0/406372001 >> v1:172.21.15.133:6789/0 conn(0x7fb14c15fbb0 legacy=0x7fb14c15ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.412+0000 7fb14b400640 1 -- 172.21.15.133:0/406372001 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c072150 con 0x7fb14c0cdee0 2024-03-07T06:53:37.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.412+0000 7fb143e00640 1 --2- 172.21.15.133:0/406372001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb14c072950 0x7fb14c0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:53:37.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.414+0000 7fb14b400640 1 --2- 172.21.15.133:0/406372001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb14c0cdee0 0x7fb14c15c550 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0730e0 tx=0x7fb14402be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbad61b9b923df97 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:37.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.415+0000 7fb14a000640 1 -- 172.21.15.133:0/406372001 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144039070 con 0x7fb14c0cdee0 2024-03-07T06:53:37.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.415+0000 7fb14a000640 1 -- 172.21.15.133:0/406372001 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14402cbb0 con 0x7fb14c0cdee0 2024-03-07T06:53:37.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.415+0000 7fb14a000640 1 -- 172.21.15.133:0/406372001 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb144034440 con 0x7fb14c0cdee0 2024-03-07T06:53:37.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.415+0000 7fb151800640 1 -- 172.21.15.133:0/406372001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb14c0cdee0 msgr2=0x7fb14c15c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.415+0000 7fb151800640 1 --2- 172.21.15.133:0/406372001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb14c0cdee0 0x7fb14c15c550 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0730e0 tx=0x7fb14402be00 comp rx=0 tx=0).stop 2024-03-07T06:53:37.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.415+0000 7fb151800640 1 -- 172.21.15.133:0/406372001 shutdown_connections 2024-03-07T06:53:37.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.415+0000 7fb151800640 1 --2- 172.21.15.133:0/406372001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb14c0cdee0 0x7fb14c15c550 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.415+0000 7fb151800640 1 --2- 172.21.15.133:0/406372001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb14c072950 0x7fb14c0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.415+0000 7fb151800640 1 -- 172.21.15.133:0/406372001 >> 172.21.15.133:0/406372001 conn(0x7fb14c0b3120 msgr2=0x7fb14c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:37.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.415+0000 7fb151800640 1 -- 172.21.15.133:0/406372001 shutdown_connections 2024-03-07T06:53:37.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.416+0000 7fb151800640 1 -- 172.21.15.133:0/406372001 wait complete. 2024-03-07T06:53:37.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.418+0000 7fb151800640 1 Processor -- start 2024-03-07T06:53:37.418 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.418+0000 7fb151800640 1 -- start start 2024-03-07T06:53:37.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.420+0000 7fb151800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb14c0cdee0 0x7fb14c0cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.420+0000 7fb151800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb14c15fbb0 0x7fb14c14d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.420+0000 7fb151800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb14c09d2e0 0x7fb14c09d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.420+0000 7fb151800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c160980 con 0x7fb14c15fbb0 2024-03-07T06:53:37.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.420+0000 7fb151800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c0ce2f0 con 0x7fb14c0cdee0 2024-03-07T06:53:37.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.420+0000 7fb151800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14c0b1a00 con 0x7fb14c09d2e0 2024-03-07T06:53:37.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.420+0000 7fb143e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb14c15fbb0 0x7fb14c14d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.420+0000 7fb143e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb14c15fbb0 0x7fb14c14d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35930/0 (socket says 172.21.15.133:35930) 2024-03-07T06:53:37.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.420+0000 7fb143e00640 1 -- 172.21.15.133:0/2508627809 learned_addr learned my addr 172.21.15.133:0/2508627809 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:37.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.421+0000 7fb14aa00640 1 --2- 172.21.15.133:0/2508627809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb14c0cdee0 0x7fb14c0cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.421+0000 7fb14b400640 1 --2- 172.21.15.133:0/2508627809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb14c09d2e0 0x7fb14c09d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.421+0000 7fb143e00640 1 -- 172.21.15.133:0/2508627809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb14c09d2e0 msgr2=0x7fb14c09d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.421+0000 7fb143e00640 1 --2- 172.21.15.133:0/2508627809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb14c09d2e0 0x7fb14c09d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.421+0000 7fb143e00640 1 -- 172.21.15.133:0/2508627809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb14c0cdee0 msgr2=0x7fb14c0cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.421+0000 7fb143e00640 1 --2- 172.21.15.133:0/2508627809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb14c0cdee0 0x7fb14c0cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.421+0000 7fb143e00640 1 -- 172.21.15.133:0/2508627809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c09de00 con 0x7fb14c15fbb0 2024-03-07T06:53:37.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.421+0000 7fb143e00640 1 --2- 172.21.15.133:0/2508627809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb14c15fbb0 0x7fb14c14d920 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb13400e810 tx=0x7fb13400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:37.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.422+0000 7fb142a00640 1 -- 172.21.15.133:0/2508627809 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13400f040 con 0x7fb14c15fbb0 2024-03-07T06:53:37.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.422+0000 7fb142a00640 1 -- 172.21.15.133:0/2508627809 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb134004030 con 0x7fb14c15fbb0 2024-03-07T06:53:37.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.422+0000 7fb142a00640 1 -- 172.21.15.133:0/2508627809 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134004400 con 0x7fb14c15fbb0 2024-03-07T06:53:37.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.422+0000 7fb151800640 1 -- 172.21.15.133:0/2508627809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14c0ca9f0 con 0x7fb14c15fbb0 2024-03-07T06:53:37.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.422+0000 7fb151800640 1 -- 172.21.15.133:0/2508627809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14c0cafb0 con 0x7fb14c15fbb0 2024-03-07T06:53:37.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.423+0000 7fb151800640 1 -- 172.21.15.133:0/2508627809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb14c062c90 con 0x7fb14c15fbb0 2024-03-07T06:53:37.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.424+0000 7fb142a00640 1 -- 172.21.15.133:0/2508627809 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fb13400c6d0 con 0x7fb14c15fbb0 2024-03-07T06:53:37.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.425+0000 7fb142a00640 1 --2- 172.21.15.133:0/2508627809 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb12806b4d0 0x7fb12806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:37.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.425+0000 7fb142a00640 1 -- 172.21.15.133:0/2508627809 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb134089fc0 con 0x7fb14c15fbb0 2024-03-07T06:53:37.425 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.425+0000 7fb14aa00640 1 --2- 172.21.15.133:0/2508627809 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb12806b4d0 0x7fb12806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:37.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.427+0000 7fb14aa00640 1 --2- 172.21.15.133:0/2508627809 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb12806b4d0 0x7fb12806d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb13c007fd0 tx=0x7fb13c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:37.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.429+0000 7fb142a00640 1 -- 172.21.15.133:0/2508627809 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fb134017020 con 0x7fb14c15fbb0 2024-03-07T06:53:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.664+0000 7fb151800640 1 -- 172.21.15.133:0/2508627809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb14c15bfe0 con 0x7fb14c15fbb0 2024-03-07T06:53:37.664 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.664+0000 7fb142a00640 1 -- 172.21.15.133:0/2508627809 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb134058e30 con 0x7fb14c15fbb0 2024-03-07T06:53:37.664 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-07T06:53:37.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.667+0000 7fb151800640 1 -- 172.21.15.133:0/2508627809 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb12806b4d0 msgr2=0x7fb12806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.667+0000 7fb151800640 1 --2- 172.21.15.133:0/2508627809 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb12806b4d0 0x7fb12806d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb13c007fd0 tx=0x7fb13c009040 comp rx=0 tx=0).stop 2024-03-07T06:53:37.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.667+0000 7fb151800640 1 -- 172.21.15.133:0/2508627809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb14c15fbb0 msgr2=0x7fb14c14d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:37.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.667+0000 7fb151800640 1 --2- 172.21.15.133:0/2508627809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb14c15fbb0 0x7fb14c14d920 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb13400e810 tx=0x7fb13400ecd0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.667+0000 7fb151800640 1 -- 172.21.15.133:0/2508627809 shutdown_connections 2024-03-07T06:53:37.667 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.667+0000 7fb151800640 1 --2- 172.21.15.133:0/2508627809 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb14c09d2e0 0x7fb14c09d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.667+0000 7fb151800640 1 --2- 172.21.15.133:0/2508627809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb14c15fbb0 0x7fb14c14d920 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.667+0000 7fb151800640 1 --2- 172.21.15.133:0/2508627809 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fb12806b4d0 0x7fb12806d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.667+0000 7fb151800640 1 --2- 172.21.15.133:0/2508627809 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb14c0cdee0 0x7fb14c0cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:37.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.667+0000 7fb151800640 1 -- 172.21.15.133:0/2508627809 >> 172.21.15.133:0/2508627809 conn(0x7fb14c0b3120 msgr2=0x7fb14c15ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:37.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.667+0000 7fb151800640 1 -- 172.21.15.133:0/2508627809 shutdown_connections 2024-03-07T06:53:37.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:37.667+0000 7fb151800640 1 -- 172.21.15.133:0/2508627809 wait complete. 2024-03-07T06:53:37.684 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-07T06:53:37.684 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-07T06:53:37.684 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-03-07T06:53:37.684 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-07T06:53:37.684 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-07T06:53:37.684 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-07T06:53:37.684 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-07T06:53:37.684 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-07T06:53:37.684 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-07T06:53:37.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:53:37.685 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-03-07T06:53:37.725 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-07T06:53:37.725 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-07T06:53:37.725 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-03-07T06:53:37.725 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:53:37.725 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-03-07T06:53:37.725 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:53:37.725 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:53:37.725 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:53:37.726 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-03-07T06:53:37.726 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:53:37.726 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:53:37.726 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-03-07T06:53:37.726 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-07T06:53:37.726 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:53:37.726 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-07T06:53:37.726 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:53:37.726 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-07T06:53:37.726 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42403sec preferred_lft 42403sec 2024-03-07T06:53:37.727 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link 2024-03-07T06:53:37.727 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:53:37.727 INFO:teuthology.orchestra.run.smithi133.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-07T06:53:37.727 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 6e:50:ef:52:0c:c1 brd ff:ff:ff:ff:ff:ff 2024-03-07T06:53:37.727 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-07T06:53:37.727 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:53:37.727 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::4c8:8aff:feed:a7f5/64 scope link 2024-03-07T06:53:37.727 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:53:37.727 INFO:teuthology.orchestra.run.smithi133.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-07T06:53:37.727 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 6e:50:ef:52:0c:c1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-07T06:53:37.727 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::6c50:efff:fe52:cc1/64 scope link 2024-03-07T06:53:37.727 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:53:37.727 INFO:teuthology.orchestra.run.smithi133.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-07T06:53:37.727 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether f2:f2:9f:72:0c:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-07T06:53:37.728 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::f0f2:9fff:fe72:c1e/64 scope link 2024-03-07T06:53:37.728 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-03-07T06:53:37.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:53:37.729 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-07T06:53:37.843 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-03-07T06:53:37.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:53:37.846 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-03-07T06:53:37.887 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-03-07T06:53:37.887 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-07T06:53:37.887 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-07T06:53:37.887 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-07T06:53:37.887 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-07T06:53:37.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:53:37.887 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2024-03-07T06:53:37.888 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-03-07T06:53:37.888 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-07T06:53:37.888 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-07T06:53:37.888 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-07T06:53:37.888 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-03-07T06:53:38.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:53:38.097 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-03-07T06:53:38.144 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-07T06:53:38.144 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-07T06:53:38.144 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-07T06:53:38.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:53:38.144 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-03-07T06:53:38.225 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T06:53:38.225 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-07T06:53:38.279 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-07T06:53:38.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-07T06:53:38.279 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-07T06:53:38.323 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-03-07T06:53:38.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:53:38.323 DEBUG:teuthology.orchestra.run.smithi133:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:53:38.500 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:53:38.500 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T06:53:38.532 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-03-07T06:53:38.532 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-03-07T06:53:38.532 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: resolved to: "172.21.15.133:3300,172.21.15.161:3300,172.21.15.161:3301" 2024-03-07T06:53:38.533 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: trying mount with new device syntax: 0@1655279d-73c2-44d4-ba1b-3e8a15ae005a.cephfs=/ 2024-03-07T06:53:38.533 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.133:3300/172.21.15.161:3300/172.21.15.161:3301" will pass to kernel 2024-03-07T06:53:38.535 INFO:teuthology.orchestra.run.smithi133.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-07T06:53:38.535 INFO:teuthology.orchestra.run.smithi133.stdout: You just mounted a file system that supports labels which does not 2024-03-07T06:53:38.535 INFO:teuthology.orchestra.run.smithi133.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-07T06:53:38.535 INFO:teuthology.orchestra.run.smithi133.stdout: applications will generate AVC messages and not be allowed access to 2024-03-07T06:53:38.535 INFO:teuthology.orchestra.run.smithi133.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-07T06:53:38.539 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-07T06:53:38.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:53:38.539 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-07T06:53:38.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:53:38.615 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-07T06:53:38.675 INFO:teuthology.orchestra.run.smithi133.stdout:client5760 2024-03-07T06:53:38.676 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:53:38.676 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:38.721 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_bal_fragment_size_max 50 2024-03-07T06:53:38.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.972+0000 7f0d55000640 1 Processor -- start 2024-03-07T06:53:38.972 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.972+0000 7f0d55000640 1 -- start start 2024-03-07T06:53:38.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.974+0000 7f0d55000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0d500cc310 0x7f0d500ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:38.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.974+0000 7f0d55000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d500b4f20 0x7f0d500ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:38.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.974+0000 7f0d55000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d500b54a0 con 0x7f0d5015b1c0 2024-03-07T06:53:38.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.974+0000 7f0d55000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d500b5760 con 0x7f0d500cc310 2024-03-07T06:53:38.974 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.974+0000 7f0d55000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500b5a40 con 0x7f0d500b4f20 2024-03-07T06:53:38.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.975+0000 7f0d4de00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0d5015b1c0 0x7f0d500cbc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:47492/0 (socket says 172.21.15.133:47492) 2024-03-07T06:53:38.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.975+0000 7f0d4de00640 1 -- 172.21.15.133:0/2741103570 learned_addr learned my addr 172.21.15.133:0/2741103570 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:38.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.975+0000 7f0d4d400640 1 --2- 172.21.15.133:0/2741103570 >> v2:172.21.15.133:3300/0 conn(0x7f0d500cc310 0x7f0d500ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:38.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.975+0000 7f0d4e800640 1 --2- 172.21.15.133:0/2741103570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d500b4f20 0x7f0d500ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:38.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.975+0000 7f0d4d400640 1 -- 172.21.15.133:0/2741103570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d500b4f20 msgr2=0x7f0d500ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:38.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.975+0000 7f0d4d400640 1 --2- 172.21.15.133:0/2741103570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d500b4f20 0x7f0d500ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:38.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.975+0000 7f0d4d400640 1 -- 172.21.15.133:0/2741103570 >> v1:172.21.15.133:6789/0 conn(0x7f0d5015b1c0 legacy=0x7f0d500cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:38.975 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.975+0000 7f0d4d400640 1 -- 172.21.15.133:0/2741103570 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d500b5d20 con 0x7f0d500cc310 2024-03-07T06:53:38.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.976+0000 7f0d4d400640 1 --2- 172.21.15.133:0/2741103570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d500cc310 0x7f0d500ce3c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009d90 tx=0x7f0d4002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5117269049a3865 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:38.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.977+0000 7f0d4ca00640 1 -- 172.21.15.133:0/2741103570 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d40039070 con 0x7f0d500cc310 2024-03-07T06:53:38.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.977+0000 7f0d4ca00640 1 -- 172.21.15.133:0/2741103570 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4002cc60 con 0x7f0d500cc310 2024-03-07T06:53:38.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.977+0000 7f0d4ca00640 1 -- 172.21.15.133:0/2741103570 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d40034660 con 0x7f0d500cc310 2024-03-07T06:53:38.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.977+0000 7f0d55000640 1 -- 172.21.15.133:0/2741103570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d500cc310 msgr2=0x7f0d500ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:38.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.977+0000 7f0d55000640 1 --2- 172.21.15.133:0/2741103570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d500cc310 0x7f0d500ce3c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009d90 tx=0x7f0d4002beb0 comp rx=0 tx=0).stop 2024-03-07T06:53:38.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.977+0000 7f0d55000640 1 -- 172.21.15.133:0/2741103570 shutdown_connections 2024-03-07T06:53:38.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.977+0000 7f0d55000640 1 --2- 172.21.15.133:0/2741103570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d500b4f20 0x7f0d500ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:38.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.977+0000 7f0d55000640 1 --2- 172.21.15.133:0/2741103570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d500cc310 0x7f0d500ce3c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:38.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.978+0000 7f0d55000640 1 -- 172.21.15.133:0/2741103570 >> 172.21.15.133:0/2741103570 conn(0x7f0d500b31d0 msgr2=0x7f0d500b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:38.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.978+0000 7f0d55000640 1 -- 172.21.15.133:0/2741103570 shutdown_connections 2024-03-07T06:53:38.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.978+0000 7f0d55000640 1 -- 172.21.15.133:0/2741103570 wait complete. 2024-03-07T06:53:38.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.979+0000 7f0d55000640 1 Processor -- start 2024-03-07T06:53:38.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.979+0000 7f0d55000640 1 -- start start 2024-03-07T06:53:38.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.980+0000 7f0d55000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d500b4f20 0x7f0d501431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:38.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.980+0000 7f0d55000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d500cc310 0x7f0d5013ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:38.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.980+0000 7f0d55000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d5015b1c0 0x7f0d50140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:38.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.980+0000 7f0d55000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500b3e70 con 0x7f0d500b4f20 2024-03-07T06:53:38.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.980+0000 7f0d55000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500b3cf0 con 0x7f0d5015b1c0 2024-03-07T06:53:38.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.980+0000 7f0d55000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d500b3ff0 con 0x7f0d500cc310 2024-03-07T06:53:38.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.981+0000 7f0d4de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d500b4f20 0x7f0d501431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:38.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.981+0000 7f0d4de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d500b4f20 0x7f0d501431f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35956/0 (socket says 172.21.15.133:35956) 2024-03-07T06:53:38.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.981+0000 7f0d4de00640 1 -- 172.21.15.133:0/2012873002 learned_addr learned my addr 172.21.15.133:0/2012873002 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:38.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.981+0000 7f0d4d400640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d500cc310 0x7f0d5013ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:38.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.981+0000 7f0d4e800640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d5015b1c0 0x7f0d50140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:38.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.981+0000 7f0d4de00640 1 -- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d500cc310 msgr2=0x7f0d5013ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:38.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.981+0000 7f0d4de00640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d500cc310 0x7f0d5013ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:38.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.981+0000 7f0d4de00640 1 -- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d5015b1c0 msgr2=0x7f0d50140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:38.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.981+0000 7f0d4de00640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d5015b1c0 0x7f0d50140750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:38.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.981+0000 7f0d4de00640 1 -- 172.21.15.133:0/2012873002 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d50143730 con 0x7f0d500b4f20 2024-03-07T06:53:38.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.981+0000 7f0d4d400640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d500cc310 0x7f0d5013ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:38.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.981+0000 7f0d4e800640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d5015b1c0 0x7f0d50140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:38.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.981+0000 7f0d4de00640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d500b4f20 0x7f0d501431f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c3f0 tx=0x7f0d3800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:38.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.982+0000 7f0d46000640 1 -- 172.21.15.133:0/2012873002 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d38017070 con 0x7f0d500b4f20 2024-03-07T06:53:38.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.982+0000 7f0d46000640 1 -- 172.21.15.133:0/2012873002 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d38005ce0 con 0x7f0d500b4f20 2024-03-07T06:53:38.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.982+0000 7f0d46000640 1 -- 172.21.15.133:0/2012873002 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d380072f0 con 0x7f0d500b4f20 2024-03-07T06:53:38.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.982+0000 7f0d55000640 1 -- 172.21.15.133:0/2012873002 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d50140450 con 0x7f0d500b4f20 2024-03-07T06:53:38.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.982+0000 7f0d55000640 1 -- 172.21.15.133:0/2012873002 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5015ba40 con 0x7f0d500b4f20 2024-03-07T06:53:38.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.983+0000 7f0d55000640 1 -- 172.21.15.133:0/2012873002 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d50062c90 con 0x7f0d500b4f20 2024-03-07T06:53:38.984 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.985+0000 7f0d46000640 1 -- 172.21.15.133:0/2012873002 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0d3801a070 con 0x7f0d500b4f20 2024-03-07T06:53:38.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.986+0000 7f0d46000640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:38.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.986+0000 7f0d46000640 1 -- 172.21.15.133:0/2012873002 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d380895a0 con 0x7f0d500b4f20 2024-03-07T06:53:38.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.987+0000 7f0d4d400640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:38.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.988+0000 7f0d4d400640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0d40036000 tx=0x7f0d40030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:38.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:38.995+0000 7f0d46000640 1 -- 172.21.15.133:0/2012873002 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0d50143730 con 0x7f0d500b4f20 2024-03-07T06:53:39.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:39.191+0000 7f0d55000640 1 -- 172.21.15.133:0/2012873002 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) v1 -- 0x7f0d50159230 con 0x7f0d500b4f20 2024-03-07T06:53:39.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:39.196+0000 7f0d46000640 1 -- 172.21.15.133:0/2012873002 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v13)=0 v13) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0d38058410 con 0x7f0d500b4f20 2024-03-07T06:53:39.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:39.198+0000 7f0d55000640 1 -- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0d2c06b4d0 msgr2=0x7f0d2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:39.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:39.199+0000 7f0d55000640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0d40036000 tx=0x7f0d40030000 comp rx=0 tx=0).stop 2024-03-07T06:53:39.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:39.199+0000 7f0d55000640 1 -- 172.21.15.133:0/2012873002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d500b4f20 msgr2=0x7f0d501431f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:39.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:39.199+0000 7f0d55000640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d500b4f20 0x7f0d501431f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c3f0 tx=0x7f0d3800c8b0 comp rx=0 tx=0).stop 2024-03-07T06:53:39.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:39.199+0000 7f0d55000640 1 -- 172.21.15.133:0/2012873002 shutdown_connections 2024-03-07T06:53:39.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:39.199+0000 7f0d55000640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0d2c06b4d0 0x7f0d2c06d990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:39.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:39.199+0000 7f0d55000640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d5015b1c0 0x7f0d50140750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:39.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:39.199+0000 7f0d55000640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d500cc310 0x7f0d5013ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:39.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:39.199+0000 7f0d55000640 1 --2- 172.21.15.133:0/2012873002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d500b4f20 0x7f0d501431f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:39.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:39.199+0000 7f0d55000640 1 -- 172.21.15.133:0/2012873002 >> 172.21.15.133:0/2012873002 conn(0x7f0d500b31d0 msgr2=0x7f0d500b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:39.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:39.199+0000 7f0d55000640 1 -- 172.21.15.133:0/2012873002 shutdown_connections 2024-03-07T06:53:39.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:39.199+0000 7f0d55000640 1 -- 172.21.15.133:0/2012873002 wait complete. 2024-03-07T06:53:49.220 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:53:49.220 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:49.279 DEBUG:teuthology.orchestra.run.smithi133:> stdin-killer -- python3 -c ' 2024-03-07T06:53:49.279 DEBUG:teuthology.orchestra.run.smithi133:> import os 2024-03-07T06:53:49.279 DEBUG:teuthology.orchestra.run.smithi133:> import uuid 2024-03-07T06:53:49.279 DEBUG:teuthology.orchestra.run.smithi133:> 2024-03-07T06:53:49.279 DEBUG:teuthology.orchestra.run.smithi133:> n = 51 2024-03-07T06:53:49.279 DEBUG:teuthology.orchestra.run.smithi133:> create_hard_links = False 2024-03-07T06:53:49.279 DEBUG:teuthology.orchestra.run.smithi133:> if 0 > 0: 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> create_hard_links = True 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> path = "/home/ubuntu/cephtest/mnt.0/subdir/file" 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> dpath = os.path.dirname(path) 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> fnameprefix = os.path.basename(path) 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> os.makedirs(dpath, exist_ok=True) 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> try: 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> for i in range(n): 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> with open(fpath, '"'"'w'"'"') as f: 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> f.write(f"{i}") 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-03-07T06:53:49.280 DEBUG:teuthology.orchestra.run.smithi133:> f.flush() 2024-03-07T06:53:49.281 DEBUG:teuthology.orchestra.run.smithi133:> os.fsync(f.fileno()) 2024-03-07T06:53:49.281 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-03-07T06:53:49.281 DEBUG:teuthology.orchestra.run.smithi133:> os.unlink(fpath) 2024-03-07T06:53:49.281 DEBUG:teuthology.orchestra.run.smithi133:> if False: 2024-03-07T06:53:49.281 DEBUG:teuthology.orchestra.run.smithi133:> os.fsync(dirfd) 2024-03-07T06:53:49.281 DEBUG:teuthology.orchestra.run.smithi133:> if create_hard_links: 2024-03-07T06:53:49.281 DEBUG:teuthology.orchestra.run.smithi133:> for j in range(0): 2024-03-07T06:53:49.281 DEBUG:teuthology.orchestra.run.smithi133:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-03-07T06:53:49.281 DEBUG:teuthology.orchestra.run.smithi133:> if True: 2024-03-07T06:53:49.281 DEBUG:teuthology.orchestra.run.smithi133:> os.fsync(dirfd) 2024-03-07T06:53:49.281 DEBUG:teuthology.orchestra.run.smithi133:> finally: 2024-03-07T06:53:49.281 DEBUG:teuthology.orchestra.run.smithi133:> os.close(dirfd) 2024-03-07T06:53:49.281 DEBUG:teuthology.orchestra.run.smithi133:> ' 2024-03-07T06:53:49.479 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-07T06:53:49.488 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-07T06:53:49.488 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-07T06:53:49.625 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-07T06:53:49.743 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-07T06:53:49.743 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:53:49.743 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:49.794 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:53:49.794 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:49.888 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:53:50.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.092+0000 7f3b6b400640 1 Processor -- start 2024-03-07T06:53:50.093 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.093+0000 7f3b6b400640 1 -- start start 2024-03-07T06:53:50.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.094+0000 7f3b6b400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3b640ce5f0 0x7f3b640b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.094+0000 7f3b6b400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3b640af950 0x7f3b64071d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.094+0000 7f3b6b400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b6406f140 con 0x7f3b6415b1b0 2024-03-07T06:53:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.094+0000 7f3b6b400640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b64071610 con 0x7f3b640af950 2024-03-07T06:53:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.094+0000 7f3b6b400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b6406f420 con 0x7f3b640ce5f0 2024-03-07T06:53:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.095+0000 7f3b69400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3b640af950 0x7f3b64071d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.095+0000 7f3b69400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3b640af950 0x7f3b64071d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53852/0 (socket says 172.21.15.133:53852) 2024-03-07T06:53:50.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.095+0000 7f3b69400640 1 -- 172.21.15.133:0/3031965176 learned_addr learned my addr 172.21.15.133:0/3031965176 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:50.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.095+0000 7f3b63e00640 1 --2- 172.21.15.133:0/3031965176 >> v2:172.21.15.161:3300/0 conn(0x7f3b640ce5f0 0x7f3b640b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:50.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.095+0000 7f3b69400640 1 -- 172.21.15.133:0/3031965176 >> v2:172.21.15.161:3300/0 conn(0x7f3b640ce5f0 msgr2=0x7f3b640b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:50.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.095+0000 7f3b69400640 1 --2- 172.21.15.133:0/3031965176 >> v2:172.21.15.161:3300/0 conn(0x7f3b640ce5f0 0x7f3b640b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.095+0000 7f3b69400640 1 -- 172.21.15.133:0/3031965176 >> v1:172.21.15.161:6789/0 conn(0x7f3b6415b1b0 legacy=0x7f3b640cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:50.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.095+0000 7f3b69400640 1 -- 172.21.15.133:0/3031965176 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b64071860 con 0x7f3b640af950 2024-03-07T06:53:50.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.096+0000 7f3b69400640 1 --2- 172.21.15.133:0/3031965176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b640af950 0x7f3b64071d80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3b640b0270 tx=0x7f3b5c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7f8b54172b9652 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:50.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.097+0000 7f3b63400640 1 -- 172.21.15.133:0/3031965176 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c02ca80 con 0x7f3b640af950 2024-03-07T06:53:50.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.097+0000 7f3b63400640 1 -- 172.21.15.133:0/3031965176 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5c02cc20 con 0x7f3b640af950 2024-03-07T06:53:50.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.097+0000 7f3b63400640 1 -- 172.21.15.133:0/3031965176 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0343f0 con 0x7f3b640af950 2024-03-07T06:53:50.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.097+0000 7f3b6b400640 1 -- 172.21.15.133:0/3031965176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b640af950 msgr2=0x7f3b64071d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:50.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.097+0000 7f3b6b400640 1 --2- 172.21.15.133:0/3031965176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b640af950 0x7f3b64071d80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3b640b0270 tx=0x7f3b5c02bee0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.104 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.097+0000 7f3b6b400640 1 -- 172.21.15.133:0/3031965176 shutdown_connections 2024-03-07T06:53:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.097+0000 7f3b6b400640 1 --2- 172.21.15.133:0/3031965176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b640af950 0x7f3b64071d80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.097+0000 7f3b6b400640 1 --2- 172.21.15.133:0/3031965176 >> v2:172.21.15.161:3300/0 conn(0x7f3b640ce5f0 0x7f3b640b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.097+0000 7f3b6b400640 1 -- 172.21.15.133:0/3031965176 >> 172.21.15.133:0/3031965176 conn(0x7f3b640b3120 msgr2=0x7f3b640b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.097+0000 7f3b6b400640 1 -- 172.21.15.133:0/3031965176 shutdown_connections 2024-03-07T06:53:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.097+0000 7f3b6b400640 1 -- 172.21.15.133:0/3031965176 wait complete. 2024-03-07T06:53:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.099+0000 7f3b6b400640 1 Processor -- start 2024-03-07T06:53:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.099+0000 7f3b6b400640 1 -- start start 2024-03-07T06:53:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.100+0000 7f3b6b400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b640af950 0x7f3b641457e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.100+0000 7f3b6b400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b640ce5f0 0x7f3b640c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.100+0000 7f3b6b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b6415b1b0 0x7f3b640ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.100+0000 7f3b6b400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6415b590 con 0x7f3b6415b1b0 2024-03-07T06:53:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.100+0000 7f3b6b400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6406ee30 con 0x7f3b640ce5f0 2024-03-07T06:53:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.100+0000 7f3b6b400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6415b710 con 0x7f3b640af950 2024-03-07T06:53:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.100+0000 7f3b69400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b6415b1b0 0x7f3b640ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.101+0000 7f3b69400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b6415b1b0 0x7f3b640ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53854/0 (socket says 172.21.15.133:53854) 2024-03-07T06:53:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.101+0000 7f3b69400640 1 -- 172.21.15.133:0/3544975927 learned_addr learned my addr 172.21.15.133:0/3544975927 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.101+0000 7f3b63e00640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b640ce5f0 0x7f3b640c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.101+0000 7f3b68a00640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b640af950 0x7f3b641457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.101+0000 7f3b69400640 1 -- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b640af950 msgr2=0x7f3b641457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.101+0000 7f3b69400640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b640af950 0x7f3b641457e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.101+0000 7f3b69400640 1 -- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b640ce5f0 msgr2=0x7f3b640c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.101+0000 7f3b69400640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b640ce5f0 0x7f3b640c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.101+0000 7f3b69400640 1 -- 172.21.15.133:0/3544975927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b640ca9e0 con 0x7f3b6415b1b0 2024-03-07T06:53:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.101+0000 7f3b68a00640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b640af950 0x7f3b641457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:50.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.101+0000 7f3b69400640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b6415b1b0 0x7f3b640ca3b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c036040 tx=0x7f3b5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:50.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.102+0000 7f3b60c00640 1 -- 172.21.15.133:0/3544975927 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c043020 con 0x7f3b6415b1b0 2024-03-07T06:53:50.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.102+0000 7f3b60c00640 1 -- 172.21.15.133:0/3544975927 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5c03e450 con 0x7f3b6415b1b0 2024-03-07T06:53:50.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.102+0000 7f3b6b400640 1 -- 172.21.15.133:0/3544975927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b640cacc0 con 0x7f3b6415b1b0 2024-03-07T06:53:50.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.102+0000 7f3b60c00640 1 -- 172.21.15.133:0/3544975927 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c009510 con 0x7f3b6415b1b0 2024-03-07T06:53:50.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.102+0000 7f3b6b400640 1 -- 172.21.15.133:0/3544975927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b640cb280 con 0x7f3b6415b1b0 2024-03-07T06:53:50.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.103+0000 7f3b6b400640 1 -- 172.21.15.133:0/3544975927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b64062c90 con 0x7f3b6415b1b0 2024-03-07T06:53:50.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.105+0000 7f3b60c00640 1 -- 172.21.15.133:0/3544975927 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3b5c046020 con 0x7f3b6415b1b0 2024-03-07T06:53:50.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.105+0000 7f3b60c00640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3b4406b4d0 0x7f3b4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:50.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.106+0000 7f3b60c00640 1 -- 172.21.15.133:0/3544975927 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b5c0aca50 con 0x7f3b6415b1b0 2024-03-07T06:53:50.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.106+0000 7f3b68a00640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3b4406b4d0 0x7f3b4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:50.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.107+0000 7f3b68a00640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3b4406b4d0 0x7f3b4406d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b58000f60 tx=0x7f3b58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:50.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.111+0000 7f3b60c00640 1 -- 172.21.15.133:0/3544975927 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3b5c04d030 con 0x7f3b6415b1b0 2024-03-07T06:53:50.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.325+0000 7f3b6b400640 1 -- 172.21.15.133:0/3544975927 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b640cc660 con 0x7f3b6415b1b0 2024-03-07T06:53:50.326 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.327+0000 7f3b60c00640 1 -- 172.21.15.133:0/3544975927 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16661 (secure 0 0 0) 0x7f3b5c07b8d0 con 0x7f3b6415b1b0 2024-03-07T06:53:50.327 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:50.327 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":46,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:53:26.170797+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:53:22.398527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-07T06:53:23.628483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:53:50.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.330+0000 7f3b6b400640 1 -- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3b4406b4d0 msgr2=0x7f3b4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:50.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.330+0000 7f3b6b400640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3b4406b4d0 0x7f3b4406d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b58000f60 tx=0x7f3b58009040 comp rx=0 tx=0).stop 2024-03-07T06:53:50.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.330+0000 7f3b6b400640 1 -- 172.21.15.133:0/3544975927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b6415b1b0 msgr2=0x7f3b640ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:50.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.330+0000 7f3b6b400640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b6415b1b0 0x7f3b640ca3b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c036040 tx=0x7f3b5c004a40 comp rx=0 tx=0).stop 2024-03-07T06:53:50.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.330+0000 7f3b6b400640 1 -- 172.21.15.133:0/3544975927 shutdown_connections 2024-03-07T06:53:50.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.330+0000 7f3b6b400640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b6415b1b0 0x7f3b640ca3b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.330+0000 7f3b6b400640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3b4406b4d0 0x7f3b4406d990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.330+0000 7f3b6b400640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3b640ce5f0 0x7f3b640c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.330+0000 7f3b6b400640 1 --2- 172.21.15.133:0/3544975927 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3b640af950 0x7f3b641457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.330+0000 7f3b6b400640 1 -- 172.21.15.133:0/3544975927 >> 172.21.15.133:0/3544975927 conn(0x7f3b640b3120 msgr2=0x7f3b640b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:50.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.331+0000 7f3b6b400640 1 -- 172.21.15.133:0/3544975927 shutdown_connections 2024-03-07T06:53:50.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.331+0000 7f3b6b400640 1 -- 172.21.15.133:0/3544975927 wait complete. 2024-03-07T06:53:50.353 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:53:22.398527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:53:23.628483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:53:50.354 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:53:50.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.558+0000 7f3399800640 1 Processor -- start 2024-03-07T06:53:50.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.559+0000 7f3399800640 1 -- start start 2024-03-07T06:53:50.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.560+0000 7f3399800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f339415e870 0x7f33940cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:50.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.560+0000 7f3399800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3394070eb0 con 0x7f33940cea70 2024-03-07T06:53:50.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.560+0000 7f3399800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3394071190 con 0x7f33940d3310 2024-03-07T06:53:50.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.560+0000 7f3399800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3394071470 con 0x7f339415e870 2024-03-07T06:53:50.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.560+0000 7f338a000640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f33940cea70 0x7f339415ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:42610/0 (socket says 172.21.15.133:42610) 2024-03-07T06:53:50.560 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.560+0000 7f338a000640 1 -- 172.21.15.133:0/497466181 learned_addr learned my addr 172.21.15.133:0/497466181 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:50.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392a00640 1 --2- 172.21.15.133:0/497466181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f339415e870 0x7f33940cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:50.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 -- 172.21.15.133:0/497466181 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576706892 0 0) 0x7f3394070eb0 con 0x7f33940cea70 2024-03-07T06:53:50.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 -- 172.21.15.133:0/497466181 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3378003520 con 0x7f33940cea70 2024-03-07T06:53:50.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 -- 172.21.15.133:0/497466181 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2301793439 0 0) 0x7f3378003520 con 0x7f33940cea70 2024-03-07T06:53:50.562 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 -- 172.21.15.133:0/497466181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f339415e870 msgr2=0x7f33940cf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 --2- 172.21.15.133:0/497466181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f339415e870 0x7f33940cf190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 -- 172.21.15.133:0/497466181 >> v1:172.21.15.161:6789/0 conn(0x7f33940d3310 legacy=0x7f3394167e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 -- 172.21.15.133:0/497466181 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3394071750 con 0x7f33940cea70 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 -- 172.21.15.133:0/497466181 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f337c001670 con 0x7f33940cea70 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 -- 172.21.15.133:0/497466181 >> v1:172.21.15.133:6789/0 conn(0x7f33940cea70 legacy=0x7f339415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 --2- 172.21.15.133:0/497466181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f339415e870 0x7f33780039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 --2- 172.21.15.133:0/497466181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3378003fb0 0x7f3378004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 --2- 172.21.15.133:0/497466181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3378004d40 0x7f33780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 -- 172.21.15.133:0/497466181 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3394071470 con 0x7f3378004d40 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 -- 172.21.15.133:0/497466181 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337800bb00 con 0x7f3378003fb0 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.561+0000 7f3392000640 1 -- 172.21.15.133:0/497466181 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337800bd00 con 0x7f339415e870 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.562+0000 7f3393400640 1 --2- 172.21.15.133:0/497466181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3378004d40 0x7f33780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.562+0000 7f3392a00640 1 --2- 172.21.15.133:0/497466181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f339415e870 0x7f33780039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.562+0000 7f338a000640 1 --2- 172.21.15.133:0/497466181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3378003fb0 0x7f3378004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.562+0000 7f3393400640 1 -- 172.21.15.133:0/497466181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f339415e870 msgr2=0x7f33780039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:50.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.562+0000 7f3393400640 1 --2- 172.21.15.133:0/497466181 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f339415e870 0x7f33780039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.562+0000 7f3393400640 1 -- 172.21.15.133:0/497466181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3378003fb0 msgr2=0x7f3378004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:50.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.562+0000 7f3393400640 1 --2- 172.21.15.133:0/497466181 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3378003fb0 0x7f3378004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.564 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.562+0000 7f3393400640 1 -- 172.21.15.133:0/497466181 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3378003520 con 0x7f3378004d40 2024-03-07T06:53:50.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.563+0000 7f3393400640 1 --2- 172.21.15.133:0/497466181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3378004d40 0x7f33780051c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f338c001750 tx=0x7f3378005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3ec2e9dbe37ce9 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:50.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.564+0000 7f3392000640 1 -- 172.21.15.133:0/497466181 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c001370 con 0x7f3378004d40 2024-03-07T06:53:50.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.564+0000 7f3392000640 1 -- 172.21.15.133:0/497466181 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338c0052d0 con 0x7f3378004d40 2024-03-07T06:53:50.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.564+0000 7f3392000640 1 -- 172.21.15.133:0/497466181 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c003830 con 0x7f3378004d40 2024-03-07T06:53:50.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.564+0000 7f3399800640 1 -- 172.21.15.133:0/497466181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3378004d40 msgr2=0x7f33780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:50.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.564+0000 7f3399800640 1 --2- 172.21.15.133:0/497466181 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3378004d40 0x7f33780051c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f338c001750 tx=0x7f3378005830 comp rx=0 tx=0).stop 2024-03-07T06:53:50.565 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.564+0000 7f3393400640 1 -- 172.21.15.133:0/497466181 reap_dead start 2024-03-07T06:53:50.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.564+0000 7f3399800640 1 -- 172.21.15.133:0/497466181 shutdown_connections 2024-03-07T06:53:50.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.564+0000 7f3399800640 1 -- 172.21.15.133:0/497466181 >> 172.21.15.133:0/497466181 conn(0x7f33940d11e0 msgr2=0x7f33940d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:50.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.564+0000 7f3399800640 1 -- 172.21.15.133:0/497466181 shutdown_connections 2024-03-07T06:53:50.566 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.564+0000 7f3399800640 1 -- 172.21.15.133:0/497466181 wait complete. 2024-03-07T06:53:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.566+0000 7f3399800640 1 Processor -- start 2024-03-07T06:53:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.566+0000 7f3399800640 1 -- start start 2024-03-07T06:53:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.567+0000 7f3399800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3378004d40 0x7f3394152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.567+0000 7f3399800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f339414e990 0x7f33941f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.567+0000 7f3399800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33940cd1c0 0x7f33940cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.567+0000 7f3399800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339415f360 con 0x7f33940cd1c0 2024-03-07T06:53:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.567+0000 7f3399800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3394070b60 con 0x7f339414e990 2024-03-07T06:53:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.567+0000 7f3399800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339415f4e0 con 0x7f3378004d40 2024-03-07T06:53:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.567+0000 7f3393400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33940cd1c0 0x7f33940cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.567+0000 7f3393400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33940cd1c0 0x7f33940cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53886/0 (socket says 172.21.15.133:53886) 2024-03-07T06:53:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.567+0000 7f3393400640 1 -- 172.21.15.133:0/208118087 learned_addr learned my addr 172.21.15.133:0/208118087 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.568+0000 7f338a000640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f339414e990 0x7f33941f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.568+0000 7f3392a00640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3378004d40 0x7f3394152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.568+0000 7f3393400640 1 -- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3378004d40 msgr2=0x7f3394152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.568+0000 7f3393400640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3378004d40 0x7f3394152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.568+0000 7f3393400640 1 -- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f339414e990 msgr2=0x7f33941f49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.568+0000 7f3393400640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f339414e990 0x7f33941f49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.568+0000 7f3393400640 1 -- 172.21.15.133:0/208118087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33941f4ef0 con 0x7f33940cd1c0 2024-03-07T06:53:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.568+0000 7f338a000640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f339414e990 0x7f33941f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.568+0000 7f3393400640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33940cd1c0 0x7f33940cd640 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f338c001870 tx=0x7f338c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.568+0000 7f338b400640 1 -- 172.21.15.133:0/208118087 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c001370 con 0x7f33940cd1c0 2024-03-07T06:53:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.568+0000 7f338b400640 1 -- 172.21.15.133:0/208118087 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338c044070 con 0x7f33940cd1c0 2024-03-07T06:53:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.568+0000 7f338b400640 1 -- 172.21.15.133:0/208118087 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c007a30 con 0x7f33940cd1c0 2024-03-07T06:53:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.569+0000 7f3399800640 1 -- 172.21.15.133:0/208118087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3394155600 con 0x7f33940cd1c0 2024-03-07T06:53:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.569+0000 7f3399800640 1 -- 172.21.15.133:0/208118087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33940cb580 con 0x7f33940cd1c0 2024-03-07T06:53:50.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.570+0000 7f3399800640 1 -- 172.21.15.133:0/208118087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3394062c90 con 0x7f33940cd1c0 2024-03-07T06:53:50.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.572+0000 7f338b400640 1 -- 172.21.15.133:0/208118087 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f338c03b040 con 0x7f33940cd1c0 2024-03-07T06:53:50.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.573+0000 7f338b400640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f337006b500 0x7f337006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:50.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.573+0000 7f338b400640 1 -- 172.21.15.133:0/208118087 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f338c0ab630 con 0x7f33940cd1c0 2024-03-07T06:53:50.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.573+0000 7f3392a00640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f337006b500 0x7f337006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:50.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.574+0000 7f3392a00640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f337006b500 0x7f337006d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3384007380 tx=0x7f338400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.577+0000 7f338b400640 1 -- 172.21.15.133:0/208118087 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f33941f4ef0 con 0x7f33940cd1c0 2024-03-07T06:53:50.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.829+0000 7f3399800640 1 -- 172.21.15.133:0/208118087 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33941587f0 con 0x7f33940cd1c0 2024-03-07T06:53:50.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.830+0000 7f338b400640 1 -- 172.21.15.133:0/208118087 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f338c07ee20 con 0x7f33940cd1c0 2024-03-07T06:53:50.830 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:53:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.833+0000 7f3399800640 1 -- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f337006b500 msgr2=0x7f337006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.833+0000 7f3399800640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f337006b500 0x7f337006d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3384007380 tx=0x7f338400b040 comp rx=0 tx=0).stop 2024-03-07T06:53:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.833+0000 7f3399800640 1 -- 172.21.15.133:0/208118087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33940cd1c0 msgr2=0x7f33940cd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.833+0000 7f3399800640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33940cd1c0 0x7f33940cd640 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f338c001870 tx=0x7f338c001120 comp rx=0 tx=0).stop 2024-03-07T06:53:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.833+0000 7f3399800640 1 -- 172.21.15.133:0/208118087 shutdown_connections 2024-03-07T06:53:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.833+0000 7f3399800640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33940cd1c0 0x7f33940cd640 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.833+0000 7f3399800640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f337006b500 0x7f337006d9c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.833+0000 7f3399800640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f339414e990 0x7f33941f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.833+0000 7f3399800640 1 --2- 172.21.15.133:0/208118087 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3378004d40 0x7f3394152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:50.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.833+0000 7f3399800640 1 -- 172.21.15.133:0/208118087 >> 172.21.15.133:0/208118087 conn(0x7f33940d11e0 msgr2=0x7f3394161010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:50.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.834+0000 7f3399800640 1 -- 172.21.15.133:0/208118087 shutdown_connections 2024-03-07T06:53:50.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:50.834+0000 7f3399800640 1 -- 172.21.15.133:0/208118087 wait complete. 2024-03-07T06:53:50.856 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:53:51.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.066+0000 7f4885200640 1 Processor -- start 2024-03-07T06:53:51.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.066+0000 7f4885200640 1 -- start start 2024-03-07T06:53:51.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.068+0000 7f4885200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f48801625c0 0x7f4880071280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:51.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.068+0000 7f4885200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4880071ff0 con 0x7f488015e350 2024-03-07T06:53:51.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.068+0000 7f4885200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48800d23b0 con 0x7f48800b7200 2024-03-07T06:53:51.067 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.068+0000 7f4885200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48800d2160 con 0x7f48801625c0 2024-03-07T06:53:51.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.068+0000 7f487de00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f488015e350 0x7f4880161eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35030/0 (socket says 172.21.15.133:35030) 2024-03-07T06:53:51.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.068+0000 7f487de00640 1 -- 172.21.15.133:0/2764979384 learned_addr learned my addr 172.21.15.133:0/2764979384 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:51.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.068+0000 7f487d400640 1 --2- 172.21.15.133:0/2764979384 >> v2:172.21.15.161:3300/0 conn(0x7f48801625c0 0x7f4880071280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:51.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.068+0000 7f487ca00640 1 -- 172.21.15.133:0/2764979384 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101931181 0 0) 0x7f4880071ff0 con 0x7f488015e350 2024-03-07T06:53:51.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487ca00640 1 -- 172.21.15.133:0/2764979384 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4864003520 con 0x7f488015e350 2024-03-07T06:53:51.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487ca00640 1 -- 172.21.15.133:0/2764979384 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1808722426 0 0) 0x7f4864003520 con 0x7f488015e350 2024-03-07T06:53:51.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487ca00640 1 -- 172.21.15.133:0/2764979384 >> v2:172.21.15.161:3300/0 conn(0x7f48801625c0 msgr2=0x7f4880071280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:51.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487ca00640 1 --2- 172.21.15.133:0/2764979384 >> v2:172.21.15.161:3300/0 conn(0x7f48801625c0 0x7f4880071280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487ca00640 1 -- 172.21.15.133:0/2764979384 >> v1:172.21.15.161:6789/0 conn(0x7f48800b7200 legacy=0x7f48800717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:51.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487ca00640 1 -- 172.21.15.133:0/2764979384 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800d2690 con 0x7f488015e350 2024-03-07T06:53:51.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487ca00640 1 -- 172.21.15.133:0/2764979384 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f4870001670 con 0x7f488015e350 2024-03-07T06:53:51.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487ca00640 1 -- 172.21.15.133:0/2764979384 >> v1:172.21.15.133:6789/0 conn(0x7f488015e350 legacy=0x7f4880161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:51.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487ca00640 1 --2- 172.21.15.133:0/2764979384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48640039b0 0x7f4864003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:51.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487ca00640 1 --2- 172.21.15.133:0/2764979384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4864004800 0x7f4864004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:51.070 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487ca00640 1 --2- 172.21.15.133:0/2764979384 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48640051c0 0x7f486400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:51.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487ca00640 1 -- 172.21.15.133:0/2764979384 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48800d2160 con 0x7f4864004800 2024-03-07T06:53:51.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487ca00640 1 -- 172.21.15.133:0/2764979384 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486400c160 con 0x7f48640039b0 2024-03-07T06:53:51.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487ca00640 1 -- 172.21.15.133:0/2764979384 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486400c360 con 0x7f48640051c0 2024-03-07T06:53:51.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.069+0000 7f487d400640 1 --2- 172.21.15.133:0/2764979384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4864004800 0x7f4864004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:51.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.070+0000 7f487de00640 1 --2- 172.21.15.133:0/2764979384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48640039b0 0x7f4864003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:51.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.070+0000 7f487e800640 1 --2- 172.21.15.133:0/2764979384 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48640051c0 0x7f486400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:51.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.070+0000 7f487d400640 1 -- 172.21.15.133:0/2764979384 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48640051c0 msgr2=0x7f486400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:51.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.070+0000 7f487d400640 1 --2- 172.21.15.133:0/2764979384 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48640051c0 0x7f486400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.070+0000 7f487d400640 1 -- 172.21.15.133:0/2764979384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48640039b0 msgr2=0x7f4864003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:51.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.070+0000 7f487d400640 1 --2- 172.21.15.133:0/2764979384 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48640039b0 0x7f4864003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.070+0000 7f487d400640 1 -- 172.21.15.133:0/2764979384 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4864003520 con 0x7f4864004800 2024-03-07T06:53:51.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.070+0000 7f487e800640 1 --2- 172.21.15.133:0/2764979384 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48640051c0 0x7f486400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:51.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.070+0000 7f487e800640 1 -- 172.21.15.133:0/2764979384 reap_dead start 2024-03-07T06:53:51.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.071+0000 7f487d400640 1 --2- 172.21.15.133:0/2764979384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4864004800 0x7f4864004c80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f486c009ac0 tx=0x7f4864005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8062f869c1efe74 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:51.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.072+0000 7f487ca00640 1 -- 172.21.15.133:0/2764979384 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c039070 con 0x7f4864004800 2024-03-07T06:53:51.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.072+0000 7f487ca00640 1 -- 172.21.15.133:0/2764979384 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c02c6b0 con 0x7f4864004800 2024-03-07T06:53:51.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.072+0000 7f487ca00640 1 -- 172.21.15.133:0/2764979384 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c009690 con 0x7f4864004800 2024-03-07T06:53:51.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.072+0000 7f4885200640 1 -- 172.21.15.133:0/2764979384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4864004800 msgr2=0x7f4864004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:51.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.072+0000 7f4885200640 1 --2- 172.21.15.133:0/2764979384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4864004800 0x7f4864004c80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f486c009ac0 tx=0x7f4864005830 comp rx=0 tx=0).stop 2024-03-07T06:53:51.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.072+0000 7f4885200640 1 -- 172.21.15.133:0/2764979384 shutdown_connections 2024-03-07T06:53:51.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.072+0000 7f4885200640 1 --2- 172.21.15.133:0/2764979384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4864004800 0x7f4864004c80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.072+0000 7f4885200640 1 -- 172.21.15.133:0/2764979384 >> 172.21.15.133:0/2764979384 conn(0x7f48800d0e70 msgr2=0x7f48800d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:51.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.072+0000 7f4885200640 1 -- 172.21.15.133:0/2764979384 shutdown_connections 2024-03-07T06:53:51.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.073+0000 7f4885200640 1 -- 172.21.15.133:0/2764979384 wait complete. 2024-03-07T06:53:51.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.075+0000 7f4885200640 1 Processor -- start 2024-03-07T06:53:51.074 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.075+0000 7f4885200640 1 -- start start 2024-03-07T06:53:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.076+0000 7f4885200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4864004800 0x7f48800b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.076+0000 7f4885200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48800cf130 0x7f48801f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.077+0000 7f4885200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4880157550 0x7f48801579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.077+0000 7f4885200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4880161eb0 con 0x7f4880157550 2024-03-07T06:53:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.077+0000 7f4885200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488015e730 con 0x7f48800cf130 2024-03-07T06:53:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.077+0000 7f4885200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4880162030 con 0x7f4864004800 2024-03-07T06:53:51.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.077+0000 7f487e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4880157550 0x7f48801579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:51.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.077+0000 7f487e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4880157550 0x7f48801579d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42278/0 (socket says 172.21.15.133:42278) 2024-03-07T06:53:51.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.077+0000 7f487e800640 1 -- 172.21.15.133:0/839678156 learned_addr learned my addr 172.21.15.133:0/839678156 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:51.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.077+0000 7f487de00640 1 --2- 172.21.15.133:0/839678156 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4864004800 0x7f48800b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:51.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.077+0000 7f487d400640 1 --2- 172.21.15.133:0/839678156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48800cf130 0x7f48801f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:51.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.077+0000 7f487e800640 1 -- 172.21.15.133:0/839678156 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4864004800 msgr2=0x7f48800b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:51.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.077+0000 7f487e800640 1 --2- 172.21.15.133:0/839678156 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4864004800 0x7f48800b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.077+0000 7f487e800640 1 -- 172.21.15.133:0/839678156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48800cf130 msgr2=0x7f48801f77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:51.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.077+0000 7f487e800640 1 --2- 172.21.15.133:0/839678156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48800cf130 0x7f48801f77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.077+0000 7f487e800640 1 -- 172.21.15.133:0/839678156 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4880160660 con 0x7f4880157550 2024-03-07T06:53:51.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.078+0000 7f487e800640 1 --2- 172.21.15.133:0/839678156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4880157550 0x7f48801579d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4880161e10 tx=0x7f487800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.078+0000 7f4876000640 1 -- 172.21.15.133:0/839678156 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878018070 con 0x7f4880157550 2024-03-07T06:53:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.078+0000 7f4876000640 1 -- 172.21.15.133:0/839678156 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4878003b90 con 0x7f4880157550 2024-03-07T06:53:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.078+0000 7f4876000640 1 -- 172.21.15.133:0/839678156 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4878005070 con 0x7f4880157550 2024-03-07T06:53:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.078+0000 7f4885200640 1 -- 172.21.15.133:0/839678156 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4880161690 con 0x7f4880157550 2024-03-07T06:53:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.078+0000 7f4885200640 1 -- 172.21.15.133:0/839678156 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4880160bb0 con 0x7f4880157550 2024-03-07T06:53:51.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.080+0000 7f4885200640 1 -- 172.21.15.133:0/839678156 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4880062c90 con 0x7f4880157550 2024-03-07T06:53:51.080 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.080+0000 7f4876000640 1 -- 172.21.15.133:0/839678156 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f487801b070 con 0x7f4880157550 2024-03-07T06:53:51.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.081+0000 7f4876000640 1 --2- 172.21.15.133:0/839678156 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f486006b500 0x7f486006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:51.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.081+0000 7f4876000640 1 -- 172.21.15.133:0/839678156 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4878089950 con 0x7f4880157550 2024-03-07T06:53:51.081 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.082+0000 7f487de00640 1 --2- 172.21.15.133:0/839678156 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f486006b500 0x7f486006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:51.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.083+0000 7f487de00640 1 --2- 172.21.15.133:0/839678156 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f486006b500 0x7f486006d9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4870001c10 tx=0x7f4870009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:51.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.086+0000 7f4876000640 1 -- 172.21.15.133:0/839678156 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f4878058730 con 0x7f4880157550 2024-03-07T06:53:51.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.356+0000 7f4885200640 1 -- 172.21.15.133:0/839678156 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48800d1d10 con 0x7f4880157550 2024-03-07T06:53:51.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.357+0000 7f4876000640 1 -- 172.21.15.133:0/839678156 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4878058030 con 0x7f4880157550 2024-03-07T06:53:51.356 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:53:51.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.359+0000 7f4885200640 1 -- 172.21.15.133:0/839678156 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f486006b500 msgr2=0x7f486006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:51.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.359+0000 7f4885200640 1 --2- 172.21.15.133:0/839678156 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f486006b500 0x7f486006d9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4870001c10 tx=0x7f4870009040 comp rx=0 tx=0).stop 2024-03-07T06:53:51.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.359+0000 7f4885200640 1 -- 172.21.15.133:0/839678156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4880157550 msgr2=0x7f48801579d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:51.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.359+0000 7f4885200640 1 --2- 172.21.15.133:0/839678156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4880157550 0x7f48801579d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4880161e10 tx=0x7f487800dc00 comp rx=0 tx=0).stop 2024-03-07T06:53:51.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.359+0000 7f4885200640 1 -- 172.21.15.133:0/839678156 shutdown_connections 2024-03-07T06:53:51.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.359+0000 7f4885200640 1 --2- 172.21.15.133:0/839678156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4880157550 0x7f48801579d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.359+0000 7f4885200640 1 --2- 172.21.15.133:0/839678156 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f486006b500 0x7f486006d9c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.359+0000 7f4885200640 1 --2- 172.21.15.133:0/839678156 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48800cf130 0x7f48801f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.359+0000 7f4885200640 1 --2- 172.21.15.133:0/839678156 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4864004800 0x7f48800b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.359+0000 7f4885200640 1 -- 172.21.15.133:0/839678156 >> 172.21.15.133:0/839678156 conn(0x7f48800d0e70 msgr2=0x7f4880071280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:51.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.359+0000 7f4885200640 1 -- 172.21.15.133:0/839678156 shutdown_connections 2024-03-07T06:53:51.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.360+0000 7f4885200640 1 -- 172.21.15.133:0/839678156 wait complete. 2024-03-07T06:53:51.377 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:53:51.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.578+0000 7f41d6000640 1 Processor -- start 2024-03-07T06:53:51.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.579+0000 7f41d6000640 1 -- start start 2024-03-07T06:53:51.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.580+0000 7f41d6000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f41d0162110 0x7f41d01624f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:51.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.580+0000 7f41d6000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f41d00d3700 0x7f41d015efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:51.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.580+0000 7f41d6000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d0162a30 con 0x7f41d0070bc0 2024-03-07T06:53:51.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.580+0000 7f41d6000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d0071610 con 0x7f41d00d3700 2024-03-07T06:53:51.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.580+0000 7f41d6000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d00718f0 con 0x7f41d0162110 2024-03-07T06:53:51.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.580+0000 7f41cfe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f41d00d3700 0x7f41d015efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:51.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.580+0000 7f41cfe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f41d00d3700 0x7f41d015efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42284/0 (socket says 172.21.15.133:42284) 2024-03-07T06:53:51.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.580+0000 7f41cfe00640 1 -- 172.21.15.133:0/1671586828 learned_addr learned my addr 172.21.15.133:0/1671586828 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:51.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.581+0000 7f41cf400640 1 --2- 172.21.15.133:0/1671586828 >> v2:172.21.15.161:3300/0 conn(0x7f41d0162110 0x7f41d01624f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:51.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.581+0000 7f41cfe00640 1 -- 172.21.15.133:0/1671586828 >> v2:172.21.15.161:3300/0 conn(0x7f41d0162110 msgr2=0x7f41d01624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:51.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.581+0000 7f41cfe00640 1 --2- 172.21.15.133:0/1671586828 >> v2:172.21.15.161:3300/0 conn(0x7f41d0162110 0x7f41d01624f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.581+0000 7f41cfe00640 1 -- 172.21.15.133:0/1671586828 >> v1:172.21.15.161:6789/0 conn(0x7f41d0070bc0 legacy=0x7f41d015e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:51.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.581+0000 7f41cfe00640 1 -- 172.21.15.133:0/1671586828 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d0071bd0 con 0x7f41d00d3700 2024-03-07T06:53:51.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.582+0000 7f41cfe00640 1 --2- 172.21.15.133:0/1671586828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41d00d3700 0x7f41d015efe0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f41d008d270 tx=0x7f41c802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93279acbcdab8575 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:51.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.582+0000 7f41cea00640 1 -- 172.21.15.133:0/1671586828 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c802ca80 con 0x7f41d00d3700 2024-03-07T06:53:51.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.582+0000 7f41cea00640 1 -- 172.21.15.133:0/1671586828 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c802cc20 con 0x7f41d00d3700 2024-03-07T06:53:51.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.582+0000 7f41cea00640 1 -- 172.21.15.133:0/1671586828 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c80343f0 con 0x7f41d00d3700 2024-03-07T06:53:51.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.583+0000 7f41d6000640 1 -- 172.21.15.133:0/1671586828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41d00d3700 msgr2=0x7f41d015efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:51.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.583+0000 7f41d6000640 1 --2- 172.21.15.133:0/1671586828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41d00d3700 0x7f41d015efe0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f41d008d270 tx=0x7f41c802bee0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.583+0000 7f41d6000640 1 -- 172.21.15.133:0/1671586828 shutdown_connections 2024-03-07T06:53:51.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.583+0000 7f41d6000640 1 --2- 172.21.15.133:0/1671586828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41d00d3700 0x7f41d015efe0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.583+0000 7f41d6000640 1 --2- 172.21.15.133:0/1671586828 >> v2:172.21.15.161:3300/0 conn(0x7f41d0162110 0x7f41d01624f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.583+0000 7f41d6000640 1 -- 172.21.15.133:0/1671586828 >> 172.21.15.133:0/1671586828 conn(0x7f41d00d0e10 msgr2=0x7f41d00d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:51.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.583+0000 7f41d6000640 1 -- 172.21.15.133:0/1671586828 shutdown_connections 2024-03-07T06:53:51.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.583+0000 7f41d6000640 1 -- 172.21.15.133:0/1671586828 wait complete. 2024-03-07T06:53:51.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.585+0000 7f41d6000640 1 Processor -- start 2024-03-07T06:53:51.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.585+0000 7f41d6000640 1 -- start start 2024-03-07T06:53:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.586+0000 7f41d6000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41d0070bc0 0x7f41d014fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.586+0000 7f41d6000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d00d3700 0x7f41d00cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.586+0000 7f41d6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41d0162110 0x7f41d013ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.586+0000 7f41d6000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d0162e00 con 0x7f41d0162110 2024-03-07T06:53:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.586+0000 7f41d6000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d00d21d0 con 0x7f41d00d3700 2024-03-07T06:53:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.586+0000 7f41d6000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d0162f80 con 0x7f41d0070bc0 2024-03-07T06:53:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.586+0000 7f41cfe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41d0162110 0x7f41d013ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.586+0000 7f41cfe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41d0162110 0x7f41d013ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42286/0 (socket says 172.21.15.133:42286) 2024-03-07T06:53:51.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.586+0000 7f41cfe00640 1 -- 172.21.15.133:0/1824994234 learned_addr learned my addr 172.21.15.133:0/1824994234 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:51.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.587+0000 7f41c7e00640 1 --2- 172.21.15.133:0/1824994234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d00d3700 0x7f41d00cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:51.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.587+0000 7f41cf400640 1 --2- 172.21.15.133:0/1824994234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41d0070bc0 0x7f41d014fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:51.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.587+0000 7f41cfe00640 1 -- 172.21.15.133:0/1824994234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41d0070bc0 msgr2=0x7f41d014fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.587+0000 7f41cfe00640 1 --2- 172.21.15.133:0/1824994234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41d0070bc0 0x7f41d014fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.587+0000 7f41cfe00640 1 -- 172.21.15.133:0/1824994234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d00d3700 msgr2=0x7f41d00cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.587+0000 7f41cfe00640 1 --2- 172.21.15.133:0/1824994234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d00d3700 0x7f41d00cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.587+0000 7f41cfe00640 1 -- 172.21.15.133:0/1824994234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d015d020 con 0x7f41d0162110 2024-03-07T06:53:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.587+0000 7f41cfe00640 1 --2- 172.21.15.133:0/1824994234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41d0162110 0x7f41d013ede0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f41c8036040 tx=0x7f41c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.587+0000 7f41c7400640 1 -- 172.21.15.133:0/1824994234 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8043020 con 0x7f41d0162110 2024-03-07T06:53:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.587+0000 7f41c7400640 1 -- 172.21.15.133:0/1824994234 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c803e450 con 0x7f41d0162110 2024-03-07T06:53:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.587+0000 7f41d6000640 1 -- 172.21.15.133:0/1824994234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d015d300 con 0x7f41d0162110 2024-03-07T06:53:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.587+0000 7f41c7400640 1 -- 172.21.15.133:0/1824994234 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8009510 con 0x7f41d0162110 2024-03-07T06:53:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.588+0000 7f41d6000640 1 -- 172.21.15.133:0/1824994234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d015d8c0 con 0x7f41d0162110 2024-03-07T06:53:51.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.589+0000 7f41d6000640 1 -- 172.21.15.133:0/1824994234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d0062c90 con 0x7f41d0162110 2024-03-07T06:53:51.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.590+0000 7f41c7400640 1 -- 172.21.15.133:0/1824994234 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f41c8046020 con 0x7f41d0162110 2024-03-07T06:53:51.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.591+0000 7f41c7400640 1 --2- 172.21.15.133:0/1824994234 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f41ac06b4d0 0x7f41ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:51.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.591+0000 7f41c7400640 1 -- 172.21.15.133:0/1824994234 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41c80acbd0 con 0x7f41d0162110 2024-03-07T06:53:51.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.591+0000 7f41cf400640 1 --2- 172.21.15.133:0/1824994234 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f41ac06b4d0 0x7f41ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:51.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.592+0000 7f41cf400640 1 --2- 172.21.15.133:0/1824994234 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f41ac06b4d0 0x7f41ac06d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f41c0006fd0 tx=0x7f41c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:51.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.600+0000 7f41c7400640 1 -- 172.21.15.133:0/1824994234 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f41c804d030 con 0x7f41d0162110 2024-03-07T06:53:51.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.793+0000 7f41d6000640 1 -- 172.21.15.133:0/1824994234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f41d00cab80 con 0x7f41d0162110 2024-03-07T06:53:51.794 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.794+0000 7f41c7400640 1 -- 172.21.15.133:0/1824994234 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7f41c807b920 con 0x7f41d0162110 2024-03-07T06:53:51.794 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:53:51.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.797+0000 7f41d6000640 1 -- 172.21.15.133:0/1824994234 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f41ac06b4d0 msgr2=0x7f41ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:51.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.797+0000 7f41d6000640 1 --2- 172.21.15.133:0/1824994234 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f41ac06b4d0 0x7f41ac06d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f41c0006fd0 tx=0x7f41c0008040 comp rx=0 tx=0).stop 2024-03-07T06:53:51.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.797+0000 7f41d6000640 1 -- 172.21.15.133:0/1824994234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41d0162110 msgr2=0x7f41d013ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:51.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.797+0000 7f41d6000640 1 --2- 172.21.15.133:0/1824994234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41d0162110 0x7f41d013ede0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f41c8036040 tx=0x7f41c8004a40 comp rx=0 tx=0).stop 2024-03-07T06:53:51.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.797+0000 7f41d6000640 1 -- 172.21.15.133:0/1824994234 shutdown_connections 2024-03-07T06:53:51.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.797+0000 7f41d6000640 1 --2- 172.21.15.133:0/1824994234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41d0162110 0x7f41d013ede0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.797+0000 7f41d6000640 1 --2- 172.21.15.133:0/1824994234 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f41ac06b4d0 0x7f41ac06d990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.797+0000 7f41d6000640 1 --2- 172.21.15.133:0/1824994234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f41d00d3700 0x7f41d00cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.797+0000 7f41d6000640 1 --2- 172.21.15.133:0/1824994234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f41d0070bc0 0x7f41d014fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:51.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.797+0000 7f41d6000640 1 -- 172.21.15.133:0/1824994234 >> 172.21.15.133:0/1824994234 conn(0x7f41d00d0e10 msgr2=0x7f41d01616b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:51.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.797+0000 7f41d6000640 1 -- 172.21.15.133:0/1824994234 shutdown_connections 2024-03-07T06:53:51.798 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:51.797+0000 7f41d6000640 1 -- 172.21.15.133:0/1824994234 wait complete. 2024-03-07T06:53:51.817 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-07T06:53:52.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.017+0000 7f2a19000640 1 Processor -- start 2024-03-07T06:53:52.017 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.017+0000 7f2a19000640 1 -- start start 2024-03-07T06:53:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.019+0000 7f2a19000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2a14140cb0 0x7f2a14162290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.019+0000 7f2a19000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a140b70d0 0x7f2a140b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.019+0000 7f2a19000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2a140b9300 0x7f2a140b9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.019+0000 7f2a19000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a140b7a70 con 0x7f2a14140cb0 2024-03-07T06:53:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.019+0000 7f2a19000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a141435c0 con 0x7f2a140b9300 2024-03-07T06:53:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.019+0000 7f2a19000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a14144420 con 0x7f2a140b70d0 2024-03-07T06:53:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.019+0000 7f2a13400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2a14140cb0 0x7f2a14162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.019+0000 7f2a13400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f2a14140cb0 0x7f2a14162290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42302/0 (socket says 172.21.15.133:42302) 2024-03-07T06:53:52.019 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.019+0000 7f2a13400640 1 -- 172.21.15.133:0/1319457473 learned_addr learned my addr 172.21.15.133:0/1319457473 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.020+0000 7f2a0be00640 1 --2- 172.21.15.133:0/1319457473 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a140b70d0 0x7f2a140b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.020+0000 7f2a13e00640 1 --2- 172.21.15.133:0/1319457473 >> v2:172.21.15.161:3300/0 conn(0x7f2a140b9300 0x7f2a140b9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.020+0000 7f2a13400640 1 -- 172.21.15.133:0/1319457473 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a140b70d0 msgr2=0x7f2a140b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.020+0000 7f2a13400640 1 --2- 172.21.15.133:0/1319457473 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a140b70d0 0x7f2a140b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.020+0000 7f2a13400640 1 -- 172.21.15.133:0/1319457473 >> v2:172.21.15.161:3300/0 conn(0x7f2a140b9300 msgr2=0x7f2a140b9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.020+0000 7f2a13400640 1 --2- 172.21.15.133:0/1319457473 >> v2:172.21.15.161:3300/0 conn(0x7f2a140b9300 0x7f2a140b9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.020 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.020+0000 7f2a13400640 1 -- 172.21.15.133:0/1319457473 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a14143810 con 0x7f2a14140cb0 2024-03-07T06:53:52.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.021+0000 7f2a13400640 1 --2- 172.21.15.133:0/1319457473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a14140cb0 0x7f2a14162290 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2a14161930 tx=0x7f2a0402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac56580a00054edb server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:52.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.021+0000 7f2a12a00640 1 -- 172.21.15.133:0/1319457473 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04035020 con 0x7f2a14140cb0 2024-03-07T06:53:52.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.021+0000 7f2a12a00640 1 -- 172.21.15.133:0/1319457473 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a04031870 con 0x7f2a14140cb0 2024-03-07T06:53:52.021 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.021+0000 7f2a12a00640 1 -- 172.21.15.133:0/1319457473 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04031c40 con 0x7f2a14140cb0 2024-03-07T06:53:52.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.022+0000 7f2a19000640 1 -- 172.21.15.133:0/1319457473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a14140cb0 msgr2=0x7f2a14162290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:52.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.022+0000 7f2a19000640 1 --2- 172.21.15.133:0/1319457473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a14140cb0 0x7f2a14162290 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2a14161930 tx=0x7f2a0402b910 comp rx=0 tx=0).stop 2024-03-07T06:53:52.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.022+0000 7f2a19000640 1 -- 172.21.15.133:0/1319457473 shutdown_connections 2024-03-07T06:53:52.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.022+0000 7f2a19000640 1 --2- 172.21.15.133:0/1319457473 >> v2:172.21.15.161:3300/0 conn(0x7f2a140b9300 0x7f2a140b9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.022+0000 7f2a19000640 1 --2- 172.21.15.133:0/1319457473 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a140b70d0 0x7f2a140b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.022+0000 7f2a19000640 1 --2- 172.21.15.133:0/1319457473 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a14140cb0 0x7f2a14162290 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.022 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.022+0000 7f2a19000640 1 -- 172.21.15.133:0/1319457473 >> 172.21.15.133:0/1319457473 conn(0x7f2a140674e0 msgr2=0x7f2a140d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:52.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.022+0000 7f2a19000640 1 -- 172.21.15.133:0/1319457473 shutdown_connections 2024-03-07T06:53:52.023 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.022+0000 7f2a19000640 1 -- 172.21.15.133:0/1319457473 wait complete. 2024-03-07T06:53:52.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.025+0000 7f2a19000640 1 Processor -- start 2024-03-07T06:53:52.025 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.025+0000 7f2a19000640 1 -- start start 2024-03-07T06:53:52.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.026+0000 7f2a19000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a140b70d0 0x7f2a140c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:52.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.026+0000 7f2a19000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a140b9300 0x7f2a140c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:52.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.026+0000 7f2a19000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a14140cb0 0x7f2a140d20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:52.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.026+0000 7f2a19000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a141440b0 con 0x7f2a14140cb0 2024-03-07T06:53:52.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.026+0000 7f2a19000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a14143dd0 con 0x7f2a140b9300 2024-03-07T06:53:52.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.026+0000 7f2a19000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a141627d0 con 0x7f2a140b70d0 2024-03-07T06:53:52.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.026+0000 7f2a13e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a14140cb0 0x7f2a140d20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:52.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.027+0000 7f2a13e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a14140cb0 0x7f2a140d20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42306/0 (socket says 172.21.15.133:42306) 2024-03-07T06:53:52.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.027+0000 7f2a13e00640 1 -- 172.21.15.133:0/1448970621 learned_addr learned my addr 172.21.15.133:0/1448970621 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:52.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.027+0000 7f2a13400640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a140b70d0 0x7f2a140c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:52.027 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.027+0000 7f2a0be00640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a140b9300 0x7f2a140c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:52.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.027+0000 7f2a13e00640 1 -- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a140b70d0 msgr2=0x7f2a140c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:52.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.027+0000 7f2a13e00640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a140b70d0 0x7f2a140c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.027+0000 7f2a13e00640 1 -- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a140b9300 msgr2=0x7f2a140c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:52.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.027+0000 7f2a13e00640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a140b9300 0x7f2a140c9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.027+0000 7f2a13e00640 1 -- 172.21.15.133:0/1448970621 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1412eb40 con 0x7f2a14140cb0 2024-03-07T06:53:52.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.027+0000 7f2a0be00640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a140b9300 0x7f2a140c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:52.028 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.027+0000 7f2a13e00640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a14140cb0 0x7f2a140d20b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c007e70 tx=0x7f2a0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:52.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.028+0000 7f2a0b400640 1 -- 172.21.15.133:0/1448970621 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c014420 con 0x7f2a14140cb0 2024-03-07T06:53:52.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.028+0000 7f2a0b400640 1 -- 172.21.15.133:0/1448970621 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0c019070 con 0x7f2a14140cb0 2024-03-07T06:53:52.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.028+0000 7f2a0b400640 1 -- 172.21.15.133:0/1448970621 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c014820 con 0x7f2a14140cb0 2024-03-07T06:53:52.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.028+0000 7f2a19000640 1 -- 172.21.15.133:0/1448970621 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1412ee20 con 0x7f2a14140cb0 2024-03-07T06:53:52.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.028+0000 7f2a19000640 1 -- 172.21.15.133:0/1448970621 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1412f3e0 con 0x7f2a14140cb0 2024-03-07T06:53:52.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.029+0000 7f2a19000640 1 -- 172.21.15.133:0/1448970621 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a14062c90 con 0x7f2a14140cb0 2024-03-07T06:53:52.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.030+0000 7f2a0b400640 1 -- 172.21.15.133:0/1448970621 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f2a0c0149c0 con 0x7f2a14140cb0 2024-03-07T06:53:52.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.031+0000 7f2a0b400640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f29f006b4d0 0x7f29f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:52.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.031+0000 7f2a0b400640 1 -- 172.21.15.133:0/1448970621 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a0c08a970 con 0x7f2a14140cb0 2024-03-07T06:53:52.030 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.031+0000 7f2a13400640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f29f006b4d0 0x7f29f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:52.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.033+0000 7f2a13400640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f29f006b4d0 0x7f29f006d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2a04036000 tx=0x7f2a0403c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:52.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.035+0000 7f2a0b400640 1 -- 172.21.15.133:0/1448970621 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2a0c059880 con 0x7f2a14140cb0 2024-03-07T06:53:52.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.242+0000 7f2a19000640 1 -- 172.21.15.133:0/1448970621 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2a140b84c0 con 0x7f2a14140cb0 2024-03-07T06:53:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.243+0000 7f2a0b400640 1 -- 172.21.15.133:0/1448970621 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2a0c059180 con 0x7f2a14140cb0 2024-03-07T06:53:52.243 INFO:teuthology.orchestra.run.smithi133.stderr:listed 0 entries 2024-03-07T06:53:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.246+0000 7f2a19000640 1 -- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f29f006b4d0 msgr2=0x7f29f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.246+0000 7f2a19000640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f29f006b4d0 0x7f29f006d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2a04036000 tx=0x7f2a0403c040 comp rx=0 tx=0).stop 2024-03-07T06:53:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.246+0000 7f2a19000640 1 -- 172.21.15.133:0/1448970621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a14140cb0 msgr2=0x7f2a140d20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.246+0000 7f2a19000640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a14140cb0 0x7f2a140d20b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c007e70 tx=0x7f2a0c00e4b0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.246+0000 7f2a19000640 1 -- 172.21.15.133:0/1448970621 shutdown_connections 2024-03-07T06:53:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.246+0000 7f2a19000640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a14140cb0 0x7f2a140d20b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.246+0000 7f2a19000640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f29f006b4d0 0x7f29f006d990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.246+0000 7f2a19000640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a140b9300 0x7f2a140c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.246+0000 7f2a19000640 1 --2- 172.21.15.133:0/1448970621 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a140b70d0 0x7f2a140c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.246+0000 7f2a19000640 1 -- 172.21.15.133:0/1448970621 >> 172.21.15.133:0/1448970621 conn(0x7f2a140674e0 msgr2=0x7f2a14160760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.246+0000 7f2a19000640 1 -- 172.21.15.133:0/1448970621 shutdown_connections 2024-03-07T06:53:52.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.246+0000 7f2a19000640 1 -- 172.21.15.133:0/1448970621 wait complete. 2024-03-07T06:53:52.264 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-07T06:53:52.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:53:52.265 DEBUG:teuthology.orchestra.run.smithi133:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-07T06:53:52.429 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:53:52.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:53:52.429 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-07T06:53:52.472 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:53:52.472 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:52.514 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053429,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-07T06:53:52.514 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:53:52.514 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:52.552 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053417,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-07T06:53:52.552 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:53:52.552 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:53:52.606 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053429,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-07T06:53:52.606 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:53:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.808+0000 7eff9d600640 1 Processor -- start 2024-03-07T06:53:52.808 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.808+0000 7eff9d600640 1 -- start start 2024-03-07T06:53:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.810+0000 7eff9d600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff980b5dd0 0x7eff98144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.810+0000 7eff9d600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7eff980b43f0 0x7eff98162570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.810+0000 7eff9d600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff98162ab0 0x7eff98071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.810+0000 7eff9d600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff98141700 con 0x7eff98162ab0 2024-03-07T06:53:52.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.810+0000 7eff9d600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff98161e00 con 0x7eff980b43f0 2024-03-07T06:53:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.810+0000 7eff9d600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff980b4bd0 con 0x7eff980b5dd0 2024-03-07T06:53:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.810+0000 7eff97400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff98162ab0 0x7eff98071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.810+0000 7eff97400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7eff98162ab0 0x7eff98071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42316/0 (socket says 172.21.15.133:42316) 2024-03-07T06:53:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.810+0000 7eff97400640 1 -- 172.21.15.133:0/3138506822 learned_addr learned my addr 172.21.15.133:0/3138506822 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.811+0000 7eff96a00640 1 --2- 172.21.15.133:0/3138506822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff980b5dd0 0x7eff98144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.811+0000 7eff8fe00640 1 --2- 172.21.15.133:0/3138506822 >> v2:172.21.15.161:3300/0 conn(0x7eff980b43f0 0x7eff98162570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.811+0000 7eff97400640 1 -- 172.21.15.133:0/3138506822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff980b5dd0 msgr2=0x7eff98144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.811+0000 7eff97400640 1 --2- 172.21.15.133:0/3138506822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff980b5dd0 0x7eff98144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.811+0000 7eff97400640 1 -- 172.21.15.133:0/3138506822 >> v2:172.21.15.161:3300/0 conn(0x7eff980b43f0 msgr2=0x7eff98162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:52.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.811+0000 7eff97400640 1 --2- 172.21.15.133:0/3138506822 >> v2:172.21.15.161:3300/0 conn(0x7eff980b43f0 0x7eff98162570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.811+0000 7eff97400640 1 -- 172.21.15.133:0/3138506822 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff980b4eb0 con 0x7eff98162ab0 2024-03-07T06:53:52.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.812+0000 7eff97400640 1 --2- 172.21.15.133:0/3138506822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff98162ab0 0x7eff98071410 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7eff980b4de0 tx=0x7eff9002be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7a9fc6c537719a server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:52.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.812+0000 7eff96000640 1 -- 172.21.15.133:0/3138506822 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9002c9d0 con 0x7eff98162ab0 2024-03-07T06:53:52.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.812+0000 7eff96000640 1 -- 172.21.15.133:0/3138506822 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9002cb70 con 0x7eff98162ab0 2024-03-07T06:53:52.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.812+0000 7eff96000640 1 -- 172.21.15.133:0/3138506822 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff900343f0 con 0x7eff98162ab0 2024-03-07T06:53:52.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.813+0000 7eff9d600640 1 -- 172.21.15.133:0/3138506822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff98162ab0 msgr2=0x7eff98071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:52.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.813+0000 7eff9d600640 1 --2- 172.21.15.133:0/3138506822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff98162ab0 0x7eff98071410 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7eff980b4de0 tx=0x7eff9002be30 comp rx=0 tx=0).stop 2024-03-07T06:53:52.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.813+0000 7eff9d600640 1 -- 172.21.15.133:0/3138506822 shutdown_connections 2024-03-07T06:53:52.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.813+0000 7eff9d600640 1 --2- 172.21.15.133:0/3138506822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff98162ab0 0x7eff98071410 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.813 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.813+0000 7eff9d600640 1 --2- 172.21.15.133:0/3138506822 >> v2:172.21.15.161:3300/0 conn(0x7eff980b43f0 0x7eff98162570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.813+0000 7eff9d600640 1 --2- 172.21.15.133:0/3138506822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff980b5dd0 0x7eff98144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.813+0000 7eff9d600640 1 -- 172.21.15.133:0/3138506822 >> 172.21.15.133:0/3138506822 conn(0x7eff980d0e70 msgr2=0x7eff980d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:52.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.813+0000 7eff9d600640 1 -- 172.21.15.133:0/3138506822 shutdown_connections 2024-03-07T06:53:52.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.813+0000 7eff9d600640 1 -- 172.21.15.133:0/3138506822 wait complete. 2024-03-07T06:53:52.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.815+0000 7eff9d600640 1 Processor -- start 2024-03-07T06:53:52.814 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.815+0000 7eff9d600640 1 -- start start 2024-03-07T06:53:52.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.816+0000 7eff9d600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff980b43f0 0x7eff98159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:52.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.816+0000 7eff9d600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff980b5dd0 0x7eff9815c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:52.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.816+0000 7eff9d600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff9815c6e0 0x7eff981430e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:52.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.816+0000 7eff9d600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff980b61b0 con 0x7eff980b43f0 2024-03-07T06:53:52.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.816+0000 7eff9d600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff98162050 con 0x7eff980b5dd0 2024-03-07T06:53:52.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.816+0000 7eff9d600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff980b4870 con 0x7eff9815c6e0 2024-03-07T06:53:52.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.816+0000 7eff96a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff980b43f0 0x7eff98159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:52.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.816+0000 7eff96a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff980b43f0 0x7eff98159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42322/0 (socket says 172.21.15.133:42322) 2024-03-07T06:53:52.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.816+0000 7eff96a00640 1 -- 172.21.15.133:0/915254343 learned_addr learned my addr 172.21.15.133:0/915254343 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:52.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.817+0000 7eff96a00640 1 -- 172.21.15.133:0/915254343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff9815c6e0 msgr2=0x7eff981430e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:52.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.817+0000 7eff96a00640 1 --2- 172.21.15.133:0/915254343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff9815c6e0 0x7eff981430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.817+0000 7eff96a00640 1 -- 172.21.15.133:0/915254343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff980b5dd0 msgr2=0x7eff9815c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:52.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.817+0000 7eff96a00640 1 --2- 172.21.15.133:0/915254343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff980b5dd0 0x7eff9815c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:52.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.817+0000 7eff96a00640 1 -- 172.21.15.133:0/915254343 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9815a6d0 con 0x7eff980b43f0 2024-03-07T06:53:52.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.817+0000 7eff96a00640 1 --2- 172.21.15.133:0/915254343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff980b43f0 0x7eff98159e40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7eff8800d800 tx=0x7eff8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:52.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.817+0000 7eff8ea00640 1 -- 172.21.15.133:0/915254343 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8800b6c0 con 0x7eff980b43f0 2024-03-07T06:53:52.818 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.817+0000 7eff8ea00640 1 -- 172.21.15.133:0/915254343 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff88019070 con 0x7eff980b43f0 2024-03-07T06:53:52.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.817+0000 7eff8ea00640 1 -- 172.21.15.133:0/915254343 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8800ba80 con 0x7eff980b43f0 2024-03-07T06:53:52.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.817+0000 7eff9d600640 1 -- 172.21.15.133:0/915254343 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff98142320 con 0x7eff980b43f0 2024-03-07T06:53:52.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.817+0000 7eff9d600640 1 -- 172.21.15.133:0/915254343 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff981428e0 con 0x7eff980b43f0 2024-03-07T06:53:52.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.818+0000 7eff9d600640 1 -- 172.21.15.133:0/915254343 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff98062c90 con 0x7eff980b43f0 2024-03-07T06:53:52.819 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.820+0000 7eff8ea00640 1 -- 172.21.15.133:0/915254343 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7eff8800bc20 con 0x7eff980b43f0 2024-03-07T06:53:52.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.821+0000 7eff8ea00640 1 --2- 172.21.15.133:0/915254343 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7eff7406b4d0 0x7eff7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:52.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.821+0000 7eff8ea00640 1 -- 172.21.15.133:0/915254343 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff8808a640 con 0x7eff980b43f0 2024-03-07T06:53:52.821 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.822+0000 7eff8fe00640 1 --2- 172.21.15.133:0/915254343 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7eff7406b4d0 0x7eff7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:52.822 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.823+0000 7eff8fe00640 1 --2- 172.21.15.133:0/915254343 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7eff7406b4d0 0x7eff7406d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7eff981f7b10 tx=0x7eff80008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:52.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:52.826+0000 7eff8ea00640 1 -- 172.21.15.133:0/915254343 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7eff88059530 con 0x7eff980b43f0 2024-03-07T06:53:53.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.059+0000 7eff9d600640 1 -- 172.21.15.133:0/915254343 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff980d0c10 con 0x7eff980b43f0 2024-03-07T06:53:53.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.060+0000 7eff8ea00640 1 -- 172.21.15.133:0/915254343 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4980 (secure 0 0 0) 0x7eff88058e30 con 0x7eff980b43f0 2024-03-07T06:53:53.059 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:53.059 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/44975892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":44975892},{"type":"v1","addr":"172.21.15.161:6835","nonce":44975892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4972,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/906115542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":906115542},{"type":"v1","addr":"172.21.15.133:6833","nonce":906115542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1998424684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1998424684},{"type":"v1","addr":"172.21.15.161:6837","nonce":1998424684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:53:24.627534+0000","modified":"2024-03-07T06:53:27.698471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/2951618124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2951618124},{"type":"v1","addr":"172.21.15.133:6835","nonce":2951618124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":3}]} 2024-03-07T06:53:53.060 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 27 2024-03-07T06:53:53.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.062+0000 7eff9d600640 1 -- 172.21.15.133:0/915254343 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7eff7406b4d0 msgr2=0x7eff7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:53.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.062+0000 7eff9d600640 1 --2- 172.21.15.133:0/915254343 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7eff7406b4d0 0x7eff7406d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7eff981f7b10 tx=0x7eff80008040 comp rx=0 tx=0).stop 2024-03-07T06:53:53.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.062+0000 7eff9d600640 1 -- 172.21.15.133:0/915254343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff980b43f0 msgr2=0x7eff98159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:53.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.062+0000 7eff9d600640 1 --2- 172.21.15.133:0/915254343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff980b43f0 0x7eff98159e40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7eff8800d800 tx=0x7eff8800dcc0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.062+0000 7eff9d600640 1 -- 172.21.15.133:0/915254343 shutdown_connections 2024-03-07T06:53:53.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.062+0000 7eff9d600640 1 --2- 172.21.15.133:0/915254343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff9815c6e0 0x7eff981430e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.062+0000 7eff9d600640 1 --2- 172.21.15.133:0/915254343 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7eff7406b4d0 0x7eff7406d990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.062+0000 7eff9d600640 1 --2- 172.21.15.133:0/915254343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff980b5dd0 0x7eff9815c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.062+0000 7eff9d600640 1 --2- 172.21.15.133:0/915254343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff980b43f0 0x7eff98159e40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.062+0000 7eff9d600640 1 -- 172.21.15.133:0/915254343 >> 172.21.15.133:0/915254343 conn(0x7eff980d0e70 msgr2=0x7eff980d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:53.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.063+0000 7eff9d600640 1 -- 172.21.15.133:0/915254343 shutdown_connections 2024-03-07T06:53:53.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.063+0000 7eff9d600640 1 -- 172.21.15.133:0/915254343 wait complete. 2024-03-07T06:53:53.081 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:53:53.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.294+0000 7f3efa800640 1 Processor -- start 2024-03-07T06:53:53.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.294+0000 7f3efa800640 1 -- start start 2024-03-07T06:53:53.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.295+0000 7f3efa800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3ef4140cb0 0x7f3ef4162290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:53.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.296+0000 7f3efa800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ef40b70d0 0x7f3ef40b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:53.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.296+0000 7f3efa800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3ef40b9300 0x7f3ef40b9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:53.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.296+0000 7f3efa800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef40b7a70 con 0x7f3ef4140cb0 2024-03-07T06:53:53.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.296+0000 7f3efa800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef41435c0 con 0x7f3ef40b9300 2024-03-07T06:53:53.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.296+0000 7f3efa800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4144420 con 0x7f3ef40b70d0 2024-03-07T06:53:53.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.296+0000 7f3ef9400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3ef4140cb0 0x7f3ef4162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:53.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.296+0000 7f3ef9400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f3ef4140cb0 0x7f3ef4162290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42336/0 (socket says 172.21.15.133:42336) 2024-03-07T06:53:53.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.296+0000 7f3ef9400640 1 -- 172.21.15.133:0/601719320 learned_addr learned my addr 172.21.15.133:0/601719320 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:53.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.296+0000 7f3ef9e00640 1 --2- 172.21.15.133:0/601719320 >> v2:172.21.15.161:3300/0 conn(0x7f3ef40b9300 0x7f3ef40b9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:53.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.296+0000 7f3ef8a00640 1 --2- 172.21.15.133:0/601719320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ef40b70d0 0x7f3ef40b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:53.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.296+0000 7f3ef9400640 1 -- 172.21.15.133:0/601719320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ef40b70d0 msgr2=0x7f3ef40b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:53.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.296+0000 7f3ef9400640 1 --2- 172.21.15.133:0/601719320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ef40b70d0 0x7f3ef40b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.296+0000 7f3ef9400640 1 -- 172.21.15.133:0/601719320 >> v2:172.21.15.161:3300/0 conn(0x7f3ef40b9300 msgr2=0x7f3ef40b9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:53.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.296+0000 7f3ef9400640 1 --2- 172.21.15.133:0/601719320 >> v2:172.21.15.161:3300/0 conn(0x7f3ef40b9300 0x7f3ef40b9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.298 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.297+0000 7f3ef9400640 1 -- 172.21.15.133:0/601719320 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef4143810 con 0x7f3ef4140cb0 2024-03-07T06:53:53.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.298+0000 7f3ef9400640 1 --2- 172.21.15.133:0/601719320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef4140cb0 0x7f3ef4162290 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4161930 tx=0x7f3ee802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c4d974262de192 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:53.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.298+0000 7f3ef3e00640 1 -- 172.21.15.133:0/601719320 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8035020 con 0x7f3ef4140cb0 2024-03-07T06:53:53.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.298+0000 7f3ef3e00640 1 -- 172.21.15.133:0/601719320 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee8031870 con 0x7f3ef4140cb0 2024-03-07T06:53:53.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.298+0000 7f3ef3e00640 1 -- 172.21.15.133:0/601719320 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8031c40 con 0x7f3ef4140cb0 2024-03-07T06:53:53.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.299+0000 7f3efa800640 1 -- 172.21.15.133:0/601719320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef4140cb0 msgr2=0x7f3ef4162290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:53.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.299+0000 7f3efa800640 1 --2- 172.21.15.133:0/601719320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef4140cb0 0x7f3ef4162290 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4161930 tx=0x7f3ee802b910 comp rx=0 tx=0).stop 2024-03-07T06:53:53.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.299+0000 7f3efa800640 1 -- 172.21.15.133:0/601719320 shutdown_connections 2024-03-07T06:53:53.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.299+0000 7f3efa800640 1 --2- 172.21.15.133:0/601719320 >> v2:172.21.15.161:3300/0 conn(0x7f3ef40b9300 0x7f3ef40b9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.299+0000 7f3efa800640 1 --2- 172.21.15.133:0/601719320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ef40b70d0 0x7f3ef40b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.299+0000 7f3efa800640 1 --2- 172.21.15.133:0/601719320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef4140cb0 0x7f3ef4162290 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.299+0000 7f3efa800640 1 -- 172.21.15.133:0/601719320 >> 172.21.15.133:0/601719320 conn(0x7f3ef40674e0 msgr2=0x7f3ef40d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:53.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.299+0000 7f3efa800640 1 -- 172.21.15.133:0/601719320 shutdown_connections 2024-03-07T06:53:53.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.299+0000 7f3efa800640 1 -- 172.21.15.133:0/601719320 wait complete. 2024-03-07T06:53:53.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.301+0000 7f3efa800640 1 Processor -- start 2024-03-07T06:53:53.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.301+0000 7f3efa800640 1 -- start start 2024-03-07T06:53:53.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.302+0000 7f3efa800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ef40b70d0 0x7f3ef413fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:53.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.302+0000 7f3efa800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef40b9300 0x7f3ef41f3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:53.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.302+0000 7f3efa800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ef4140cb0 0x7f3ef413d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:53.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.302+0000 7f3efa800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef41440b0 con 0x7f3ef40b9300 2024-03-07T06:53:53.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.302+0000 7f3efa800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4143dd0 con 0x7f3ef40b70d0 2024-03-07T06:53:53.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.302+0000 7f3efa800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef41627d0 con 0x7f3ef4140cb0 2024-03-07T06:53:53.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.302+0000 7f3ef8a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef40b9300 0x7f3ef41f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:53.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.302+0000 7f3ef8a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef40b9300 0x7f3ef41f3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42342/0 (socket says 172.21.15.133:42342) 2024-03-07T06:53:53.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.302+0000 7f3ef8a00640 1 -- 172.21.15.133:0/1941262901 learned_addr learned my addr 172.21.15.133:0/1941262901 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:53.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.303+0000 7f3ef9400640 1 --2- 172.21.15.133:0/1941262901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ef40b70d0 0x7f3ef413fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:53.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.303+0000 7f3ef8a00640 1 -- 172.21.15.133:0/1941262901 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ef4140cb0 msgr2=0x7f3ef413d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:53.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.303+0000 7f3ef8a00640 1 --2- 172.21.15.133:0/1941262901 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ef4140cb0 0x7f3ef413d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.303+0000 7f3ef8a00640 1 -- 172.21.15.133:0/1941262901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ef40b70d0 msgr2=0x7f3ef413fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:53.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.303+0000 7f3ef8a00640 1 --2- 172.21.15.133:0/1941262901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ef40b70d0 0x7f3ef413fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.303+0000 7f3ef8a00640 1 -- 172.21.15.133:0/1941262901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef413e840 con 0x7f3ef40b9300 2024-03-07T06:53:53.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.303+0000 7f3ef8a00640 1 --2- 172.21.15.133:0/1941262901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef40b9300 0x7f3ef41f3620 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000d830 tx=0x7f3ee000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:53.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.303+0000 7f3ef1600640 1 -- 172.21.15.133:0/1941262901 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000b6c0 con 0x7f3ef40b9300 2024-03-07T06:53:53.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.303+0000 7f3ef1600640 1 -- 172.21.15.133:0/1941262901 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee0019070 con 0x7f3ef40b9300 2024-03-07T06:53:53.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.303+0000 7f3ef1600640 1 -- 172.21.15.133:0/1941262901 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000ba80 con 0x7f3ef40b9300 2024-03-07T06:53:53.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.303+0000 7f3efa800640 1 -- 172.21.15.133:0/1941262901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef4152790 con 0x7f3ef40b9300 2024-03-07T06:53:53.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.304+0000 7f3efa800640 1 -- 172.21.15.133:0/1941262901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef4153030 con 0x7f3ef40b9300 2024-03-07T06:53:53.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.305+0000 7f3efa800640 1 -- 172.21.15.133:0/1941262901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef4062c90 con 0x7f3ef40b9300 2024-03-07T06:53:53.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.306+0000 7f3ef1600640 1 -- 172.21.15.133:0/1941262901 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f3ee001e460 con 0x7f3ef40b9300 2024-03-07T06:53:53.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.306+0000 7f3ef1600640 1 --2- 172.21.15.133:0/1941262901 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3ed406b4d0 0x7f3ed406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:53.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.306+0000 7f3ef1600640 1 -- 172.21.15.133:0/1941262901 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ee008a8d0 con 0x7f3ef40b9300 2024-03-07T06:53:53.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.307+0000 7f3ef9400640 1 --2- 172.21.15.133:0/1941262901 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3ed406b4d0 0x7f3ed406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:53.307 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.308+0000 7f3ef9400640 1 --2- 172.21.15.133:0/1941262901 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3ed406b4d0 0x7f3ed406d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8036000 tx=0x7f3ee8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:53.310 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.311+0000 7f3ef1600640 1 -- 172.21.15.133:0/1941262901 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3ef413e840 con 0x7f3ef40b9300 2024-03-07T06:53:53.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.508+0000 7f3efa800640 1 -- 172.21.15.133:0/1941262901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ef4072000 con 0x7f3ef40b9300 2024-03-07T06:53:53.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.509+0000 7f3ef1600640 1 -- 172.21.15.133:0/1941262901 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16661 (secure 0 0 0) 0x7f3ee0059740 con 0x7f3ef40b9300 2024-03-07T06:53:53.509 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:53.510 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":46,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:53:26.170797+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:53:22.398527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-07T06:53:23.628483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:53:53.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.512+0000 7f3efa800640 1 -- 172.21.15.133:0/1941262901 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3ed406b4d0 msgr2=0x7f3ed406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:53.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.512+0000 7f3efa800640 1 --2- 172.21.15.133:0/1941262901 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3ed406b4d0 0x7f3ed406d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8036000 tx=0x7f3ee8008030 comp rx=0 tx=0).stop 2024-03-07T06:53:53.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.512+0000 7f3efa800640 1 -- 172.21.15.133:0/1941262901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef40b9300 msgr2=0x7f3ef41f3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:53.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.512+0000 7f3efa800640 1 --2- 172.21.15.133:0/1941262901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef40b9300 0x7f3ef41f3620 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000d830 tx=0x7f3ee000dcf0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.512+0000 7f3efa800640 1 -- 172.21.15.133:0/1941262901 shutdown_connections 2024-03-07T06:53:53.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.512+0000 7f3efa800640 1 --2- 172.21.15.133:0/1941262901 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ef4140cb0 0x7f3ef413d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.512+0000 7f3efa800640 1 --2- 172.21.15.133:0/1941262901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ef40b9300 0x7f3ef41f3620 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.512 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.512+0000 7f3efa800640 1 --2- 172.21.15.133:0/1941262901 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f3ed406b4d0 0x7f3ed406d990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.512+0000 7f3efa800640 1 --2- 172.21.15.133:0/1941262901 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ef40b70d0 0x7f3ef413fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.512+0000 7f3efa800640 1 -- 172.21.15.133:0/1941262901 >> 172.21.15.133:0/1941262901 conn(0x7f3ef40674e0 msgr2=0x7f3ef40d0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:53.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.512+0000 7f3efa800640 1 -- 172.21.15.133:0/1941262901 shutdown_connections 2024-03-07T06:53:53.513 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.512+0000 7f3efa800640 1 -- 172.21.15.133:0/1941262901 wait complete. 2024-03-07T06:53:53.529 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-07T06:51:04.544943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-07T06:53:22.398527+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-03-07T06:53:23.628483+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-07T06:53:53.529 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-07T06:53:53.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.746+0000 7f193d600640 1 Processor -- start 2024-03-07T06:53:53.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.746+0000 7f193d600640 1 -- start start 2024-03-07T06:53:53.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.747+0000 7f193d600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f193815b370 0x7f19380cef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:53.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.747+0000 7f193d600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f193806d180 0x7f19380cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:53.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.747+0000 7f193d600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f193815b750 0x7f193815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:53.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.747+0000 7f193d600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193806d740 con 0x7f193815b370 2024-03-07T06:53:53.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.747+0000 7f193d600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193806da20 con 0x7f193806d180 2024-03-07T06:53:53.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.747+0000 7f193d600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193806dd00 con 0x7f193815b750 2024-03-07T06:53:53.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.748+0000 7f1936a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f193815b370 0x7f19380cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:53.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.748+0000 7f1936a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f193815b370 0x7f19380cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42352/0 (socket says 172.21.15.133:42352) 2024-03-07T06:53:53.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.748+0000 7f1936a00640 1 -- 172.21.15.133:0/1972938178 learned_addr learned my addr 172.21.15.133:0/1972938178 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:53.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.748+0000 7f1937400640 1 --2- 172.21.15.133:0/1972938178 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f193815b750 0x7f193815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:53.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.748+0000 7f1936a00640 1 -- 172.21.15.133:0/1972938178 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f193815b750 msgr2=0x7f193815bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:53.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.748+0000 7f1936a00640 1 --2- 172.21.15.133:0/1972938178 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f193815b750 0x7f193815bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.748+0000 7f192fe00640 1 --2- 172.21.15.133:0/1972938178 >> v2:172.21.15.161:3300/0 conn(0x7f193806d180 0x7f19380cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:53.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.748+0000 7f1936a00640 1 -- 172.21.15.133:0/1972938178 >> v2:172.21.15.161:3300/0 conn(0x7f193806d180 msgr2=0x7f19380cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:53.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.748+0000 7f1936a00640 1 --2- 172.21.15.133:0/1972938178 >> v2:172.21.15.161:3300/0 conn(0x7f193806d180 0x7f19380cf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.748+0000 7f1936a00640 1 -- 172.21.15.133:0/1972938178 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193806dfe0 con 0x7f193815b370 2024-03-07T06:53:53.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.749+0000 7f1936a00640 1 --2- 172.21.15.133:0/1972938178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193815b370 0x7f19380cef50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f19380cdfa0 tx=0x7f192802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca09de9b73c74b2e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:53.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.749+0000 7f1936000640 1 -- 172.21.15.133:0/1972938178 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1928035020 con 0x7f193815b370 2024-03-07T06:53:53.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.750+0000 7f1936000640 1 -- 172.21.15.133:0/1972938178 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192802d040 con 0x7f193815b370 2024-03-07T06:53:53.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.750+0000 7f1936000640 1 -- 172.21.15.133:0/1972938178 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1928031a80 con 0x7f193815b370 2024-03-07T06:53:53.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.750+0000 7f193d600640 1 -- 172.21.15.133:0/1972938178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193815b370 msgr2=0x7f19380cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:53.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.750+0000 7f193d600640 1 --2- 172.21.15.133:0/1972938178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193815b370 0x7f19380cef50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f19380cdfa0 tx=0x7f192802b900 comp rx=0 tx=0).stop 2024-03-07T06:53:53.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.750+0000 7f193d600640 1 -- 172.21.15.133:0/1972938178 shutdown_connections 2024-03-07T06:53:53.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.750+0000 7f193d600640 1 --2- 172.21.15.133:0/1972938178 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f193815b750 0x7f193815bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.750+0000 7f193d600640 1 --2- 172.21.15.133:0/1972938178 >> v2:172.21.15.161:3300/0 conn(0x7f193806d180 0x7f19380cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.750+0000 7f193d600640 1 --2- 172.21.15.133:0/1972938178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193815b370 0x7f19380cef50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.750+0000 7f193d600640 1 -- 172.21.15.133:0/1972938178 >> 172.21.15.133:0/1972938178 conn(0x7f19380b1d30 msgr2=0x7f19380b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:53.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.750+0000 7f193d600640 1 -- 172.21.15.133:0/1972938178 shutdown_connections 2024-03-07T06:53:53.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.750+0000 7f193d600640 1 -- 172.21.15.133:0/1972938178 wait complete. 2024-03-07T06:53:53.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.752+0000 7f193d600640 1 Processor -- start 2024-03-07T06:53:53.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.752+0000 7f193d600640 1 -- start start 2024-03-07T06:53:53.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.753+0000 7f193d600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193806d180 0x7f193813c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:53.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.753+0000 7f193d600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f193815b370 0x7f19381f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:53.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.753+0000 7f193d600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193815b750 0x7f19381e5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:53.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.753+0000 7f193d600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1938072230 con 0x7f193815b750 2024-03-07T06:53:53.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.753+0000 7f193d600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193815c140 con 0x7f193806d180 2024-03-07T06:53:53.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.753+0000 7f193d600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19380723b0 con 0x7f193815b370 2024-03-07T06:53:53.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.753+0000 7f1937400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193815b750 0x7f19381e5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:53.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.753+0000 7f1937400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193815b750 0x7f19381e5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42362/0 (socket says 172.21.15.133:42362) 2024-03-07T06:53:53.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.753+0000 7f1937400640 1 -- 172.21.15.133:0/220157009 learned_addr learned my addr 172.21.15.133:0/220157009 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:53.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.754+0000 7f1936a00640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193806d180 0x7f193813c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:53.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.754+0000 7f192fe00640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f193815b370 0x7f19381f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:53.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.754+0000 7f1937400640 1 -- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f193815b370 msgr2=0x7f19381f7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:53.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.754+0000 7f1937400640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f193815b370 0x7f19381f7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.754+0000 7f1937400640 1 -- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193806d180 msgr2=0x7f193813c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:53.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.754+0000 7f1937400640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193806d180 0x7f193813c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.754+0000 7f1937400640 1 -- 172.21.15.133:0/220157009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193814fd40 con 0x7f193815b750 2024-03-07T06:53:53.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.754+0000 7f192fe00640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f193815b370 0x7f19381f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:53.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.754+0000 7f1936a00640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193806d180 0x7f193813c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:53.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.754+0000 7f1937400640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193815b750 0x7f19381e5830 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f193000cc30 tx=0x7f193000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:53.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.754+0000 7f192ea00640 1 -- 172.21.15.133:0/220157009 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193000ce40 con 0x7f193815b750 2024-03-07T06:53:53.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.754+0000 7f192ea00640 1 -- 172.21.15.133:0/220157009 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1930003990 con 0x7f193815b750 2024-03-07T06:53:53.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.754+0000 7f192ea00640 1 -- 172.21.15.133:0/220157009 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1930003d40 con 0x7f193815b750 2024-03-07T06:53:53.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.754+0000 7f193d600640 1 -- 172.21.15.133:0/220157009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1938150020 con 0x7f193815b750 2024-03-07T06:53:53.756 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.755+0000 7f193d600640 1 -- 172.21.15.133:0/220157009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193813be50 con 0x7f193815b750 2024-03-07T06:53:53.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.756+0000 7f193d600640 1 -- 172.21.15.133:0/220157009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1938062c90 con 0x7f193815b750 2024-03-07T06:53:53.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.757+0000 7f192ea00640 1 -- 172.21.15.133:0/220157009 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1930003ee0 con 0x7f193815b750 2024-03-07T06:53:53.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.757+0000 7f192ea00640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f191406b4d0 0x7f191406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:53.757 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.758+0000 7f192ea00640 1 -- 172.21.15.133:0/220157009 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f193008a730 con 0x7f193815b750 2024-03-07T06:53:53.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.758+0000 7f1936a00640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f191406b4d0 0x7f191406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:53.758 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.759+0000 7f1936a00640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f191406b4d0 0x7f191406d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1928036000 tx=0x7f1928008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:53.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.766+0000 7f192ea00640 1 -- 172.21.15.133:0/220157009 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1930014030 con 0x7f193815b750 2024-03-07T06:53:53.960 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.961+0000 7f193d600640 1 -- 172.21.15.133:0/220157009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19381542f0 con 0x7f193815b750 2024-03-07T06:53:53.961 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.961+0000 7f192ea00640 1 -- 172.21.15.133:0/220157009 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1930059510 con 0x7f193815b750 2024-03-07T06:53:53.961 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2024-03-07T06:53:53.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.964+0000 7f193d600640 1 -- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f191406b4d0 msgr2=0x7f191406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:53.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.964+0000 7f193d600640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f191406b4d0 0x7f191406d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1928036000 tx=0x7f1928008030 comp rx=0 tx=0).stop 2024-03-07T06:53:53.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.964+0000 7f193d600640 1 -- 172.21.15.133:0/220157009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193815b750 msgr2=0x7f19381e5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:53.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.964+0000 7f193d600640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193815b750 0x7f19381e5830 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f193000cc30 tx=0x7f193000a5c0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.964+0000 7f193d600640 1 -- 172.21.15.133:0/220157009 shutdown_connections 2024-03-07T06:53:53.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.964+0000 7f193d600640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f193815b750 0x7f19381e5830 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.964+0000 7f193d600640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f193815b370 0x7f19381f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.964+0000 7f193d600640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f191406b4d0 0x7f191406d990 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.964+0000 7f193d600640 1 --2- 172.21.15.133:0/220157009 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f193806d180 0x7f193813c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:53.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.964+0000 7f193d600640 1 -- 172.21.15.133:0/220157009 >> 172.21.15.133:0/220157009 conn(0x7f19380b1d30 msgr2=0x7f1938152b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:53.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.964+0000 7f193d600640 1 -- 172.21.15.133:0/220157009 shutdown_connections 2024-03-07T06:53:53.965 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:53.964+0000 7f193d600640 1 -- 172.21.15.133:0/220157009 wait complete. 2024-03-07T06:53:53.981 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-07T06:53:54.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.184+0000 7f7069800640 1 Processor -- start 2024-03-07T06:53:54.184 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.184+0000 7f7069800640 1 -- start start 2024-03-07T06:53:54.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.186+0000 7f7069800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f706415e700 0x7f7064070e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:54.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.186+0000 7f7069800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70640b2af0 0x7f70640b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:54.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.186+0000 7f7069800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706415ec90 con 0x7f7064144960 2024-03-07T06:53:54.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.186+0000 7f7069800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706415ef70 con 0x7f70640b2af0 2024-03-07T06:53:54.186 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.186+0000 7f7069800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706415f250 con 0x7f706415e700 2024-03-07T06:53:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.186+0000 7f7063400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70640b2af0 0x7f70640b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.186+0000 7f7063400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f70640b2af0 0x7f70640b2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42364/0 (socket says 172.21.15.133:42364) 2024-03-07T06:53:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.186+0000 7f7063400640 1 -- 172.21.15.133:0/1145272772 learned_addr learned my addr 172.21.15.133:0/1145272772 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.187+0000 7f7062000640 1 --2- 172.21.15.133:0/1145272772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f706415e700 0x7f7064070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.187+0000 7f7063400640 1 -- 172.21.15.133:0/1145272772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f706415e700 msgr2=0x7f7064070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.187+0000 7f7063400640 1 --2- 172.21.15.133:0/1145272772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f706415e700 0x7f7064070e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.187+0000 7f7063400640 1 -- 172.21.15.133:0/1145272772 >> v1:172.21.15.161:6789/0 conn(0x7f7064144960 legacy=0x7f706415f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:54.187 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.187+0000 7f7063400640 1 -- 172.21.15.133:0/1145272772 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706415f530 con 0x7f70640b2af0 2024-03-07T06:53:54.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.188+0000 7f7063400640 1 --2- 172.21.15.133:0/1145272772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70640b2af0 0x7f70640b2f70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f705c009e50 tx=0x7f705c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e47e949cd2b1093 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:54.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.188+0000 7f7061600640 1 -- 172.21.15.133:0/1145272772 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c02cb00 con 0x7f70640b2af0 2024-03-07T06:53:54.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.188+0000 7f7061600640 1 -- 172.21.15.133:0/1145272772 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c02cca0 con 0x7f70640b2af0 2024-03-07T06:53:54.188 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.188+0000 7f7061600640 1 -- 172.21.15.133:0/1145272772 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c034610 con 0x7f70640b2af0 2024-03-07T06:53:54.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.189+0000 7f7069800640 1 -- 172.21.15.133:0/1145272772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70640b2af0 msgr2=0x7f70640b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:54.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.189+0000 7f7069800640 1 --2- 172.21.15.133:0/1145272772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70640b2af0 0x7f70640b2f70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f705c009e50 tx=0x7f705c02bf60 comp rx=0 tx=0).stop 2024-03-07T06:53:54.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.189+0000 7f7069800640 1 -- 172.21.15.133:0/1145272772 shutdown_connections 2024-03-07T06:53:54.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.189+0000 7f7069800640 1 --2- 172.21.15.133:0/1145272772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70640b2af0 0x7f70640b2f70 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.189+0000 7f7069800640 1 --2- 172.21.15.133:0/1145272772 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f706415e700 0x7f7064070e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.189+0000 7f7069800640 1 -- 172.21.15.133:0/1145272772 >> 172.21.15.133:0/1145272772 conn(0x7f706406b040 msgr2=0x7f706406b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:54.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.189+0000 7f7069800640 1 -- 172.21.15.133:0/1145272772 shutdown_connections 2024-03-07T06:53:54.189 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.189+0000 7f7069800640 1 -- 172.21.15.133:0/1145272772 wait complete. 2024-03-07T06:53:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.192+0000 7f7069800640 1 Processor -- start 2024-03-07T06:53:54.191 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.192+0000 7f7069800640 1 -- start start 2024-03-07T06:53:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.193+0000 7f7069800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70640b2af0 0x7f70641418a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.193+0000 7f7069800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7064144960 0x7f7064144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.193+0000 7f7069800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f706415e700 0x7f70640b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.193+0000 7f7069800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7064162210 con 0x7f7064144960 2024-03-07T06:53:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.193+0000 7f7069800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706415ff20 con 0x7f70640b2af0 2024-03-07T06:53:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.193+0000 7f7069800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7064162390 con 0x7f706415e700 2024-03-07T06:53:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.193+0000 7f7062000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7064144960 0x7f7064144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.193+0000 7f7062000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7064144960 0x7f7064144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42370/0 (socket says 172.21.15.133:42370) 2024-03-07T06:53:54.193 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.193+0000 7f7062000640 1 -- 172.21.15.133:0/2909039117 learned_addr learned my addr 172.21.15.133:0/2909039117 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.194+0000 7f7063400640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f706415e700 0x7f70640b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.194+0000 7f7062a00640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70640b2af0 0x7f70641418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.194+0000 7f7062000640 1 -- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f706415e700 msgr2=0x7f70640b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.194+0000 7f7062000640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f706415e700 0x7f70640b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.194+0000 7f7062000640 1 -- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70640b2af0 msgr2=0x7f70641418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.194+0000 7f7062000640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70640b2af0 0x7f70641418a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.194+0000 7f7062000640 1 -- 172.21.15.133:0/2909039117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7064145720 con 0x7f7064144960 2024-03-07T06:53:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.194+0000 7f7063400640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f706415e700 0x7f70640b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.194+0000 7f7062a00640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70640b2af0 0x7f70641418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:54.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.194+0000 7f7062000640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7064144960 0x7f7064144d40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f705000d5d0 tx=0x7f705000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.194+0000 7f705aa00640 1 -- 172.21.15.133:0/2909039117 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050004240 con 0x7f7064144960 2024-03-07T06:53:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.195+0000 7f705aa00640 1 -- 172.21.15.133:0/2909039117 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70500043e0 con 0x7f7064144960 2024-03-07T06:53:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.195+0000 7f7069800640 1 -- 172.21.15.133:0/2909039117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7064145a00 con 0x7f7064144960 2024-03-07T06:53:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.195+0000 7f705aa00640 1 -- 172.21.15.133:0/2909039117 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050003830 con 0x7f7064144960 2024-03-07T06:53:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.195+0000 7f7069800640 1 -- 172.21.15.133:0/2909039117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7064145fc0 con 0x7f7064144960 2024-03-07T06:53:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.196+0000 7f7069800640 1 -- 172.21.15.133:0/2909039117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7064062c90 con 0x7f7064144960 2024-03-07T06:53:54.196 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.197+0000 7f705aa00640 1 -- 172.21.15.133:0/2909039117 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f70500039d0 con 0x7f7064144960 2024-03-07T06:53:54.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.198+0000 7f705aa00640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f704406b500 0x7f704406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:54.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.198+0000 7f705aa00640 1 -- 172.21.15.133:0/2909039117 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f705008a4b0 con 0x7f7064144960 2024-03-07T06:53:54.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.199+0000 7f7062a00640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f704406b500 0x7f704406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:54.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.200+0000 7f7062a00640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f704406b500 0x7f704406d9c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7054003eb0 tx=0x7f7054009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:54.202 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.202+0000 7f705aa00640 1 -- 172.21.15.133:0/2909039117 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f705001c360 con 0x7f7064144960 2024-03-07T06:53:54.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.516+0000 7f7069800640 1 -- 172.21.15.133:0/2909039117 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f70640b2700 con 0x7f7064144960 2024-03-07T06:53:54.516 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.517+0000 7f705aa00640 1 -- 172.21.15.133:0/2909039117 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7050059320 con 0x7f7064144960 2024-03-07T06:53:54.517 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2024-03-07T06:53:54.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.520+0000 7f7069800640 1 -- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f704406b500 msgr2=0x7f704406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:54.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.520+0000 7f7069800640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f704406b500 0x7f704406d9c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7054003eb0 tx=0x7f7054009040 comp rx=0 tx=0).stop 2024-03-07T06:53:54.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.520+0000 7f7069800640 1 -- 172.21.15.133:0/2909039117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7064144960 msgr2=0x7f7064144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:54.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.520+0000 7f7069800640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7064144960 0x7f7064144d40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f705000d5d0 tx=0x7f705000da90 comp rx=0 tx=0).stop 2024-03-07T06:53:54.520 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.520+0000 7f7069800640 1 -- 172.21.15.133:0/2909039117 shutdown_connections 2024-03-07T06:53:54.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.520+0000 7f7069800640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f706415e700 0x7f70640b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.520+0000 7f7069800640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7064144960 0x7f7064144d40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.520+0000 7f7069800640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f704406b500 0x7f704406d9c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.520+0000 7f7069800640 1 --2- 172.21.15.133:0/2909039117 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f70640b2af0 0x7f70641418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.520+0000 7f7069800640 1 -- 172.21.15.133:0/2909039117 >> 172.21.15.133:0/2909039117 conn(0x7f706406b040 msgr2=0x7f706415f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:54.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.521+0000 7f7069800640 1 -- 172.21.15.133:0/2909039117 shutdown_connections 2024-03-07T06:53:54.521 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.521+0000 7f7069800640 1 -- 172.21.15.133:0/2909039117 wait complete. 2024-03-07T06:53:54.546 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-07T06:53:54.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.747+0000 7f146e200640 1 Processor -- start 2024-03-07T06:53:54.747 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.747+0000 7f146e200640 1 -- start start 2024-03-07T06:53:54.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.750+0000 7f146e200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f146815c050 0x7f1468160570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:54.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.750+0000 7f146e200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14680b5b50 0x7f14680b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:54.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.750+0000 7f146e200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f146806daf0 0x7f146806df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:54.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.750+0000 7f146e200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14680b64d0 con 0x7f146815c050 2024-03-07T06:53:54.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.750+0000 7f146e200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146815c430 con 0x7f146806daf0 2024-03-07T06:53:54.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.750+0000 7f146e200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1468160ab0 con 0x7f14680b5b50 2024-03-07T06:53:54.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.750+0000 7f1467400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f146815c050 0x7f1468160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:54.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.751+0000 7f1467400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f146815c050 0x7f1468160570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42384/0 (socket says 172.21.15.133:42384) 2024-03-07T06:53:54.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.751+0000 7f1467400640 1 -- 172.21.15.133:0/1537814355 learned_addr learned my addr 172.21.15.133:0/1537814355 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:54.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.751+0000 7f1467e00640 1 --2- 172.21.15.133:0/1537814355 >> v2:172.21.15.161:3300/0 conn(0x7f146806daf0 0x7f146806df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:54.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.751+0000 7f1467400640 1 -- 172.21.15.133:0/1537814355 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14680b5b50 msgr2=0x7f14680b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:54.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.751+0000 7f1467400640 1 --2- 172.21.15.133:0/1537814355 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14680b5b50 0x7f14680b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.751+0000 7f1467400640 1 -- 172.21.15.133:0/1537814355 >> v2:172.21.15.161:3300/0 conn(0x7f146806daf0 msgr2=0x7f146806df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:54.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.751+0000 7f1467400640 1 --2- 172.21.15.133:0/1537814355 >> v2:172.21.15.161:3300/0 conn(0x7f146806daf0 0x7f146806df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.751+0000 7f1467400640 1 -- 172.21.15.133:0/1537814355 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14680cea10 con 0x7f146815c050 2024-03-07T06:53:54.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.752+0000 7f1467400640 1 --2- 172.21.15.133:0/1537814355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f146815c050 0x7f1468160570 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1468067030 tx=0x7f145802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7082afe265c97aa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:54.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.752+0000 7f1466a00640 1 -- 172.21.15.133:0/1537814355 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145802c950 con 0x7f146815c050 2024-03-07T06:53:54.752 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.753+0000 7f1466a00640 1 -- 172.21.15.133:0/1537814355 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145802caf0 con 0x7f146815c050 2024-03-07T06:53:54.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.753+0000 7f1466a00640 1 -- 172.21.15.133:0/1537814355 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14580343f0 con 0x7f146815c050 2024-03-07T06:53:54.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.753+0000 7f146e200640 1 -- 172.21.15.133:0/1537814355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f146815c050 msgr2=0x7f1468160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:54.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.753+0000 7f146e200640 1 --2- 172.21.15.133:0/1537814355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f146815c050 0x7f1468160570 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1468067030 tx=0x7f145802bdb0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.753 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.753+0000 7f146e200640 1 -- 172.21.15.133:0/1537814355 shutdown_connections 2024-03-07T06:53:54.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.753+0000 7f146e200640 1 --2- 172.21.15.133:0/1537814355 >> v2:172.21.15.161:3300/0 conn(0x7f146806daf0 0x7f146806df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.753+0000 7f146e200640 1 --2- 172.21.15.133:0/1537814355 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14680b5b50 0x7f14680b5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.753+0000 7f146e200640 1 --2- 172.21.15.133:0/1537814355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f146815c050 0x7f1468160570 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.753+0000 7f146e200640 1 -- 172.21.15.133:0/1537814355 >> 172.21.15.133:0/1537814355 conn(0x7f146806b370 msgr2=0x7f146806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:54.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.753+0000 7f146e200640 1 -- 172.21.15.133:0/1537814355 shutdown_connections 2024-03-07T06:53:54.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.753+0000 7f146e200640 1 -- 172.21.15.133:0/1537814355 wait complete. 2024-03-07T06:53:54.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.755+0000 7f146e200640 1 Processor -- start 2024-03-07T06:53:54.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.755+0000 7f146e200640 1 -- start start 2024-03-07T06:53:54.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.756+0000 7f146e200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f146806daf0 0x7f1468150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:54.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.756+0000 7f146e200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14680b5b50 0x7f146815ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:54.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.756+0000 7f146e200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f146815c050 0x7f14680c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:54.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.756+0000 7f146e200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14680cecf0 con 0x7f146815c050 2024-03-07T06:53:54.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.756+0000 7f146e200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146806f120 con 0x7f146806daf0 2024-03-07T06:53:54.759 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.756+0000 7f146e200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14680aff70 con 0x7f14680b5b50 2024-03-07T06:53:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.757+0000 7f1467e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f146815c050 0x7f14680c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.757+0000 7f1467e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f146815c050 0x7f14680c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42390/0 (socket says 172.21.15.133:42390) 2024-03-07T06:53:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.757+0000 7f1467e00640 1 -- 172.21.15.133:0/1877466925 learned_addr learned my addr 172.21.15.133:0/1877466925 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.757+0000 7f1467400640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f146806daf0 0x7f1468150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.757+0000 7f145ea00640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14680b5b50 0x7f146815ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.757+0000 7f1467e00640 1 -- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14680b5b50 msgr2=0x7f146815ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.757+0000 7f1467e00640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14680b5b50 0x7f146815ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.757+0000 7f1467e00640 1 -- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f146806daf0 msgr2=0x7f1468150250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.757+0000 7f1467e00640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f146806daf0 0x7f1468150250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.757+0000 7f1467e00640 1 -- 172.21.15.133:0/1877466925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146815d1a0 con 0x7f146815c050 2024-03-07T06:53:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.757+0000 7f1467400640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f146806daf0 0x7f1468150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:54.760 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.757+0000 7f145ea00640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14680b5b50 0x7f146815ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.757+0000 7f1467e00640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f146815c050 0x7f14680c7990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f146000c580 tx=0x7f146000ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.758+0000 7f145fe00640 1 -- 172.21.15.133:0/1877466925 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460015070 con 0x7f146815c050 2024-03-07T06:53:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.758+0000 7f145fe00640 1 -- 172.21.15.133:0/1877466925 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1460004220 con 0x7f146815c050 2024-03-07T06:53:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.758+0000 7f145fe00640 1 -- 172.21.15.133:0/1877466925 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460003610 con 0x7f146815c050 2024-03-07T06:53:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.758+0000 7f146e200640 1 -- 172.21.15.133:0/1877466925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146815d460 con 0x7f146815c050 2024-03-07T06:53:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.758+0000 7f146e200640 1 -- 172.21.15.133:0/1877466925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146815da20 con 0x7f146815c050 2024-03-07T06:53:54.761 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.759+0000 7f146e200640 1 -- 172.21.15.133:0/1877466925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1468066d90 con 0x7f146815c050 2024-03-07T06:53:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.760+0000 7f145fe00640 1 -- 172.21.15.133:0/1877466925 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f14600037b0 con 0x7f146815c050 2024-03-07T06:53:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.761+0000 7f145fe00640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f144406b4d0 0x7f144406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.761+0000 7f145fe00640 1 -- 172.21.15.133:0/1877466925 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1460089fd0 con 0x7f146815c050 2024-03-07T06:53:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.761+0000 7f1467400640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f144406b4d0 0x7f144406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:54.762 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.762+0000 7f1467400640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f144406b4d0 0x7f144406d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1458035000 tx=0x7f1458030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:54.769 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.770+0000 7f145fe00640 1 -- 172.21.15.133:0/1877466925 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1468066d90 con 0x7f146815c050 2024-03-07T06:53:54.963 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.964+0000 7f146e200640 1 -- 172.21.15.133:0/1877466925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1468062c90 con 0x7f146815c050 2024-03-07T06:53:54.964 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.965+0000 7f145fe00640 1 -- 172.21.15.133:0/1877466925 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1460058dc0 con 0x7f146815c050 2024-03-07T06:53:54.965 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2024-03-07T06:53:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.967+0000 7f146e200640 1 -- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f144406b4d0 msgr2=0x7f144406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.967+0000 7f146e200640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f144406b4d0 0x7f144406d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1458035000 tx=0x7f1458030000 comp rx=0 tx=0).stop 2024-03-07T06:53:54.967 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.967+0000 7f146e200640 1 -- 172.21.15.133:0/1877466925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f146815c050 msgr2=0x7f14680c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:54.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.967+0000 7f146e200640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f146815c050 0x7f14680c7990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f146000c580 tx=0x7f146000ca40 comp rx=0 tx=0).stop 2024-03-07T06:53:54.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.967+0000 7f146e200640 1 -- 172.21.15.133:0/1877466925 shutdown_connections 2024-03-07T06:53:54.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.967+0000 7f146e200640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f146815c050 0x7f14680c7990 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.967+0000 7f146e200640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14680b5b50 0x7f146815ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.967+0000 7f146e200640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f144406b4d0 0x7f144406d990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.967+0000 7f146e200640 1 --2- 172.21.15.133:0/1877466925 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f146806daf0 0x7f1468150250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:54.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.967+0000 7f146e200640 1 -- 172.21.15.133:0/1877466925 >> 172.21.15.133:0/1877466925 conn(0x7f146806b370 msgr2=0x7f146806b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:54.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.968+0000 7f146e200640 1 -- 172.21.15.133:0/1877466925 shutdown_connections 2024-03-07T06:53:54.968 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:54.968+0000 7f146e200640 1 -- 172.21.15.133:0/1877466925 wait complete. 2024-03-07T06:53:54.991 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:53:55.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.195+0000 7f07d6a00640 1 Processor -- start 2024-03-07T06:53:55.195 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.195+0000 7f07d6a00640 1 -- start start 2024-03-07T06:53:55.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.196+0000 7f07d6a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f07d015b1b0 0x7f07d00b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:55.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.196+0000 7f07d6a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f07d00af950 0x7f07d00cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:55.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.196+0000 7f07d6a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07d00ce420 0x7f07d006de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:55.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.196+0000 7f07d6a00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d0071610 con 0x7f07d015b1b0 2024-03-07T06:53:55.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.196+0000 7f07d6a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d00ce830 con 0x7f07d00af950 2024-03-07T06:53:55.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.196+0000 7f07d6a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d006f140 con 0x7f07d00ce420 2024-03-07T06:53:55.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.197+0000 7f07cfe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f07d015b1b0 0x7f07d00b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:55.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.197+0000 7f07cfe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f07d015b1b0 0x7f07d00b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42398/0 (socket says 172.21.15.133:42398) 2024-03-07T06:53:55.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.197+0000 7f07cfe00640 1 -- 172.21.15.133:0/2548776286 learned_addr learned my addr 172.21.15.133:0/2548776286 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:55.197 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.197+0000 7f07d4a00640 1 --2- 172.21.15.133:0/2548776286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07d00ce420 0x7f07d006de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:55.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.197+0000 7f07c7e00640 1 --2- 172.21.15.133:0/2548776286 >> v2:172.21.15.161:3300/0 conn(0x7f07d00af950 0x7f07d00cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:55.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.197+0000 7f07cfe00640 1 -- 172.21.15.133:0/2548776286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07d00ce420 msgr2=0x7f07d006de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:55.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.197+0000 7f07cfe00640 1 --2- 172.21.15.133:0/2548776286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07d00ce420 0x7f07d006de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.197+0000 7f07cfe00640 1 -- 172.21.15.133:0/2548776286 >> v2:172.21.15.161:3300/0 conn(0x7f07d00af950 msgr2=0x7f07d00cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:55.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.197+0000 7f07cfe00640 1 --2- 172.21.15.133:0/2548776286 >> v2:172.21.15.161:3300/0 conn(0x7f07d00af950 0x7f07d00cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.197+0000 7f07cfe00640 1 -- 172.21.15.133:0/2548776286 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d006f420 con 0x7f07d015b1b0 2024-03-07T06:53:55.198 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.197+0000 7f07c7e00640 1 --2- 172.21.15.133:0/2548776286 >> v2:172.21.15.161:3300/0 conn(0x7f07d00af950 0x7f07d00cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:55.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.198+0000 7f07cfe00640 1 --2- 172.21.15.133:0/2548776286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d015b1b0 0x7f07d00b3c10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f07d00ceac0 tx=0x7f07c002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870fb9db96a96208 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:55.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.199+0000 7f07cf400640 1 -- 172.21.15.133:0/2548776286 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c002e070 con 0x7f07d015b1b0 2024-03-07T06:53:55.199 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.199+0000 7f07cf400640 1 -- 172.21.15.133:0/2548776286 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c0037070 con 0x7f07d015b1b0 2024-03-07T06:53:55.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.199+0000 7f07cf400640 1 -- 172.21.15.133:0/2548776286 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c0031a00 con 0x7f07d015b1b0 2024-03-07T06:53:55.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.199+0000 7f07d6a00640 1 -- 172.21.15.133:0/2548776286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d015b1b0 msgr2=0x7f07d00b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:55.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.199+0000 7f07d6a00640 1 --2- 172.21.15.133:0/2548776286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d015b1b0 0x7f07d00b3c10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f07d00ceac0 tx=0x7f07c002b880 comp rx=0 tx=0).stop 2024-03-07T06:53:55.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.199+0000 7f07d6a00640 1 -- 172.21.15.133:0/2548776286 shutdown_connections 2024-03-07T06:53:55.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.199+0000 7f07d6a00640 1 --2- 172.21.15.133:0/2548776286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07d00ce420 0x7f07d006de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.199+0000 7f07d6a00640 1 --2- 172.21.15.133:0/2548776286 >> v2:172.21.15.161:3300/0 conn(0x7f07d00af950 0x7f07d00cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.199+0000 7f07d6a00640 1 --2- 172.21.15.133:0/2548776286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d015b1b0 0x7f07d00b3c10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.199+0000 7f07d6a00640 1 -- 172.21.15.133:0/2548776286 >> 172.21.15.133:0/2548776286 conn(0x7f07d00b3120 msgr2=0x7f07d00b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:55.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.199+0000 7f07d6a00640 1 -- 172.21.15.133:0/2548776286 shutdown_connections 2024-03-07T06:53:55.200 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.199+0000 7f07d6a00640 1 -- 172.21.15.133:0/2548776286 wait complete. 2024-03-07T06:53:55.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.201+0000 7f07d6a00640 1 Processor -- start 2024-03-07T06:53:55.201 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.201+0000 7f07d6a00640 1 -- start start 2024-03-07T06:53:55.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.202+0000 7f07d6a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d00af950 0x7f07d015b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:55.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.202+0000 7f07d6a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d00ce420 0x7f07d01523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:55.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.202+0000 7f07d6a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07d015b1b0 0x7f07d0159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:55.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.202+0000 7f07d6a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d015b590 con 0x7f07d00ce420 2024-03-07T06:53:55.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.202+0000 7f07d6a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d00cc880 con 0x7f07d00af950 2024-03-07T06:53:55.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.202+0000 7f07d6a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d015b710 con 0x7f07d015b1b0 2024-03-07T06:53:55.203 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.202+0000 7f07c7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d00ce420 0x7f07d01523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.202+0000 7f07c7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d00ce420 0x7f07d01523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42402/0 (socket says 172.21.15.133:42402) 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.202+0000 7f07c7e00640 1 -- 172.21.15.133:0/3214467130 learned_addr learned my addr 172.21.15.133:0/3214467130 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.203+0000 7f07cfe00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d00af950 0x7f07d015b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.203+0000 7f07d4a00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07d015b1b0 0x7f07d0159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.203+0000 7f07c7e00640 1 -- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07d015b1b0 msgr2=0x7f07d0159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.203+0000 7f07c7e00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07d015b1b0 0x7f07d0159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.203+0000 7f07c7e00640 1 -- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d00af950 msgr2=0x7f07d015b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.203+0000 7f07c7e00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d00af950 0x7f07d015b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.203+0000 7f07c7e00640 1 -- 172.21.15.133:0/3214467130 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d015bfa0 con 0x7f07d00ce420 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.203+0000 7f07cfe00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d00af950 0x7f07d015b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.203+0000 7f07d4a00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07d015b1b0 0x7f07d0159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.203+0000 7f07c7e00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d00ce420 0x7f07d01523e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f07b800c7c0 tx=0x7f07b800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.203+0000 7f07ccc00640 1 -- 172.21.15.133:0/3214467130 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8015070 con 0x7f07d00ce420 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.203+0000 7f07ccc00640 1 -- 172.21.15.133:0/3214467130 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b8007a30 con 0x7f07d00ce420 2024-03-07T06:53:55.204 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.203+0000 7f07ccc00640 1 -- 172.21.15.133:0/3214467130 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b8007e00 con 0x7f07d00ce420 2024-03-07T06:53:55.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.204+0000 7f07d6a00640 1 -- 172.21.15.133:0/3214467130 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d015c280 con 0x7f07d00ce420 2024-03-07T06:53:55.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.204+0000 7f07d6a00640 1 -- 172.21.15.133:0/3214467130 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d015c840 con 0x7f07d00ce420 2024-03-07T06:53:55.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.205+0000 7f07d6a00640 1 -- 172.21.15.133:0/3214467130 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d0062c90 con 0x7f07d00ce420 2024-03-07T06:53:55.205 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.206+0000 7f07ccc00640 1 -- 172.21.15.133:0/3214467130 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f07b8018070 con 0x7f07d00ce420 2024-03-07T06:53:55.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.207+0000 7f07ccc00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f07ac06b4d0 0x7f07ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:55.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.207+0000 7f07ccc00640 1 -- 172.21.15.133:0/3214467130 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07b808a210 con 0x7f07d00ce420 2024-03-07T06:53:55.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.207+0000 7f07cfe00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f07ac06b4d0 0x7f07ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:55.208 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.209+0000 7f07cfe00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f07ac06b4d0 0x7f07ac06d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f07c0036000 tx=0x7f07c0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:55.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.211+0000 7f07ccc00640 1 -- 172.21.15.133:0/3214467130 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f07b8058ff0 con 0x7f07d00ce420 2024-03-07T06:53:55.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.448+0000 7f07d6a00640 1 -- 172.21.15.133:0/3214467130 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07d00cc660 con 0x7f07d00ce420 2024-03-07T06:53:55.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.449+0000 7f07ccc00640 1 -- 172.21.15.133:0/3214467130 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4980 (secure 0 0 0) 0x7f07b80588f0 con 0x7f07d00ce420 2024-03-07T06:53:55.448 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:55.449 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/44975892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":44975892},{"type":"v1","addr":"172.21.15.161:6835","nonce":44975892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4972,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/906115542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":906115542},{"type":"v1","addr":"172.21.15.133:6833","nonce":906115542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1998424684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1998424684},{"type":"v1","addr":"172.21.15.161:6837","nonce":1998424684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:53:24.627534+0000","modified":"2024-03-07T06:53:27.698471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/2951618124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2951618124},{"type":"v1","addr":"172.21.15.133:6835","nonce":2951618124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":3}]} 2024-03-07T06:53:55.449 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 27 2024-03-07T06:53:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.452+0000 7f07d6a00640 1 -- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f07ac06b4d0 msgr2=0x7f07ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.452+0000 7f07d6a00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f07ac06b4d0 0x7f07ac06d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f07c0036000 tx=0x7f07c0008030 comp rx=0 tx=0).stop 2024-03-07T06:53:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.452+0000 7f07d6a00640 1 -- 172.21.15.133:0/3214467130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d00ce420 msgr2=0x7f07d01523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.452+0000 7f07d6a00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d00ce420 0x7f07d01523e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f07b800c7c0 tx=0x7f07b800cc80 comp rx=0 tx=0).stop 2024-03-07T06:53:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.452+0000 7f07d6a00640 1 -- 172.21.15.133:0/3214467130 shutdown_connections 2024-03-07T06:53:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.452+0000 7f07d6a00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07d015b1b0 0x7f07d0159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.452+0000 7f07d6a00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07d00ce420 0x7f07d01523e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.452+0000 7f07d6a00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f07ac06b4d0 0x7f07ac06d990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.452+0000 7f07d6a00640 1 --2- 172.21.15.133:0/3214467130 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f07d00af950 0x7f07d015b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.452+0000 7f07d6a00640 1 -- 172.21.15.133:0/3214467130 >> 172.21.15.133:0/3214467130 conn(0x7f07d00b3120 msgr2=0x7f07d00aed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.452+0000 7f07d6a00640 1 -- 172.21.15.133:0/3214467130 shutdown_connections 2024-03-07T06:53:55.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.452+0000 7f07d6a00640 1 -- 172.21.15.133:0/3214467130 wait complete. 2024-03-07T06:53:55.478 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:53:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.672+0000 7f64b6400640 1 Processor -- start 2024-03-07T06:53:55.672 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.672+0000 7f64b6400640 1 -- start start 2024-03-07T06:53:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.674+0000 7f64b6400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f64b015bd30 0x7f64b006e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.674+0000 7f64b6400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f64b0072f40 0x7f64b015c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.674+0000 7f64b6400640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64b0160760 con 0x7f64b00cf5d0 2024-03-07T06:53:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.674+0000 7f64b6400640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b015f9a0 con 0x7f64b0072f40 2024-03-07T06:53:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.674+0000 7f64b6400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64b015fc80 con 0x7f64b015bd30 2024-03-07T06:53:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.674+0000 7f64b5a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f64b0072f40 0x7f64b015c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.674+0000 7f64b5a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f64b0072f40 0x7f64b015c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42404/0 (socket says 172.21.15.133:42404) 2024-03-07T06:53:55.674 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.674+0000 7f64b5a00640 1 -- 172.21.15.133:0/2825889050 learned_addr learned my addr 172.21.15.133:0/2825889050 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.675+0000 7f64b5a00640 1 -- 172.21.15.133:0/2825889050 >> v2:172.21.15.161:3300/0 conn(0x7f64b015bd30 msgr2=0x7f64b006e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.675+0000 7f64b5000640 1 --2- 172.21.15.133:0/2825889050 >> v2:172.21.15.161:3300/0 conn(0x7f64b015bd30 0x7f64b006e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.675+0000 7f64b5a00640 1 --2- 172.21.15.133:0/2825889050 >> v2:172.21.15.161:3300/0 conn(0x7f64b015bd30 0x7f64b006e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.675+0000 7f64b5a00640 1 -- 172.21.15.133:0/2825889050 >> v1:172.21.15.133:6789/0 conn(0x7f64b00cf5d0 legacy=0x7f64b0072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.675+0000 7f64b5a00640 1 -- 172.21.15.133:0/2825889050 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b015ff60 con 0x7f64b0072f40 2024-03-07T06:53:55.675 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.675+0000 7f64b5000640 1 --2- 172.21.15.133:0/2825889050 >> v2:172.21.15.161:3300/0 conn(0x7f64b015bd30 0x7f64b006e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:55.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.676+0000 7f64b5a00640 1 --2- 172.21.15.133:0/2825889050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b0072f40 0x7f64b015c110 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f64b00685a0 tx=0x7f64a802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162610ece1aa83e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:55.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.676+0000 7f64af400640 1 -- 172.21.15.133:0/2825889050 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a802cb20 con 0x7f64b0072f40 2024-03-07T06:53:55.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.676+0000 7f64af400640 1 -- 172.21.15.133:0/2825889050 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a802ccc0 con 0x7f64b0072f40 2024-03-07T06:53:55.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.676+0000 7f64af400640 1 -- 172.21.15.133:0/2825889050 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8034610 con 0x7f64b0072f40 2024-03-07T06:53:55.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.677+0000 7f64b6400640 1 -- 172.21.15.133:0/2825889050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b0072f40 msgr2=0x7f64b015c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:55.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.677+0000 7f64b6400640 1 --2- 172.21.15.133:0/2825889050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b0072f40 0x7f64b015c110 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f64b00685a0 tx=0x7f64a802bf80 comp rx=0 tx=0).stop 2024-03-07T06:53:55.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.677+0000 7f64b6400640 1 -- 172.21.15.133:0/2825889050 shutdown_connections 2024-03-07T06:53:55.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.677+0000 7f64b6400640 1 --2- 172.21.15.133:0/2825889050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b0072f40 0x7f64b015c110 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.677+0000 7f64b6400640 1 --2- 172.21.15.133:0/2825889050 >> v2:172.21.15.161:3300/0 conn(0x7f64b015bd30 0x7f64b006e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.677+0000 7f64b6400640 1 -- 172.21.15.133:0/2825889050 >> 172.21.15.133:0/2825889050 conn(0x7f64b00b2e90 msgr2=0x7f64b00b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:55.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.677+0000 7f64b6400640 1 -- 172.21.15.133:0/2825889050 shutdown_connections 2024-03-07T06:53:55.677 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.677+0000 7f64b6400640 1 -- 172.21.15.133:0/2825889050 wait complete. 2024-03-07T06:53:55.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.679+0000 7f64b6400640 1 Processor -- start 2024-03-07T06:53:55.678 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.679+0000 7f64b6400640 1 -- start start 2024-03-07T06:53:55.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.680+0000 7f64b6400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64b0072f40 0x7f64b015ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.680+0000 7f64b6400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64b00cf5d0 0x7f64b0149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.680+0000 7f64b6400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b015bd30 0x7f64b014a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.680+0000 7f64b6400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b006f130 con 0x7f64b015bd30 2024-03-07T06:53:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.680+0000 7f64b6400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b006ee20 con 0x7f64b0072f40 2024-03-07T06:53:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.680+0000 7f64b6400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b015c650 con 0x7f64b00cf5d0 2024-03-07T06:53:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.680+0000 7f64b5a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b015bd30 0x7f64b014a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.680+0000 7f64b5a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b015bd30 0x7f64b014a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42414/0 (socket says 172.21.15.133:42414) 2024-03-07T06:53:55.680 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.680+0000 7f64b5a00640 1 -- 172.21.15.133:0/3867634839 learned_addr learned my addr 172.21.15.133:0/3867634839 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:55.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.680+0000 7f64b5000640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64b0072f40 0x7f64b015ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:55.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.681+0000 7f64afe00640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64b00cf5d0 0x7f64b0149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:55.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.681+0000 7f64b5a00640 1 -- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64b00cf5d0 msgr2=0x7f64b0149300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:55.681 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.681+0000 7f64b5a00640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64b00cf5d0 0x7f64b0149300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.681+0000 7f64b5a00640 1 -- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64b0072f40 msgr2=0x7f64b015ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:55.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.681+0000 7f64b5a00640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64b0072f40 0x7f64b015ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.681+0000 7f64b5a00640 1 -- 172.21.15.133:0/3867634839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b014a970 con 0x7f64b015bd30 2024-03-07T06:53:55.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.681+0000 7f64b5000640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64b0072f40 0x7f64b015ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:55.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.681+0000 7f64b5a00640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b015bd30 0x7f64b014a430 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009f40 tx=0x7f64a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:55.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.681+0000 7f64acc00640 1 -- 172.21.15.133:0/3867634839 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8043020 con 0x7f64b015bd30 2024-03-07T06:53:55.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.681+0000 7f64acc00640 1 -- 172.21.15.133:0/3867634839 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a803e450 con 0x7f64b015bd30 2024-03-07T06:53:55.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.681+0000 7f64acc00640 1 -- 172.21.15.133:0/3867634839 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8009510 con 0x7f64b015bd30 2024-03-07T06:53:55.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.681+0000 7f64b6400640 1 -- 172.21.15.133:0/3867634839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b014ac10 con 0x7f64b015bd30 2024-03-07T06:53:55.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.682+0000 7f64b6400640 1 -- 172.21.15.133:0/3867634839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b014b1d0 con 0x7f64b015bd30 2024-03-07T06:53:55.682 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.683+0000 7f64b6400640 1 -- 172.21.15.133:0/3867634839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64b0074810 con 0x7f64b015bd30 2024-03-07T06:53:55.683 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.683+0000 7f64acc00640 1 -- 172.21.15.133:0/3867634839 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f64a8046020 con 0x7f64b015bd30 2024-03-07T06:53:55.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.685+0000 7f64acc00640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f649006b4d0 0x7f649006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:55.684 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.685+0000 7f64acc00640 1 -- 172.21.15.133:0/3867634839 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64a80acbc0 con 0x7f64b015bd30 2024-03-07T06:53:55.685 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.686+0000 7f64b5000640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f649006b4d0 0x7f649006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:55.686 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.687+0000 7f64b5000640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f649006b4d0 0x7f649006d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f64a4006fd0 tx=0x7f64a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:55.688 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.690+0000 7f64acc00640 1 -- 172.21.15.133:0/3867634839 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f64b0074810 con 0x7f64b015bd30 2024-03-07T06:53:55.910 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.910+0000 7f64b6400640 1 -- 172.21.15.133:0/3867634839 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64b0062c90 con 0x7f64b015bd30 2024-03-07T06:53:55.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.912+0000 7f64acc00640 1 -- 172.21.15.133:0/3867634839 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16661 (secure 0 0 0) 0x7f64a804d030 con 0x7f64b015bd30 2024-03-07T06:53:55.913 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:55.913 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":46,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:53:26.170797+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:53:22.398527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-07T06:53:23.628483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:53:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.915+0000 7f64b6400640 1 -- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f649006b4d0 msgr2=0x7f649006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.915+0000 7f64b6400640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f649006b4d0 0x7f649006d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f64a4006fd0 tx=0x7f64a4008040 comp rx=0 tx=0).stop 2024-03-07T06:53:55.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.915+0000 7f64b6400640 1 -- 172.21.15.133:0/3867634839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b015bd30 msgr2=0x7f64b014a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.915+0000 7f64b6400640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b015bd30 0x7f64b014a430 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f64a8009f40 tx=0x7f64a8004a40 comp rx=0 tx=0).stop 2024-03-07T06:53:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.915+0000 7f64b6400640 1 -- 172.21.15.133:0/3867634839 shutdown_connections 2024-03-07T06:53:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.915+0000 7f64b6400640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64b015bd30 0x7f64b014a430 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.915+0000 7f64b6400640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64b00cf5d0 0x7f64b0149300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.915+0000 7f64b6400640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f649006b4d0 0x7f649006d990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.916 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.915+0000 7f64b6400640 1 --2- 172.21.15.133:0/3867634839 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64b0072f40 0x7f64b015ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:55.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.915+0000 7f64b6400640 1 -- 172.21.15.133:0/3867634839 >> 172.21.15.133:0/3867634839 conn(0x7f64b00b2e90 msgr2=0x7f64b0071820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:55.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.915+0000 7f64b6400640 1 -- 172.21.15.133:0/3867634839 shutdown_connections 2024-03-07T06:53:55.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:55.915+0000 7f64b6400640 1 -- 172.21.15.133:0/3867634839 wait complete. 2024-03-07T06:53:55.934 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-07T06:53:55.934 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:53:56.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.130+0000 7fd69b200640 1 Processor -- start 2024-03-07T06:53:56.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.130+0000 7fd69b200640 1 -- start start 2024-03-07T06:53:56.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.133+0000 7fd69b200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd69c140cb0 0x7fd69c162290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.133+0000 7fd69b200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69c143810 con 0x7fd69c0d2b00 2024-03-07T06:53:56.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.133+0000 7fd69b200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69c143af0 con 0x7fd69c0b70d0 2024-03-07T06:53:56.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.133+0000 7fd69b200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69c1435c0 con 0x7fd69c140cb0 2024-03-07T06:53:56.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.133+0000 7fd69a800640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fd69c0d2b00 0x7fd69c0b9a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35056/0 (socket says 172.21.15.133:35056) 2024-03-07T06:53:56.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.133+0000 7fd69a800640 1 -- 172.21.15.133:0/3739154494 learned_addr learned my addr 172.21.15.133:0/3739154494 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:56.133 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.133+0000 7fd699e00640 1 --2- 172.21.15.133:0/3739154494 >> v2:172.21.15.161:3300/0 conn(0x7fd69c140cb0 0x7fd69c162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 -- 172.21.15.133:0/3739154494 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409928722 0 0) 0x7fd69c143810 con 0x7fd69c0d2b00 2024-03-07T06:53:56.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 -- 172.21.15.133:0/3739154494 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd680003520 con 0x7fd69c0d2b00 2024-03-07T06:53:56.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 -- 172.21.15.133:0/3739154494 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 709530143 0 0) 0x7fd680003520 con 0x7fd69c0d2b00 2024-03-07T06:53:56.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 -- 172.21.15.133:0/3739154494 >> v2:172.21.15.161:3300/0 conn(0x7fd69c140cb0 msgr2=0x7fd69c162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:56.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 --2- 172.21.15.133:0/3739154494 >> v2:172.21.15.161:3300/0 conn(0x7fd69c140cb0 0x7fd69c162290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 -- 172.21.15.133:0/3739154494 >> v1:172.21.15.161:6789/0 conn(0x7fd69c0b70d0 legacy=0x7fd69c0b9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:56.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 -- 172.21.15.133:0/3739154494 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c143dd0 con 0x7fd69c0d2b00 2024-03-07T06:53:56.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 -- 172.21.15.133:0/3739154494 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fd6940019f0 con 0x7fd69c0d2b00 2024-03-07T06:53:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 -- 172.21.15.133:0/3739154494 >> v1:172.21.15.133:6789/0 conn(0x7fd69c0d2b00 legacy=0x7fd69c0b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 --2- 172.21.15.133:0/3739154494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6800039b0 0x7fd680003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 --2- 172.21.15.133:0/3739154494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd680004800 0x7fd680004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 --2- 172.21.15.133:0/3739154494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6800051c0 0x7fd68000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 -- 172.21.15.133:0/3739154494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c1435c0 con 0x7fd6800039b0 2024-03-07T06:53:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 -- 172.21.15.133:0/3739154494 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68000c160 con 0x7fd680004800 2024-03-07T06:53:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd698a00640 1 -- 172.21.15.133:0/3739154494 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68000c360 con 0x7fd6800051c0 2024-03-07T06:53:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.134+0000 7fd699e00640 1 --2- 172.21.15.133:0/3739154494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6800039b0 0x7fd680003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.135+0000 7fd699e00640 1 -- 172.21.15.133:0/3739154494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6800051c0 msgr2=0x7fd68000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.135+0000 7fd699e00640 1 --2- 172.21.15.133:0/3739154494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd6800051c0 0x7fd68000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.135+0000 7fd699e00640 1 -- 172.21.15.133:0/3739154494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd680004800 msgr2=0x7fd680004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.135+0000 7fd699400640 1 --2- 172.21.15.133:0/3739154494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd680004800 0x7fd680004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.135+0000 7fd699e00640 1 --2- 172.21.15.133:0/3739154494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd680004800 0x7fd680004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.136 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.135+0000 7fd69a800640 1 -- 172.21.15.133:0/3739154494 reap_dead start 2024-03-07T06:53:56.137 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.135+0000 7fd699e00640 1 -- 172.21.15.133:0/3739154494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd680003520 con 0x7fd6800039b0 2024-03-07T06:53:56.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.136+0000 7fd699e00640 1 --2- 172.21.15.133:0/3739154494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6800039b0 0x7fd680003dd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009ac0 tx=0x7fd680005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68918d2a3550ffa5 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:56.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.136+0000 7fd698a00640 1 -- 172.21.15.133:0/3739154494 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c039070 con 0x7fd6800039b0 2024-03-07T06:53:56.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.136+0000 7fd698a00640 1 -- 172.21.15.133:0/3739154494 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c02c6d0 con 0x7fd6800039b0 2024-03-07T06:53:56.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.136+0000 7fd698a00640 1 -- 172.21.15.133:0/3739154494 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c009690 con 0x7fd6800039b0 2024-03-07T06:53:56.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.137+0000 7fd69b200640 1 -- 172.21.15.133:0/3739154494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6800039b0 msgr2=0x7fd680003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:56.138 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.137+0000 7fd69b200640 1 --2- 172.21.15.133:0/3739154494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6800039b0 0x7fd680003dd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009ac0 tx=0x7fd680005830 comp rx=0 tx=0).stop 2024-03-07T06:53:56.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.137+0000 7fd69b200640 1 -- 172.21.15.133:0/3739154494 shutdown_connections 2024-03-07T06:53:56.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.137+0000 7fd69b200640 1 --2- 172.21.15.133:0/3739154494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6800039b0 0x7fd680003dd0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.137+0000 7fd69b200640 1 -- 172.21.15.133:0/3739154494 >> 172.21.15.133:0/3739154494 conn(0x7fd69c0674e0 msgr2=0x7fd69c0d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:56.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.137+0000 7fd69b200640 1 -- 172.21.15.133:0/3739154494 shutdown_connections 2024-03-07T06:53:56.139 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.137+0000 7fd69b200640 1 -- 172.21.15.133:0/3739154494 wait complete. 2024-03-07T06:53:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.139+0000 7fd69b200640 1 Processor -- start 2024-03-07T06:53:56.140 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.139+0000 7fd69b200640 1 -- start start 2024-03-07T06:53:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd69b200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6800039b0 0x7fd69c1f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd69b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd69c1f41d0 0x7fd69c14f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd69b200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd69c14fcf0 0x7fd69c152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd69b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c161bb0 con 0x7fd69c1f41d0 2024-03-07T06:53:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd69b200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c144420 con 0x7fd6800039b0 2024-03-07T06:53:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd69b200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c161d30 con 0x7fd69c14fcf0 2024-03-07T06:53:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd699400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd69c1f41d0 0x7fd69c14f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd699400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd69c1f41d0 0x7fd69c14f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42428/0 (socket says 172.21.15.133:42428) 2024-03-07T06:53:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd699400640 1 -- 172.21.15.133:0/4099676147 learned_addr learned my addr 172.21.15.133:0/4099676147 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:56.142 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd699e00640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6800039b0 0x7fd69c1f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd69a800640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd69c14fcf0 0x7fd69c152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd699400640 1 -- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd69c14fcf0 msgr2=0x7fd69c152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd699400640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd69c14fcf0 0x7fd69c152e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd699400640 1 -- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6800039b0 msgr2=0x7fd69c1f3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd699400640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6800039b0 0x7fd69c1f3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd699400640 1 -- 172.21.15.133:0/4099676147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c14e9f0 con 0x7fd69c1f41d0 2024-03-07T06:53:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.140+0000 7fd699e00640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6800039b0 0x7fd69c1f3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.141+0000 7fd699400640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd69c1f41d0 0x7fd69c14f7b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd68800d480 tx=0x7fd68800d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.141+0000 7fd692000640 1 -- 172.21.15.133:0/4099676147 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688005480 con 0x7fd69c1f41d0 2024-03-07T06:53:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.141+0000 7fd692000640 1 -- 172.21.15.133:0/4099676147 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd688017070 con 0x7fd69c1f41d0 2024-03-07T06:53:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.141+0000 7fd692000640 1 -- 172.21.15.133:0/4099676147 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688003810 con 0x7fd69c1f41d0 2024-03-07T06:53:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.141+0000 7fd69b200640 1 -- 172.21.15.133:0/4099676147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69c14ecd0 con 0x7fd69c1f41d0 2024-03-07T06:53:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.141+0000 7fd69b200640 1 -- 172.21.15.133:0/4099676147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69c14f290 con 0x7fd69c1f41d0 2024-03-07T06:53:56.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.142+0000 7fd69b200640 1 -- 172.21.15.133:0/4099676147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69c062c90 con 0x7fd69c1f41d0 2024-03-07T06:53:56.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.144+0000 7fd692000640 1 -- 172.21.15.133:0/4099676147 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fd6880039b0 con 0x7fd69c1f41d0 2024-03-07T06:53:56.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.145+0000 7fd692000640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd67c06b4d0 0x7fd67c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.145+0000 7fd692000640 1 -- 172.21.15.133:0/4099676147 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd688089fe0 con 0x7fd69c1f41d0 2024-03-07T06:53:56.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.145+0000 7fd699e00640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd67c06b4d0 0x7fd67c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.146+0000 7fd699e00640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd67c06b4d0 0x7fd67c06d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd68c036000 tx=0x7fd68c030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:56.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.150+0000 7fd692000640 1 -- 172.21.15.133:0/4099676147 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fd688058f70 con 0x7fd69c1f41d0 2024-03-07T06:53:56.391 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.391+0000 7fd69b200640 1 -- 172.21.15.133:0/4099676147 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd69c15f220 con 0x7fd69c1f41d0 2024-03-07T06:53:56.392 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.393+0000 7fd692000640 1 -- 172.21.15.133:0/4099676147 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4980 (secure 0 0 0) 0x7fd688058870 con 0x7fd69c1f41d0 2024-03-07T06:53:56.393 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:56.393 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/44975892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":44975892},{"type":"v1","addr":"172.21.15.161:6835","nonce":44975892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4972,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/906115542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":906115542},{"type":"v1","addr":"172.21.15.133:6833","nonce":906115542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1998424684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1998424684},{"type":"v1","addr":"172.21.15.161:6837","nonce":1998424684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:53:24.627534+0000","modified":"2024-03-07T06:53:27.698471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/2951618124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2951618124},{"type":"v1","addr":"172.21.15.133:6835","nonce":2951618124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":3}]} 2024-03-07T06:53:56.393 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 27 2024-03-07T06:53:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.395+0000 7fd69b200640 1 -- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd67c06b4d0 msgr2=0x7fd67c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.395+0000 7fd69b200640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd67c06b4d0 0x7fd67c06d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd68c036000 tx=0x7fd68c030000 comp rx=0 tx=0).stop 2024-03-07T06:53:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.395+0000 7fd69b200640 1 -- 172.21.15.133:0/4099676147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd69c1f41d0 msgr2=0x7fd69c14f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:56.396 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.395+0000 7fd69b200640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd69c1f41d0 0x7fd69c14f7b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd68800d480 tx=0x7fd68800d940 comp rx=0 tx=0).stop 2024-03-07T06:53:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.395+0000 7fd69b200640 1 -- 172.21.15.133:0/4099676147 shutdown_connections 2024-03-07T06:53:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.395+0000 7fd69b200640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd69c14fcf0 0x7fd69c152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.395+0000 7fd69b200640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd69c1f41d0 0x7fd69c14f7b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.395+0000 7fd69b200640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fd67c06b4d0 0x7fd67c06d990 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.395+0000 7fd69b200640 1 --2- 172.21.15.133:0/4099676147 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6800039b0 0x7fd69c1f3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.395+0000 7fd69b200640 1 -- 172.21.15.133:0/4099676147 >> 172.21.15.133:0/4099676147 conn(0x7fd69c0674e0 msgr2=0x7fd69c0719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.396+0000 7fd69b200640 1 -- 172.21.15.133:0/4099676147 shutdown_connections 2024-03-07T06:53:56.397 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.396+0000 7fd69b200640 1 -- 172.21.15.133:0/4099676147 wait complete. 2024-03-07T06:53:56.413 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:53:56.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.615+0000 7f394e800640 1 Processor -- start 2024-03-07T06:53:56.615 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.615+0000 7f394e800640 1 -- start start 2024-03-07T06:53:56.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.617+0000 7f394e800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f39480cf7d0 0x7f39480cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.617+0000 7f394e800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39480731b0 con 0x7f394815fbb0 2024-03-07T06:53:56.616 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.617+0000 7f394e800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3948072150 con 0x7f3948072950 2024-03-07T06:53:56.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.617+0000 7f394e800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3948072f60 con 0x7f39480cf7d0 2024-03-07T06:53:56.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.617+0000 7f3947400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f394815fbb0 0x7f394815ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35062/0 (socket says 172.21.15.133:35062) 2024-03-07T06:53:56.617 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.617+0000 7f3947400640 1 -- 172.21.15.133:0/2404433423 learned_addr learned my addr 172.21.15.133:0/2404433423 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:56.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.617+0000 7f3946000640 1 -- 172.21.15.133:0/2404433423 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611295205 0 0) 0x7f39480731b0 con 0x7f394815fbb0 2024-03-07T06:53:56.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.617+0000 7f3947e00640 1 --2- 172.21.15.133:0/2404433423 >> v2:172.21.15.161:3300/0 conn(0x7f39480cf7d0 0x7f39480cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.617+0000 7f3946000640 1 -- 172.21.15.133:0/2404433423 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f392c003520 con 0x7f394815fbb0 2024-03-07T06:53:56.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.618+0000 7f3946000640 1 -- 172.21.15.133:0/2404433423 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1189662066 0 0) 0x7f392c003520 con 0x7f394815fbb0 2024-03-07T06:53:56.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.618+0000 7f3946000640 1 -- 172.21.15.133:0/2404433423 >> v2:172.21.15.161:3300/0 conn(0x7f39480cf7d0 msgr2=0x7f39480cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:56.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.618+0000 7f3946000640 1 --2- 172.21.15.133:0/2404433423 >> v2:172.21.15.161:3300/0 conn(0x7f39480cf7d0 0x7f39480cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.618+0000 7f3946000640 1 -- 172.21.15.133:0/2404433423 >> v1:172.21.15.161:6789/0 conn(0x7f3948072950 legacy=0x7f394815c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:56.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.618+0000 7f3946000640 1 -- 172.21.15.133:0/2404433423 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3948072430 con 0x7f394815fbb0 2024-03-07T06:53:56.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.618+0000 7f3946000640 1 -- 172.21.15.133:0/2404433423 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f3938001670 con 0x7f394815fbb0 2024-03-07T06:53:56.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.618+0000 7f3946000640 1 -- 172.21.15.133:0/2404433423 >> v1:172.21.15.133:6789/0 conn(0x7f394815fbb0 legacy=0x7f394815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:56.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.618+0000 7f3946000640 1 --2- 172.21.15.133:0/2404433423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f392c003a00 0x7f392c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.618+0000 7f3946000640 1 --2- 172.21.15.133:0/2404433423 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f392c004800 0x7f392c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.618+0000 7f3946000640 1 --2- 172.21.15.133:0/2404433423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f392c0052e0 0x7f392c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.618+0000 7f3946000640 1 -- 172.21.15.133:0/2404433423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3948072f60 con 0x7f392c003a00 2024-03-07T06:53:56.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.618+0000 7f3946000640 1 -- 172.21.15.133:0/2404433423 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c00c200 con 0x7f392c0052e0 2024-03-07T06:53:56.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.618+0000 7f3946000640 1 -- 172.21.15.133:0/2404433423 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c00c400 con 0x7f392c004800 2024-03-07T06:53:56.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.618+0000 7f3947400640 1 --2- 172.21.15.133:0/2404433423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f392c003a00 0x7f392c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.619+0000 7f3946a00640 1 --2- 172.21.15.133:0/2404433423 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f392c004800 0x7f392c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.620 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.619+0000 7f3947e00640 1 --2- 172.21.15.133:0/2404433423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f392c0052e0 0x7f392c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.619+0000 7f3947400640 1 -- 172.21.15.133:0/2404433423 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f392c004800 msgr2=0x7f392c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:56.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.619+0000 7f3947400640 1 --2- 172.21.15.133:0/2404433423 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f392c004800 0x7f392c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.619+0000 7f3947400640 1 -- 172.21.15.133:0/2404433423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f392c0052e0 msgr2=0x7f392c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:56.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.619+0000 7f3947400640 1 --2- 172.21.15.133:0/2404433423 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f392c0052e0 0x7f392c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.619+0000 7f3947e00640 1 -- 172.21.15.133:0/2404433423 reap_dead start 2024-03-07T06:53:56.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.619+0000 7f3947400640 1 -- 172.21.15.133:0/2404433423 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c003520 con 0x7f392c003a00 2024-03-07T06:53:56.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.619+0000 7f3946a00640 1 --2- 172.21.15.133:0/2404433423 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f392c004800 0x7f392c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:56.621 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.620+0000 7f3947400640 1 --2- 172.21.15.133:0/2404433423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f392c003a00 0x7f392c003e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f393800c110 tx=0x7f392c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed23e453ea45454 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:56.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.620+0000 7f3946000640 1 -- 172.21.15.133:0/2404433423 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938007c40 con 0x7f392c003a00 2024-03-07T06:53:56.622 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.620+0000 7f3946000640 1 -- 172.21.15.133:0/2404433423 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3938030070 con 0x7f392c003a00 2024-03-07T06:53:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.620+0000 7f3946000640 1 -- 172.21.15.133:0/2404433423 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938005d70 con 0x7f392c003a00 2024-03-07T06:53:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.621+0000 7f394e800640 1 -- 172.21.15.133:0/2404433423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f392c003a00 msgr2=0x7f392c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.621+0000 7f394e800640 1 --2- 172.21.15.133:0/2404433423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f392c003a00 0x7f392c003e20 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f393800c110 tx=0x7f392c005830 comp rx=0 tx=0).stop 2024-03-07T06:53:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.621+0000 7f394e800640 1 -- 172.21.15.133:0/2404433423 shutdown_connections 2024-03-07T06:53:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.621+0000 7f394e800640 1 --2- 172.21.15.133:0/2404433423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f392c003a00 0x7f392c003e20 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.621+0000 7f394e800640 1 -- 172.21.15.133:0/2404433423 >> 172.21.15.133:0/2404433423 conn(0x7f39480b3120 msgr2=0x7f39480b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.621+0000 7f394e800640 1 -- 172.21.15.133:0/2404433423 shutdown_connections 2024-03-07T06:53:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.621+0000 7f394e800640 1 -- 172.21.15.133:0/2404433423 wait complete. 2024-03-07T06:53:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.622+0000 7f394e800640 1 Processor -- start 2024-03-07T06:53:56.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.623+0000 7f394e800640 1 -- start start 2024-03-07T06:53:56.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.623+0000 7f394e800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f392c003a00 0x7f394809afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.623+0000 7f394e800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39480cc1b0 0x7f3948145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.623+0000 7f394e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3948146040 0x7f39480afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.623+0000 7f394e800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3948073490 con 0x7f3948146040 2024-03-07T06:53:56.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.623+0000 7f394e800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39480cfc00 con 0x7f39480cc1b0 2024-03-07T06:53:56.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.623+0000 7f394e800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39480b1a00 con 0x7f392c003a00 2024-03-07T06:53:56.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f3947e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3948146040 0x7f39480afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f3947e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3948146040 0x7f39480afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42440/0 (socket says 172.21.15.133:42440) 2024-03-07T06:53:56.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f3947e00640 1 -- 172.21.15.133:0/3119769948 learned_addr learned my addr 172.21.15.133:0/3119769948 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:56.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f3946a00640 1 --2- 172.21.15.133:0/3119769948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39480cc1b0 0x7f3948145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f3947e00640 1 -- 172.21.15.133:0/3119769948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f392c003a00 msgr2=0x7f394809afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:56.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f3947400640 1 --2- 172.21.15.133:0/3119769948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f392c003a00 0x7f394809afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f3947e00640 1 --2- 172.21.15.133:0/3119769948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f392c003a00 0x7f394809afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f3947e00640 1 -- 172.21.15.133:0/3119769948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39480cc1b0 msgr2=0x7f3948145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:56.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f3947e00640 1 --2- 172.21.15.133:0/3119769948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39480cc1b0 0x7f3948145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f3947e00640 1 -- 172.21.15.133:0/3119769948 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39480aebc0 con 0x7f3948146040 2024-03-07T06:53:56.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f3947e00640 1 --2- 172.21.15.133:0/3119769948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3948146040 0x7f39480afc60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f39400098c0 tx=0x7f3940009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:56.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f393f400640 1 -- 172.21.15.133:0/3119769948 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940019070 con 0x7f3948146040 2024-03-07T06:53:56.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f393f400640 1 -- 172.21.15.133:0/3119769948 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3940007cc0 con 0x7f3948146040 2024-03-07T06:53:56.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f394e800640 1 -- 172.21.15.133:0/3119769948 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39480aeea0 con 0x7f3948146040 2024-03-07T06:53:56.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f393f400640 1 -- 172.21.15.133:0/3119769948 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940014640 con 0x7f3948146040 2024-03-07T06:53:56.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.625+0000 7f394e800640 1 -- 172.21.15.133:0/3119769948 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39480af460 con 0x7f3948146040 2024-03-07T06:53:56.627 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.626+0000 7f394e800640 1 -- 172.21.15.133:0/3119769948 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3948062c90 con 0x7f3948146040 2024-03-07T06:53:56.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.627+0000 7f393f400640 1 -- 172.21.15.133:0/3119769948 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f394001c070 con 0x7f3948146040 2024-03-07T06:53:56.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.628+0000 7f393f400640 1 --2- 172.21.15.133:0/3119769948 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f392806b4d0 0x7f392806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:56.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.628+0000 7f393f400640 1 -- 172.21.15.133:0/3119769948 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3940089d00 con 0x7f3948146040 2024-03-07T06:53:56.628 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.629+0000 7f3947400640 1 --2- 172.21.15.133:0/3119769948 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f392806b4d0 0x7f392806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:56.630 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.630+0000 7f3947400640 1 --2- 172.21.15.133:0/3119769948 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f392806b4d0 0x7f392806d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3938034000 tx=0x7f3938039040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:56.633 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.634+0000 7f393f400640 1 -- 172.21.15.133:0/3119769948 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f3940058b40 con 0x7f3948146040 2024-03-07T06:53:56.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.869+0000 7f394e800640 1 -- 172.21.15.133:0/3119769948 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3948072d40 con 0x7f3948146040 2024-03-07T06:53:56.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.870+0000 7f393f400640 1 -- 172.21.15.133:0/3119769948 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4980 (secure 0 0 0) 0x7f3940058440 con 0x7f3948146040 2024-03-07T06:53:56.869 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:56.869 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/44975892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":44975892},{"type":"v1","addr":"172.21.15.161:6835","nonce":44975892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4972,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/906115542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":906115542},{"type":"v1","addr":"172.21.15.133:6833","nonce":906115542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1998424684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1998424684},{"type":"v1","addr":"172.21.15.161:6837","nonce":1998424684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-07T06:53:24.627534+0000","modified":"2024-03-07T06:53:27.698471+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6835/2951618124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2951618124},{"type":"v1","addr":"172.21.15.133:6835","nonce":2951618124}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5580,"qdb_cluster":[5580]},"id":3}]} 2024-03-07T06:53:56.869 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 27 2024-03-07T06:53:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.873+0000 7f394e800640 1 -- 172.21.15.133:0/3119769948 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f392806b4d0 msgr2=0x7f392806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.873+0000 7f394e800640 1 --2- 172.21.15.133:0/3119769948 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f392806b4d0 0x7f392806d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3938034000 tx=0x7f3938039040 comp rx=0 tx=0).stop 2024-03-07T06:53:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.873+0000 7f394e800640 1 -- 172.21.15.133:0/3119769948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3948146040 msgr2=0x7f39480afc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.873+0000 7f394e800640 1 --2- 172.21.15.133:0/3119769948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3948146040 0x7f39480afc60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f39400098c0 tx=0x7f3940009d80 comp rx=0 tx=0).stop 2024-03-07T06:53:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.873+0000 7f394e800640 1 -- 172.21.15.133:0/3119769948 shutdown_connections 2024-03-07T06:53:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.873+0000 7f394e800640 1 --2- 172.21.15.133:0/3119769948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3948146040 0x7f39480afc60 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.873+0000 7f394e800640 1 --2- 172.21.15.133:0/3119769948 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f392806b4d0 0x7f392806d990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.873+0000 7f394e800640 1 --2- 172.21.15.133:0/3119769948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f39480cc1b0 0x7f3948145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.873+0000 7f394e800640 1 --2- 172.21.15.133:0/3119769948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f392c003a00 0x7f394809afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:56.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.873+0000 7f394e800640 1 -- 172.21.15.133:0/3119769948 >> 172.21.15.133:0/3119769948 conn(0x7f39480b3120 msgr2=0x7f39480740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:56.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.873+0000 7f394e800640 1 -- 172.21.15.133:0/3119769948 shutdown_connections 2024-03-07T06:53:56.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:56.873+0000 7f394e800640 1 -- 172.21.15.133:0/3119769948 wait complete. 2024-03-07T06:53:56.890 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:53:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.115+0000 7f5682200640 1 Processor -- start 2024-03-07T06:53:57.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.115+0000 7f5682200640 1 -- start start 2024-03-07T06:53:57.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.117+0000 7f5682200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f567c0d2ce0 0x7f567c0b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:57.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.117+0000 7f5682200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f567c0b6380 0x7f567c0b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.117+0000 7f5682200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567c0b9020 con 0x7f567c147770 2024-03-07T06:53:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.117+0000 7f5682200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f567c15e6b0 con 0x7f567c0b6380 2024-03-07T06:53:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.117+0000 7f5682200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567c15e990 con 0x7f567c0d2ce0 2024-03-07T06:53:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.117+0000 7f5680e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f567c147770 0x7f567c0d25d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35074/0 (socket says 172.21.15.133:35074) 2024-03-07T06:53:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.117+0000 7f5680e00640 1 -- 172.21.15.133:0/922602305 learned_addr learned my addr 172.21.15.133:0/922602305 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.117+0000 7f5681800640 1 --2- 172.21.15.133:0/922602305 >> v2:172.21.15.133:3300/0 conn(0x7f567c0b6380 0x7f567c0b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.117+0000 7f567be00640 1 --2- 172.21.15.133:0/922602305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f567c0d2ce0 0x7f567c0b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.117+0000 7f567b400640 1 -- 172.21.15.133:0/922602305 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 97751800 0 0) 0x7f567c0b9020 con 0x7f567c147770 2024-03-07T06:53:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.118+0000 7f5681800640 1 -- 172.21.15.133:0/922602305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f567c0d2ce0 msgr2=0x7f567c0b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.118+0000 7f5681800640 1 --2- 172.21.15.133:0/922602305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f567c0d2ce0 0x7f567c0b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.118+0000 7f5681800640 1 -- 172.21.15.133:0/922602305 >> v1:172.21.15.133:6789/0 conn(0x7f567c147770 legacy=0x7f567c0d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.118+0000 7f5681800640 1 -- 172.21.15.133:0/922602305 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c15ec70 con 0x7f567c0b6380 2024-03-07T06:53:57.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.119+0000 7f5681800640 1 --2- 172.21.15.133:0/922602305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c0b6380 0x7f567c0b8ae0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f56740097a0 tx=0x7f567402b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967186b814c37f0c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:57.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.119+0000 7f567b400640 1 -- 172.21.15.133:0/922602305 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674034070 con 0x7f567c0b6380 2024-03-07T06:53:57.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.119+0000 7f567b400640 1 -- 172.21.15.133:0/922602305 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5674030800 con 0x7f567c0b6380 2024-03-07T06:53:57.119 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.119+0000 7f567b400640 1 -- 172.21.15.133:0/922602305 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5674030bd0 con 0x7f567c0b6380 2024-03-07T06:53:57.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.119+0000 7f5682200640 1 -- 172.21.15.133:0/922602305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c0b6380 msgr2=0x7f567c0b8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:57.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.119+0000 7f5682200640 1 --2- 172.21.15.133:0/922602305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c0b6380 0x7f567c0b8ae0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f56740097a0 tx=0x7f567402b940 comp rx=0 tx=0).stop 2024-03-07T06:53:57.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.120+0000 7f5682200640 1 -- 172.21.15.133:0/922602305 shutdown_connections 2024-03-07T06:53:57.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.120+0000 7f5682200640 1 --2- 172.21.15.133:0/922602305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c0b6380 0x7f567c0b8ae0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.120+0000 7f5682200640 1 --2- 172.21.15.133:0/922602305 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f567c0d2ce0 0x7f567c0b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.120+0000 7f5682200640 1 -- 172.21.15.133:0/922602305 >> 172.21.15.133:0/922602305 conn(0x7f567c067590 msgr2=0x7f567c0d08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:57.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.120+0000 7f5682200640 1 -- 172.21.15.133:0/922602305 shutdown_connections 2024-03-07T06:53:57.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.120+0000 7f5682200640 1 -- 172.21.15.133:0/922602305 wait complete. 2024-03-07T06:53:57.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.122+0000 7f5682200640 1 Processor -- start 2024-03-07T06:53:57.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.123+0000 7f5682200640 1 -- start start 2024-03-07T06:53:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.124+0000 7f5682200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c0b6380 0x7f567c13c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.124+0000 7f5682200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f567c0d2ce0 0x7f567c13f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.124+0000 7f5682200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f567c147770 0x7f567c0d1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:57.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.124+0000 7f5682200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c147b50 con 0x7f567c0b6380 2024-03-07T06:53:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.124+0000 7f5682200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c15f230 con 0x7f567c147770 2024-03-07T06:53:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.124+0000 7f5682200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567c0b44f0 con 0x7f567c0d2ce0 2024-03-07T06:53:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.125+0000 7f5680e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c0b6380 0x7f567c13c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.125+0000 7f5680e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c0b6380 0x7f567c13c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42462/0 (socket says 172.21.15.133:42462) 2024-03-07T06:53:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.125+0000 7f5680e00640 1 -- 172.21.15.133:0/2944956202 learned_addr learned my addr 172.21.15.133:0/2944956202 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.125+0000 7f567be00640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f567c0d2ce0 0x7f567c13f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.125+0000 7f5681800640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f567c147770 0x7f567c0d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.125+0000 7f5680e00640 1 -- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f567c0d2ce0 msgr2=0x7f567c13f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.125+0000 7f5680e00640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f567c0d2ce0 0x7f567c13f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.125+0000 7f5680e00640 1 -- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f567c147770 msgr2=0x7f567c0d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.125+0000 7f5680e00640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f567c147770 0x7f567c0d1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.125+0000 7f5680e00640 1 -- 172.21.15.133:0/2944956202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c13f7f0 con 0x7f567c0b6380 2024-03-07T06:53:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.125+0000 7f567be00640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f567c0d2ce0 0x7f567c13f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:57.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.125+0000 7f5680e00640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c0b6380 0x7f567c13c560 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c7c0 tx=0x7f566c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:57.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.126+0000 7f5678c00640 1 -- 172.21.15.133:0/2944956202 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c018070 con 0x7f567c0b6380 2024-03-07T06:53:57.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.126+0000 7f5678c00640 1 -- 172.21.15.133:0/2944956202 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c00a640 con 0x7f567c0b6380 2024-03-07T06:53:57.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.126+0000 7f5678c00640 1 -- 172.21.15.133:0/2944956202 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c00aa30 con 0x7f567c0b6380 2024-03-07T06:53:57.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.126+0000 7f5682200640 1 -- 172.21.15.133:0/2944956202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567c15c8d0 con 0x7f567c0b6380 2024-03-07T06:53:57.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.126+0000 7f5682200640 1 -- 172.21.15.133:0/2944956202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567c15ce90 con 0x7f567c0b6380 2024-03-07T06:53:57.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.127+0000 7f5682200640 1 -- 172.21.15.133:0/2944956202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567c062c90 con 0x7f567c0b6380 2024-03-07T06:53:57.126 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.128+0000 7f5678c00640 1 -- 172.21.15.133:0/2944956202 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f566c01b070 con 0x7f567c0b6380 2024-03-07T06:53:57.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.129+0000 7f5678c00640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f565806b4d0 0x7f565806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:57.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.129+0000 7f5678c00640 1 -- 172.21.15.133:0/2944956202 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f566c0897c0 con 0x7f567c0b6380 2024-03-07T06:53:57.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.130+0000 7f567be00640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f565806b4d0 0x7f565806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:57.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.131+0000 7f567be00640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f565806b4d0 0x7f565806d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f567c15a6c0 tx=0x7f5664008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:57.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.133+0000 7f5678c00640 1 -- 172.21.15.133:0/2944956202 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f566c058690 con 0x7f567c0b6380 2024-03-07T06:53:57.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.353+0000 7f5682200640 1 -- 172.21.15.133:0/2944956202 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f567c0b5c50 con 0x7f567c0b6380 2024-03-07T06:53:57.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.354+0000 7f5678c00640 1 -- 172.21.15.133:0/2944956202 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16752 (secure 0 0 0) 0x7f566c057f90 con 0x7f567c0b6380 2024-03-07T06:53:57.353 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:53:57.354 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":48,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:53:57.283178+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-07T06:53:22.398527+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-07T06:53:23.628483+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":4}]},{"pgid":"7.13","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:53:57.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.357+0000 7f5682200640 1 -- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f565806b4d0 msgr2=0x7f565806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:57.356 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.357+0000 7f5682200640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f565806b4d0 0x7f565806d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f567c15a6c0 tx=0x7f5664008040 comp rx=0 tx=0).stop 2024-03-07T06:53:57.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.357+0000 7f5682200640 1 -- 172.21.15.133:0/2944956202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c0b6380 msgr2=0x7f567c13c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:57.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.357+0000 7f5682200640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c0b6380 0x7f567c13c560 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c7c0 tx=0x7f566c00cc80 comp rx=0 tx=0).stop 2024-03-07T06:53:57.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.357+0000 7f5682200640 1 -- 172.21.15.133:0/2944956202 shutdown_connections 2024-03-07T06:53:57.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.357+0000 7f5682200640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f565806b4d0 0x7f565806d990 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.358+0000 7f5682200640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f567c147770 0x7f567c0d1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.358+0000 7f5682200640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f567c0d2ce0 0x7f567c13f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.358+0000 7f5682200640 1 --2- 172.21.15.133:0/2944956202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c0b6380 0x7f567c13c560 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.358+0000 7f5682200640 1 -- 172.21.15.133:0/2944956202 >> 172.21.15.133:0/2944956202 conn(0x7f567c067590 msgr2=0x7f567c0d08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:57.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.358+0000 7f5682200640 1 -- 172.21.15.133:0/2944956202 shutdown_connections 2024-03-07T06:53:57.357 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.358+0000 7f5682200640 1 -- 172.21.15.133:0/2944956202 wait complete. 2024-03-07T06:53:57.379 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-07T06:53:57.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.586+0000 7feebf200640 1 Processor -- start 2024-03-07T06:53:57.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.586+0000 7feebf200640 1 -- start start 2024-03-07T06:53:57.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.587+0000 7feebf200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feec0140cb0 0x7feec0162290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:57.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.588+0000 7feebf200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7feec00d2b00 0x7feec00b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:57.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.588+0000 7feebf200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec0143810 con 0x7feec00b70d0 2024-03-07T06:53:57.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.588+0000 7feebf200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec01435c0 con 0x7feec0140cb0 2024-03-07T06:53:57.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.588+0000 7feebf200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec0144420 con 0x7feec00d2b00 2024-03-07T06:53:57.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.588+0000 7feebde00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7feec0140cb0 0x7feec0162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:57.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.588+0000 7feebd400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7feec00b70d0 0x7feec00b9300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35086/0 (socket says 172.21.15.133:35086) 2024-03-07T06:53:57.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.588+0000 7feebd400640 1 -- 172.21.15.133:0/1518566124 learned_addr learned my addr 172.21.15.133:0/1518566124 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:57.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.588+0000 7feebe800640 1 --2- 172.21.15.133:0/1518566124 >> v2:172.21.15.161:3300/0 conn(0x7feec00d2b00 0x7feec00b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:57.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.588+0000 7feebde00640 1 -- 172.21.15.133:0/1518566124 >> v2:172.21.15.161:3300/0 conn(0x7feec00d2b00 msgr2=0x7feec00b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:57.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.588+0000 7feebde00640 1 --2- 172.21.15.133:0/1518566124 >> v2:172.21.15.161:3300/0 conn(0x7feec00d2b00 0x7feec00b7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.589+0000 7feebde00640 1 -- 172.21.15.133:0/1518566124 >> v1:172.21.15.133:6789/0 conn(0x7feec00b70d0 legacy=0x7feec00b9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:57.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.589+0000 7feebde00640 1 -- 172.21.15.133:0/1518566124 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec0143af0 con 0x7feec0140cb0 2024-03-07T06:53:57.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.589+0000 7feebca00640 1 -- 172.21.15.133:0/1518566124 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954199002 0 0) 0x7feec0143810 con 0x7feec00b70d0 2024-03-07T06:53:57.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.590+0000 7feebde00640 1 --2- 172.21.15.133:0/1518566124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec0140cb0 0x7feec0162290 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7feec015f4e0 tx=0x7feeb002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866a6b794fc45165 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:57.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.590+0000 7feebca00640 1 -- 172.21.15.133:0/1518566124 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb00315a0 con 0x7feec0140cb0 2024-03-07T06:53:57.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.590+0000 7feebca00640 1 -- 172.21.15.133:0/1518566124 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb0037070 con 0x7feec0140cb0 2024-03-07T06:53:57.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.590+0000 7feebca00640 1 -- 172.21.15.133:0/1518566124 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb0031960 con 0x7feec0140cb0 2024-03-07T06:53:57.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.591+0000 7feebf200640 1 -- 172.21.15.133:0/1518566124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec0140cb0 msgr2=0x7feec0162290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:57.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.591+0000 7feebf200640 1 --2- 172.21.15.133:0/1518566124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec0140cb0 0x7feec0162290 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7feec015f4e0 tx=0x7feeb002b840 comp rx=0 tx=0).stop 2024-03-07T06:53:57.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.591+0000 7feebf200640 1 -- 172.21.15.133:0/1518566124 shutdown_connections 2024-03-07T06:53:57.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.591+0000 7feebf200640 1 --2- 172.21.15.133:0/1518566124 >> v2:172.21.15.161:3300/0 conn(0x7feec00d2b00 0x7feec00b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.591+0000 7feebf200640 1 --2- 172.21.15.133:0/1518566124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec0140cb0 0x7feec0162290 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.591+0000 7feebf200640 1 -- 172.21.15.133:0/1518566124 >> 172.21.15.133:0/1518566124 conn(0x7feec00674e0 msgr2=0x7feec00d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:57.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.591+0000 7feebf200640 1 -- 172.21.15.133:0/1518566124 shutdown_connections 2024-03-07T06:53:57.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.591+0000 7feebf200640 1 -- 172.21.15.133:0/1518566124 wait complete. 2024-03-07T06:53:57.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.593+0000 7feebf200640 1 Processor -- start 2024-03-07T06:53:57.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.594+0000 7feebf200640 1 -- start start 2024-03-07T06:53:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.595+0000 7feebf200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feec00b70d0 0x7feec01390e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.595+0000 7feebf200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec00d2b00 0x7feec0157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.595+0000 7feebf200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feec0140cb0 0x7feec0145650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.595+0000 7feebf200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec01627d0 con 0x7feec00d2b00 2024-03-07T06:53:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.595+0000 7feebf200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec01440b0 con 0x7feec0140cb0 2024-03-07T06:53:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.595+0000 7feebf200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec0161bb0 con 0x7feec00b70d0 2024-03-07T06:53:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.596+0000 7feebd400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec00d2b00 0x7feec0157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.596+0000 7feebd400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec00d2b00 0x7feec0157140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42470/0 (socket says 172.21.15.133:42470) 2024-03-07T06:53:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.596+0000 7feebd400640 1 -- 172.21.15.133:0/2432383462 learned_addr learned my addr 172.21.15.133:0/2432383462 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:57.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.596+0000 7feebde00640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feec00b70d0 0x7feec01390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:57.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.596+0000 7feebe800640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feec0140cb0 0x7feec0145650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:57.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.596+0000 7feebd400640 1 -- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feec00b70d0 msgr2=0x7feec01390e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:57.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.596+0000 7feebd400640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feec00b70d0 0x7feec01390e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.596+0000 7feebd400640 1 -- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feec0140cb0 msgr2=0x7feec0145650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:57.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.596+0000 7feebd400640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feec0140cb0 0x7feec0145650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.596+0000 7feebd400640 1 -- 172.21.15.133:0/2432383462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec0139620 con 0x7feec00d2b00 2024-03-07T06:53:57.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.596+0000 7feebde00640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feec00b70d0 0x7feec01390e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:57.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.596+0000 7feebe800640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feec0140cb0 0x7feec0145650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:57.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.596+0000 7feebd400640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec00d2b00 0x7feec0157140 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c430 tx=0x7feeac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:57.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.597+0000 7feeb6000640 1 -- 172.21.15.133:0/2432383462 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac0074e0 con 0x7feec00d2b00 2024-03-07T06:53:57.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.597+0000 7feeb6000640 1 -- 172.21.15.133:0/2432383462 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac0047e0 con 0x7feec00d2b00 2024-03-07T06:53:57.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.597+0000 7feebf200640 1 -- 172.21.15.133:0/2432383462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec0157990 con 0x7feec00d2b00 2024-03-07T06:53:57.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.597+0000 7feeb6000640 1 -- 172.21.15.133:0/2432383462 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac004bd0 con 0x7feec00d2b00 2024-03-07T06:53:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.597+0000 7feebf200640 1 -- 172.21.15.133:0/2432383462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec0157f50 con 0x7feec00d2b00 2024-03-07T06:53:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.598+0000 7feebf200640 1 -- 172.21.15.133:0/2432383462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec015fe20 con 0x7feec00d2b00 2024-03-07T06:53:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.599+0000 7feeb6000640 1 -- 172.21.15.133:0/2432383462 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7feeac011070 con 0x7feec00d2b00 2024-03-07T06:53:57.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.600+0000 7feeb6000640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7feea006b4d0 0x7feea006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:57.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.600+0000 7feeb6000640 1 -- 172.21.15.133:0/2432383462 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5767+0+0 (secure 0 0 0) 0x7feeac089f50 con 0x7feec00d2b00 2024-03-07T06:53:57.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.601+0000 7feebde00640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7feea006b4d0 0x7feea006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:57.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.602+0000 7feebde00640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7feea006b4d0 0x7feea006d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7feeb0036000 tx=0x7feeb0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:57.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.604+0000 7feeb6000640 1 -- 172.21.15.133:0/2432383462 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7feec015fe20 con 0x7feec00d2b00 2024-03-07T06:53:57.852 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.854+0000 7feebf200640 1 -- 172.21.15.133:0/2432383462 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7feec0157680 con 0x7feec00d2b00 2024-03-07T06:53:57.853 INFO:tasks.ceph.mon.a.smithi133.stderr:2024-03-07T06:53:57.855+0000 7fb4e8800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-07T06:53:57.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.858+0000 7feeb6000640 1 -- 172.21.15.133:0/2432383462 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7feeac058cf0 con 0x7feec00d2b00 2024-03-07T06:53:57.857 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-07T06:53:57.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.861+0000 7feebf200640 1 -- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7feea006b4d0 msgr2=0x7feea006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:57.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.861+0000 7feebf200640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7feea006b4d0 0x7feea006d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7feeb0036000 tx=0x7feeb0008030 comp rx=0 tx=0).stop 2024-03-07T06:53:57.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.861+0000 7feebf200640 1 -- 172.21.15.133:0/2432383462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec00d2b00 msgr2=0x7feec0157140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:57.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.861+0000 7feebf200640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec00d2b00 0x7feec0157140 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c430 tx=0x7feeac00c8f0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.861+0000 7feebf200640 1 -- 172.21.15.133:0/2432383462 shutdown_connections 2024-03-07T06:53:57.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.861+0000 7feebf200640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7feea006b4d0 0x7feea006d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.861+0000 7feebf200640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7feec0140cb0 0x7feec0145650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.861+0000 7feebf200640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feec00d2b00 0x7feec0157140 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.861+0000 7feebf200640 1 --2- 172.21.15.133:0/2432383462 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7feec00b70d0 0x7feec01390e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:57.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.861+0000 7feebf200640 1 -- 172.21.15.133:0/2432383462 >> 172.21.15.133:0/2432383462 conn(0x7feec00674e0 msgr2=0x7feec00b92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:57.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.861+0000 7feebf200640 1 -- 172.21.15.133:0/2432383462 shutdown_connections 2024-03-07T06:53:57.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:57.862+0000 7feebf200640 1 -- 172.21.15.133:0/2432383462 wait complete. 2024-03-07T06:53:57.888 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-07T06:53:57.952 INFO:tasks.ceph.mds.a.smithi133.stdout:starting mds.a at 2024-03-07T06:53:58.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.085+0000 7fe8b8600640 1 Processor -- start 2024-03-07T06:53:58.084 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.086+0000 7fe8b8600640 1 -- start start 2024-03-07T06:53:58.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.087+0000 7fe8b8600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe8b00ce230 0x7fe8b00b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:58.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.087+0000 7fe8b8600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe8b00b6750 0x7fe8b00b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:58.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.087+0000 7fe8b8600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b00b6b60 con 0x7fe8b015f300 2024-03-07T06:53:58.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.087+0000 7fe8b8600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b006f1f0 con 0x7fe8b00b6750 2024-03-07T06:53:58.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.087+0000 7fe8b8600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b006e430 con 0x7fe8b00ce230 2024-03-07T06:53:58.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.088+0000 7fe8b6600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe8b00b6750 0x7fe8b00b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:58.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.088+0000 7fe8b5c00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fe8b015f300 0x7fe8b00b2040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35090/0 (socket says 172.21.15.133:35090) 2024-03-07T06:53:58.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.088+0000 7fe8b6600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe8b00b6750 0x7fe8b00b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42490/0 (socket says 172.21.15.133:42490) 2024-03-07T06:53:58.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.088+0000 7fe8b5c00640 1 -- 172.21.15.133:0/461333879 learned_addr learned my addr 172.21.15.133:0/461333879 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:58.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.088+0000 7fe8b5200640 1 --2- 172.21.15.133:0/461333879 >> v2:172.21.15.161:3300/0 conn(0x7fe8b00ce230 0x7fe8b00b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:58.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.088+0000 7fe8afe00640 1 -- 172.21.15.133:0/461333879 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926823402 0 0) 0x7fe8a4001110 con 0x7fe8b015f300 2024-03-07T06:53:58.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.088+0000 7fe8afe00640 1 -- 172.21.15.133:0/461333879 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe898003520 con 0x7fe8b015f300 2024-03-07T06:53:58.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.088+0000 7fe8b6600640 1 -- 172.21.15.133:0/461333879 >> v2:172.21.15.161:3300/0 conn(0x7fe8b00ce230 msgr2=0x7fe8b00b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:58.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.088+0000 7fe8b6600640 1 --2- 172.21.15.133:0/461333879 >> v2:172.21.15.161:3300/0 conn(0x7fe8b00ce230 0x7fe8b00b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:58.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.088+0000 7fe8b6600640 1 -- 172.21.15.133:0/461333879 >> v1:172.21.15.133:6789/0 conn(0x7fe8b015f300 legacy=0x7fe8b00b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:58.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.088+0000 7fe8b6600640 1 -- 172.21.15.133:0/461333879 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b006e710 con 0x7fe8b00b6750 2024-03-07T06:53:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.090+0000 7fe8b6600640 1 --2- 172.21.15.133:0/461333879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b00b6750 0x7fe8b00b3cc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00ce6c0 tx=0x7fe8a802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896e32741dae4a0b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.090+0000 7fe8afe00640 1 -- 172.21.15.133:0/461333879 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8039070 con 0x7fe8b00b6750 2024-03-07T06:53:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.090+0000 7fe8afe00640 1 -- 172.21.15.133:0/461333879 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a802cbf0 con 0x7fe8b00b6750 2024-03-07T06:53:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.090+0000 7fe8afe00640 1 -- 172.21.15.133:0/461333879 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8034440 con 0x7fe8b00b6750 2024-03-07T06:53:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.090+0000 7fe8b8600640 1 -- 172.21.15.133:0/461333879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b00b6750 msgr2=0x7fe8b00b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:58.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.090+0000 7fe8b8600640 1 --2- 172.21.15.133:0/461333879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b00b6750 0x7fe8b00b3cc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe8b00ce6c0 tx=0x7fe8a802be40 comp rx=0 tx=0).stop 2024-03-07T06:53:58.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.091+0000 7fe8b8600640 1 -- 172.21.15.133:0/461333879 shutdown_connections 2024-03-07T06:53:58.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.091+0000 7fe8b8600640 1 --2- 172.21.15.133:0/461333879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b00b6750 0x7fe8b00b3cc0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:58.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.091+0000 7fe8b8600640 1 --2- 172.21.15.133:0/461333879 >> v2:172.21.15.161:3300/0 conn(0x7fe8b00ce230 0x7fe8b00b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:58.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.091+0000 7fe8b8600640 1 -- 172.21.15.133:0/461333879 >> 172.21.15.133:0/461333879 conn(0x7fe8b00cd710 msgr2=0x7fe8b00cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:58.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.091+0000 7fe8b8600640 1 -- 172.21.15.133:0/461333879 shutdown_connections 2024-03-07T06:53:58.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.091+0000 7fe8b8600640 1 -- 172.21.15.133:0/461333879 wait complete. 2024-03-07T06:53:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.092+0000 7fe8b8600640 1 Processor -- start 2024-03-07T06:53:58.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.093+0000 7fe8b8600640 1 -- start start 2024-03-07T06:53:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.093+0000 7fe8b8600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8b00b6750 0x7fe8b0149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.093+0000 7fe8b8600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8b00ce230 0x7fe8b0159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.093+0000 7fe8b8600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b015f300 0x7fe8b01598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.093+0000 7fe8b8600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b015f6e0 con 0x7fe8b015f300 2024-03-07T06:53:58.095 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.093+0000 7fe8b8600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b006ecd0 con 0x7fe8b00ce230 2024-03-07T06:53:58.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.093+0000 7fe8b8600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b015f860 con 0x7fe8b00b6750 2024-03-07T06:53:58.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.094+0000 7fe8b6600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b015f300 0x7fe8b01598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:58.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.094+0000 7fe8b6600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b015f300 0x7fe8b01598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42504/0 (socket says 172.21.15.133:42504) 2024-03-07T06:53:58.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.094+0000 7fe8b6600640 1 -- 172.21.15.133:0/1653710231 learned_addr learned my addr 172.21.15.133:0/1653710231 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:58.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.094+0000 7fe8b5c00640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8b00b6750 0x7fe8b0149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:58.096 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.094+0000 7fe8b6600640 1 -- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8b00b6750 msgr2=0x7fe8b0149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:58.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.094+0000 7fe8b5200640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8b00ce230 0x7fe8b0159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:58.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.094+0000 7fe8b6600640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8b00b6750 0x7fe8b0149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:58.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.094+0000 7fe8b6600640 1 -- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8b00ce230 msgr2=0x7fe8b0159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:58.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.094+0000 7fe8b6600640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8b00ce230 0x7fe8b0159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:58.097 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.094+0000 7fe8b6600640 1 -- 172.21.15.133:0/1653710231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b0157e80 con 0x7fe8b015f300 2024-03-07T06:53:58.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.094+0000 7fe8b5c00640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8b00b6750 0x7fe8b0149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:58.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.095+0000 7fe8b6600640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b015f300 0x7fe8b01598b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009e80 tx=0x7fe8a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:58.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.095+0000 7fe8ad600640 1 -- 172.21.15.133:0/1653710231 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8039050 con 0x7fe8b015f300 2024-03-07T06:53:58.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.095+0000 7fe8ad600640 1 -- 172.21.15.133:0/1653710231 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a8045070 con 0x7fe8b015f300 2024-03-07T06:53:58.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.095+0000 7fe8b8600640 1 -- 172.21.15.133:0/1653710231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b0159ec0 con 0x7fe8b015f300 2024-03-07T06:53:58.098 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.095+0000 7fe8ad600640 1 -- 172.21.15.133:0/1653710231 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8009450 con 0x7fe8b015f300 2024-03-07T06:53:58.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.095+0000 7fe8b8600640 1 -- 172.21.15.133:0/1653710231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b014a790 con 0x7fe8b015f300 2024-03-07T06:53:58.099 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.096+0000 7fe8b8600640 1 -- 172.21.15.133:0/1653710231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b015a3e0 con 0x7fe8b015f300 2024-03-07T06:53:58.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.097+0000 7fe8ad600640 1 -- 172.21.15.133:0/1653710231 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe8a803e850 con 0x7fe8b015f300 2024-03-07T06:53:58.100 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.098+0000 7fe8ad600640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe89006b4d0 0x7fe89006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:58.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.099+0000 7fe8ad600640 1 -- 172.21.15.133:0/1653710231 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe8a80ad330 con 0x7fe8b015f300 2024-03-07T06:53:58.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.099+0000 7fe8b5c00640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe89006b4d0 0x7fe89006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:58.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.100+0000 7fe8b5c00640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe89006b4d0 0x7fe89006d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40012b0 tx=0x7fe8a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:58.102 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.103+0000 7fe8ad600640 1 -- 172.21.15.133:0/1653710231 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe8b015a3e0 con 0x7fe8b015f300 2024-03-07T06:53:58.339 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.340+0000 7fe8b8600640 1 -- 172.21.15.133:0/1653710231 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe8b0062c90 con 0x7fe8b015f300 2024-03-07T06:53:58.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.862+0000 7fe8ad600640 1 -- 172.21.15.133:0/1653710231 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe8a807c1a0 con 0x7fe8b015f300 2024-03-07T06:53:58.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.865+0000 7fe8b8600640 1 -- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe89006b4d0 msgr2=0x7fe89006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:58.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.865+0000 7fe8b8600640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe89006b4d0 0x7fe89006d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40012b0 tx=0x7fe8a4009040 comp rx=0 tx=0).stop 2024-03-07T06:53:58.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.865+0000 7fe8b8600640 1 -- 172.21.15.133:0/1653710231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b015f300 msgr2=0x7fe8b01598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:58.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.865+0000 7fe8b8600640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b015f300 0x7fe8b01598b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009e80 tx=0x7fe8a8004a40 comp rx=0 tx=0).stop 2024-03-07T06:53:58.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.866+0000 7fe8b8600640 1 -- 172.21.15.133:0/1653710231 shutdown_connections 2024-03-07T06:53:58.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.866+0000 7fe8b8600640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8b015f300 0x7fe8b01598b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:58.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.866+0000 7fe8b8600640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe89006b4d0 0x7fe89006d990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:58.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.866+0000 7fe8b8600640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8b00ce230 0x7fe8b0159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:58.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.866+0000 7fe8b8600640 1 --2- 172.21.15.133:0/1653710231 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8b00b6750 0x7fe8b0149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:58.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.866+0000 7fe8b8600640 1 -- 172.21.15.133:0/1653710231 >> 172.21.15.133:0/1653710231 conn(0x7fe8b00cd710 msgr2=0x7fe8b00b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:58.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.866+0000 7fe8b8600640 1 -- 172.21.15.133:0/1653710231 shutdown_connections 2024-03-07T06:53:58.865 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:58.866+0000 7fe8b8600640 1 -- 172.21.15.133:0/1653710231 wait complete. 2024-03-07T06:53:58.888 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-07T06:53:59.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.102+0000 7fa3cbe00640 1 Processor -- start 2024-03-07T06:53:59.101 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.102+0000 7fa3cbe00640 1 -- start start 2024-03-07T06:53:59.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.104+0000 7fa3cbe00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa3cc15afc0 0x7fa3cc15bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:59.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.104+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3cc072b80 0x7fa3cc15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:59.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.104+0000 7fa3cbe00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3cc0705a0 con 0x7fa3cc0ce0b0 2024-03-07T06:53:59.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.104+0000 7fa3cbe00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3cc070880 con 0x7fa3cc15afc0 2024-03-07T06:53:59.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.104+0000 7fa3cbe00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc15b450 con 0x7fa3cc072b80 2024-03-07T06:53:59.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.104+0000 7fa3caa00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa3cc0ce0b0 0x7fa3cc0ce490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:35092/0 (socket says 172.21.15.133:35092) 2024-03-07T06:53:59.103 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.104+0000 7fa3caa00640 1 -- 172.21.15.133:0/136611751 learned_addr learned my addr 172.21.15.133:0/136611751 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:53:59.105 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3c2000640 1 --2- 172.21.15.133:0/136611751 >> v2:172.21.15.161:3300/0 conn(0x7fa3cc15afc0 0x7fa3cc15bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 -- 172.21.15.133:0/136611751 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 72935201 0 0) 0x7fa3cc0705a0 con 0x7fa3cc0ce0b0 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 -- 172.21.15.133:0/136611751 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3b0003520 con 0x7fa3cc0ce0b0 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3cb400640 1 --2- 172.21.15.133:0/136611751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3cc072b80 0x7fa3cc15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 -- 172.21.15.133:0/136611751 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1829860449 0 0) 0x7fa3b0003520 con 0x7fa3cc0ce0b0 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 -- 172.21.15.133:0/136611751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3cc072b80 msgr2=0x7fa3cc15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 --2- 172.21.15.133:0/136611751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3cc072b80 0x7fa3cc15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 -- 172.21.15.133:0/136611751 >> v2:172.21.15.161:3300/0 conn(0x7fa3cc15afc0 msgr2=0x7fa3cc15bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 --2- 172.21.15.133:0/136611751 >> v2:172.21.15.161:3300/0 conn(0x7fa3cc15afc0 0x7fa3cc15bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 -- 172.21.15.133:0/136611751 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc15b730 con 0x7fa3cc0ce0b0 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 -- 172.21.15.133:0/136611751 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fa3bc0019f0 con 0x7fa3cc0ce0b0 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3c2000640 1 --2- 172.21.15.133:0/136611751 >> v2:172.21.15.161:3300/0 conn(0x7fa3cc15afc0 0x7fa3cc15bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 -- 172.21.15.133:0/136611751 >> v1:172.21.15.133:6789/0 conn(0x7fa3cc0ce0b0 legacy=0x7fa3cc0ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 --2- 172.21.15.133:0/136611751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b00039b0 0x7fa3b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 --2- 172.21.15.133:0/136611751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3cc072b80 0x7fa3b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:59.106 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 --2- 172.21.15.133:0/136611751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b0004d40 0x7fa3b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 -- 172.21.15.133:0/136611751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc070880 con 0x7fa3b00039b0 2024-03-07T06:53:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 -- 172.21.15.133:0/136611751 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc15b450 con 0x7fa3b0004d40 2024-03-07T06:53:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3ca000640 1 -- 172.21.15.133:0/136611751 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b000dd30 con 0x7fa3cc072b80 2024-03-07T06:53:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.105+0000 7fa3caa00640 1 --2- 172.21.15.133:0/136611751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b00039b0 0x7fa3b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.106+0000 7fa3c2000640 1 --2- 172.21.15.133:0/136611751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3cc072b80 0x7fa3b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.106+0000 7fa3cb400640 1 --2- 172.21.15.133:0/136611751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b0004d40 0x7fa3b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.106+0000 7fa3caa00640 1 -- 172.21.15.133:0/136611751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3cc072b80 msgr2=0x7fa3b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.106+0000 7fa3caa00640 1 --2- 172.21.15.133:0/136611751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3cc072b80 0x7fa3b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.106+0000 7fa3caa00640 1 -- 172.21.15.133:0/136611751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b0004d40 msgr2=0x7fa3b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:53:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.106+0000 7fa3caa00640 1 --2- 172.21.15.133:0/136611751 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b0004d40 0x7fa3b00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.106+0000 7fa3caa00640 1 -- 172.21.15.133:0/136611751 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b0003520 con 0x7fa3b00039b0 2024-03-07T06:53:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.106+0000 7fa3c2000640 1 --2- 172.21.15.133:0/136611751 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3cc072b80 0x7fa3b0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:53:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.107+0000 7fa3caa00640 1 --2- 172.21.15.133:0/136611751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b00039b0 0x7fa3b0003dd0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00a8a0 tx=0x7fa3b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f29a658922a64d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:59.107 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.107+0000 7fa3ca000640 1 -- 172.21.15.133:0/136611751 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc00a4c0 con 0x7fa3b00039b0 2024-03-07T06:53:59.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.107+0000 7fa3ca000640 1 -- 172.21.15.133:0/136611751 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc034650 con 0x7fa3b00039b0 2024-03-07T06:53:59.108 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.107+0000 7fa3ca000640 1 -- 172.21.15.133:0/136611751 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc034a40 con 0x7fa3b00039b0 2024-03-07T06:53:59.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.108+0000 7fa3cbe00640 1 -- 172.21.15.133:0/136611751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b00039b0 msgr2=0x7fa3b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:59.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.108+0000 7fa3cbe00640 1 --2- 172.21.15.133:0/136611751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b00039b0 0x7fa3b0003dd0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00a8a0 tx=0x7fa3b0005830 comp rx=0 tx=0).stop 2024-03-07T06:53:59.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.108+0000 7fa3cb400640 1 -- 172.21.15.133:0/136611751 reap_dead start 2024-03-07T06:53:59.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.108+0000 7fa3cbe00640 1 -- 172.21.15.133:0/136611751 shutdown_connections 2024-03-07T06:53:59.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.108+0000 7fa3cbe00640 1 -- 172.21.15.133:0/136611751 >> 172.21.15.133:0/136611751 conn(0x7fa3cc0af920 msgr2=0x7fa3cc0afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:53:59.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.108+0000 7fa3cbe00640 1 -- 172.21.15.133:0/136611751 shutdown_connections 2024-03-07T06:53:59.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.108+0000 7fa3cbe00640 1 -- 172.21.15.133:0/136611751 wait complete. 2024-03-07T06:53:59.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.110+0000 7fa3cbe00640 1 Processor -- start 2024-03-07T06:53:59.109 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.110+0000 7fa3cbe00640 1 -- start start 2024-03-07T06:53:59.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b00039b0 0x7fa3cc14b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:59.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3cc147c10 0x7fa3cc14baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:59.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3cc14bfe0 0x7fa3cc0adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:59.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3cbe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc073250 con 0x7fa3b00039b0 2024-03-07T06:53:59.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3cbe00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc0730d0 con 0x7fa3cc147c10 2024-03-07T06:53:59.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3cbe00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc15c190 con 0x7fa3cc14bfe0 2024-03-07T06:53:59.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3caa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b00039b0 0x7fa3cc14b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:59.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3caa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b00039b0 0x7fa3cc14b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42514/0 (socket says 172.21.15.133:42514) 2024-03-07T06:53:59.111 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3caa00640 1 -- 172.21.15.133:0/1280653424 learned_addr learned my addr 172.21.15.133:0/1280653424 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:53:59.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3c2000640 1 --2- 172.21.15.133:0/1280653424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3cc147c10 0x7fa3cc14baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:59.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3caa00640 1 -- 172.21.15.133:0/1280653424 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3cc14bfe0 msgr2=0x7fa3cc0adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:59.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3cb400640 1 --2- 172.21.15.133:0/1280653424 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3cc14bfe0 0x7fa3cc0adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:59.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3caa00640 1 --2- 172.21.15.133:0/1280653424 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3cc14bfe0 0x7fa3cc0adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:59.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3caa00640 1 -- 172.21.15.133:0/1280653424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3cc147c10 msgr2=0x7fa3cc14baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:53:59.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3caa00640 1 --2- 172.21.15.133:0/1280653424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3cc147c10 0x7fa3cc14baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:53:59.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.111+0000 7fa3caa00640 1 -- 172.21.15.133:0/1280653424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc14b540 con 0x7fa3b00039b0 2024-03-07T06:53:59.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.112+0000 7fa3caa00640 1 --2- 172.21.15.133:0/1280653424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b00039b0 0x7fa3cc14b000 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00a9c0 tx=0x7fa3bc001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:59.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.112+0000 7fa3c3400640 1 -- 172.21.15.133:0/1280653424 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc045070 con 0x7fa3b00039b0 2024-03-07T06:53:59.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.112+0000 7fa3c3400640 1 -- 172.21.15.133:0/1280653424 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3bc03a040 con 0x7fa3b00039b0 2024-03-07T06:53:59.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.112+0000 7fa3c3400640 1 -- 172.21.15.133:0/1280653424 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc0019c0 con 0x7fa3b00039b0 2024-03-07T06:53:59.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.112+0000 7fa3cbe00640 1 -- 172.21.15.133:0/1280653424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3cc148990 con 0x7fa3b00039b0 2024-03-07T06:53:59.112 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.112+0000 7fa3cbe00640 1 -- 172.21.15.133:0/1280653424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3cc148f50 con 0x7fa3b00039b0 2024-03-07T06:53:59.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.113+0000 7fa3cbe00640 1 -- 172.21.15.133:0/1280653424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3cc062c90 con 0x7fa3b00039b0 2024-03-07T06:53:59.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.114+0000 7fa3c3400640 1 -- 172.21.15.133:0/1280653424 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa3bc034650 con 0x7fa3b00039b0 2024-03-07T06:53:59.113 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.115+0000 7fa3c3400640 1 --2- 172.21.15.133:0/1280653424 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa3a806b500 0x7fa3a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:53:59.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.115+0000 7fa3c3400640 1 -- 172.21.15.133:0/1280653424 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa3bc0abda0 con 0x7fa3b00039b0 2024-03-07T06:53:59.114 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.116+0000 7fa3c2000640 1 --2- 172.21.15.133:0/1280653424 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa3a806b500 0x7fa3a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:53:59.115 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.117+0000 7fa3c2000640 1 --2- 172.21.15.133:0/1280653424 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa3a806b500 0x7fa3a806d9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0ae9a0 tx=0x7fa3b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:53:59.117 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.119+0000 7fa3c3400640 1 -- 172.21.15.133:0/1280653424 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa3bc07aaf0 con 0x7fa3b00039b0 2024-03-07T06:53:59.315 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:53:59.316+0000 7fa3cbe00640 1 -- 172.21.15.133:0/1280653424 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa3cc0aefb0 con 0x7fa3b00039b0 2024-03-07T06:54:00.161 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:54:00.169 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:54:00.318 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.319+0000 7fa3c3400640 1 -- 172.21.15.133:0/1280653424 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v50) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa3bc07a3f0 con 0x7fa3b00039b0 2024-03-07T06:54:00.318 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2024-03-07T06:54:00.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.322+0000 7fa3cbe00640 1 -- 172.21.15.133:0/1280653424 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa3a806b500 msgr2=0x7fa3a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:00.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.322+0000 7fa3cbe00640 1 --2- 172.21.15.133:0/1280653424 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa3a806b500 0x7fa3a806d9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0ae9a0 tx=0x7fa3b4008040 comp rx=0 tx=0).stop 2024-03-07T06:54:00.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.322+0000 7fa3cbe00640 1 -- 172.21.15.133:0/1280653424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b00039b0 msgr2=0x7fa3cc14b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:00.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.322+0000 7fa3cbe00640 1 --2- 172.21.15.133:0/1280653424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b00039b0 0x7fa3cc14b000 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00a9c0 tx=0x7fa3bc001dc0 comp rx=0 tx=0).stop 2024-03-07T06:54:00.321 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.322+0000 7fa3cbe00640 1 -- 172.21.15.133:0/1280653424 shutdown_connections 2024-03-07T06:54:00.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.322+0000 7fa3cbe00640 1 --2- 172.21.15.133:0/1280653424 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3cc14bfe0 0x7fa3cc0adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:00.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.322+0000 7fa3cbe00640 1 --2- 172.21.15.133:0/1280653424 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa3a806b500 0x7fa3a806d9c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:00.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.322+0000 7fa3cbe00640 1 --2- 172.21.15.133:0/1280653424 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3cc147c10 0x7fa3cc14baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:00.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.322+0000 7fa3cbe00640 1 --2- 172.21.15.133:0/1280653424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3b00039b0 0x7fa3cc14b000 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:00.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.322+0000 7fa3cbe00640 1 -- 172.21.15.133:0/1280653424 >> 172.21.15.133:0/1280653424 conn(0x7fa3cc0af920 msgr2=0x7fa3cc1550c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:00.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.323+0000 7fa3cbe00640 1 -- 172.21.15.133:0/1280653424 shutdown_connections 2024-03-07T06:54:00.322 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.323+0000 7fa3cbe00640 1 -- 172.21.15.133:0/1280653424 wait complete. 2024-03-07T06:54:00.347 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-07T06:54:00.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.540+0000 7f23bd400640 1 Processor -- start 2024-03-07T06:54:00.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.541+0000 7f23bd400640 1 -- start start 2024-03-07T06:54:00.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.542+0000 7f23bd400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f23b80cc6f0 0x7f23b80b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:00.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.542+0000 7f23bd400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b8070820 0x7f23b8070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:00.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.542+0000 7f23bd400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b80b2190 con 0x7f23b80cdf00 2024-03-07T06:54:00.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.542+0000 7f23bd400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b80b2430 con 0x7f23b80cc6f0 2024-03-07T06:54:00.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.542+0000 7f23bd400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b80b2710 con 0x7f23b8070820 2024-03-07T06:54:00.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.543+0000 7f23b6800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f23b80cc6f0 0x7f23b80b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:00.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.543+0000 7f23b6800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f23b80cc6f0 0x7f23b80b1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.133:56244/0 (socket says 172.21.15.133:56244) 2024-03-07T06:54:00.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.543+0000 7f23b6800640 1 -- 172.21.15.133:0/3253709759 learned_addr learned my addr 172.21.15.133:0/3253709759 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:00.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.543+0000 7f23b7200640 1 --2- 172.21.15.133:0/3253709759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b8070820 0x7f23b8070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:00.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.544+0000 7f23b7200640 1 -- 172.21.15.133:0/3253709759 >> v2:172.21.15.161:3300/0 conn(0x7f23b80cc6f0 msgr2=0x7f23b80b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:00.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.544+0000 7f23b7200640 1 --2- 172.21.15.133:0/3253709759 >> v2:172.21.15.161:3300/0 conn(0x7f23b80cc6f0 0x7f23b80b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:00.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.544+0000 7f23b7200640 1 -- 172.21.15.133:0/3253709759 >> v1:172.21.15.161:6789/0 conn(0x7f23b80cdf00 legacy=0x7f23b815ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:00.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.544+0000 7f23b7200640 1 -- 172.21.15.133:0/3253709759 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b80b29f0 con 0x7f23b8070820 2024-03-07T06:54:00.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.545+0000 7f23b7200640 1 --2- 172.21.15.133:0/3253709759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b8070820 0x7f23b8070cb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009d90 tx=0x7f23b002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f1217e11d7d19d server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:00.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.546+0000 7f23b5e00640 1 -- 172.21.15.133:0/3253709759 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0039070 con 0x7f23b8070820 2024-03-07T06:54:00.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.546+0000 7f23b5e00640 1 -- 172.21.15.133:0/3253709759 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b002cc50 con 0x7f23b8070820 2024-03-07T06:54:00.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.546+0000 7f23b5e00640 1 -- 172.21.15.133:0/3253709759 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0034440 con 0x7f23b8070820 2024-03-07T06:54:00.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.547+0000 7f23bd400640 1 -- 172.21.15.133:0/3253709759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b8070820 msgr2=0x7f23b8070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:00.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.547+0000 7f23bd400640 1 --2- 172.21.15.133:0/3253709759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b8070820 0x7f23b8070cb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f23b0009d90 tx=0x7f23b002bea0 comp rx=0 tx=0).stop 2024-03-07T06:54:00.545 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.547+0000 7f23bd400640 1 -- 172.21.15.133:0/3253709759 shutdown_connections 2024-03-07T06:54:00.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.547+0000 7f23bd400640 1 --2- 172.21.15.133:0/3253709759 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b8070820 0x7f23b8070cb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:00.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.547+0000 7f23bd400640 1 --2- 172.21.15.133:0/3253709759 >> v2:172.21.15.161:3300/0 conn(0x7f23b80cc6f0 0x7f23b80b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:00.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.547+0000 7f23bd400640 1 -- 172.21.15.133:0/3253709759 >> 172.21.15.133:0/3253709759 conn(0x7f23b8066eb0 msgr2=0x7f23b8067290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:00.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.547+0000 7f23bd400640 1 -- 172.21.15.133:0/3253709759 shutdown_connections 2024-03-07T06:54:00.546 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.547+0000 7f23bd400640 1 -- 172.21.15.133:0/3253709759 wait complete. 2024-03-07T06:54:00.547 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.549+0000 7f23bd400640 1 Processor -- start 2024-03-07T06:54:00.548 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.549+0000 7f23bd400640 1 -- start start 2024-03-07T06:54:00.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.550+0000 7f23bd400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b8070820 0x7f23b81f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:00.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.550+0000 7f23bd400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b80cc6f0 0x7f23b80b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:00.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.550+0000 7f23bd400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b80cdf00 0x7f23b80b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:00.550 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.550+0000 7f23bd400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b815fa50 con 0x7f23b8070820 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.550+0000 7f23bd400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b815c460 con 0x7f23b80cc6f0 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.550+0000 7f23bd400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b815fbd0 con 0x7f23b80cdf00 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.551+0000 7f23b6800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b8070820 0x7f23b81f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.551+0000 7f23b6800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b8070820 0x7f23b81f6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42518/0 (socket says 172.21.15.133:42518) 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.551+0000 7f23b6800640 1 -- 172.21.15.133:0/959584771 learned_addr learned my addr 172.21.15.133:0/959584771 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.551+0000 7f23afe00640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b80cc6f0 0x7f23b80b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.551+0000 7f23b7200640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b80cdf00 0x7f23b80b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.551+0000 7f23b6800640 1 -- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b80cdf00 msgr2=0x7f23b80b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.551+0000 7f23b6800640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b80cdf00 0x7f23b80b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.551+0000 7f23b6800640 1 -- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b80cc6f0 msgr2=0x7f23b80b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.551+0000 7f23b6800640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b80cc6f0 0x7f23b80b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.551+0000 7f23b6800640 1 -- 172.21.15.133:0/959584771 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b80ae270 con 0x7f23b8070820 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.551+0000 7f23b7200640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b80cdf00 0x7f23b80b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.551+0000 7f23b6800640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b8070820 0x7f23b81f6d70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009fd0 tx=0x7f23a800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:00.551 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.552+0000 7f23aea00640 1 -- 172.21.15.133:0/959584771 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8017070 con 0x7f23b8070820 2024-03-07T06:54:00.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.552+0000 7f23aea00640 1 -- 172.21.15.133:0/959584771 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a800c940 con 0x7f23b8070820 2024-03-07T06:54:00.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.552+0000 7f23aea00640 1 -- 172.21.15.133:0/959584771 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a800cd10 con 0x7f23b8070820 2024-03-07T06:54:00.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.552+0000 7f23bd400640 1 -- 172.21.15.133:0/959584771 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b80ae550 con 0x7f23b8070820 2024-03-07T06:54:00.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.552+0000 7f23bd400640 1 -- 172.21.15.133:0/959584771 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b80aeb10 con 0x7f23b8070820 2024-03-07T06:54:00.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.553+0000 7f23bd400640 1 -- 172.21.15.133:0/959584771 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b8062c90 con 0x7f23b8070820 2024-03-07T06:54:00.552 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.554+0000 7f23aea00640 1 -- 172.21.15.133:0/959584771 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f23a801a070 con 0x7f23b8070820 2024-03-07T06:54:00.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.555+0000 7f23aea00640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f239406b4d0 0x7f239406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:00.553 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.555+0000 7f23aea00640 1 -- 172.21.15.133:0/959584771 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f23a808a3b0 con 0x7f23b8070820 2024-03-07T06:54:00.554 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.556+0000 7f23afe00640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f239406b4d0 0x7f239406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:00.555 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.557+0000 7f23afe00640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f239406b4d0 0x7f239406d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f23a0007fd0 tx=0x7f23a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:00.558 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.559+0000 7f23aea00640 1 -- 172.21.15.133:0/959584771 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f23a8059340 con 0x7f23b8070820 2024-03-07T06:54:00.755 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:00.756+0000 7f23bd400640 1 -- 172.21.15.133:0/959584771 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f23b80afc30 con 0x7f23b8070820 2024-03-07T06:54:01.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.329+0000 7f23aea00640 1 -- 172.21.15.133:0/959584771 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v51) v1 ==== 169+0+0 (secure 0 0 0) 0x7f23a8058c40 con 0x7f23b8070820 2024-03-07T06:54:01.328 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2024-03-07T06:54:01.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.332+0000 7f23bd400640 1 -- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f239406b4d0 msgr2=0x7f239406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:01.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.332+0000 7f23bd400640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f239406b4d0 0x7f239406d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f23a0007fd0 tx=0x7f23a0009040 comp rx=0 tx=0).stop 2024-03-07T06:54:01.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.332+0000 7f23bd400640 1 -- 172.21.15.133:0/959584771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b8070820 msgr2=0x7f23b81f6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:01.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.332+0000 7f23bd400640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b8070820 0x7f23b81f6d70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009fd0 tx=0x7f23a800ed40 comp rx=0 tx=0).stop 2024-03-07T06:54:01.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.333+0000 7f23bd400640 1 -- 172.21.15.133:0/959584771 shutdown_connections 2024-03-07T06:54:01.332 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.333+0000 7f23bd400640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b80cdf00 0x7f23b80b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.333+0000 7f23bd400640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f239406b4d0 0x7f239406d990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.333+0000 7f23bd400640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b80cc6f0 0x7f23b80b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.333+0000 7f23bd400640 1 --2- 172.21.15.133:0/959584771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23b8070820 0x7f23b81f6d70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.333+0000 7f23bd400640 1 -- 172.21.15.133:0/959584771 >> 172.21.15.133:0/959584771 conn(0x7f23b8066eb0 msgr2=0x7f23b8074360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.333+0000 7f23bd400640 1 -- 172.21.15.133:0/959584771 shutdown_connections 2024-03-07T06:54:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.333+0000 7f23bd400640 1 -- 172.21.15.133:0/959584771 wait complete. 2024-03-07T06:54:01.358 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-07T06:54:01.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.598+0000 7f2543e00640 1 Processor -- start 2024-03-07T06:54:01.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.598+0000 7f2543e00640 1 -- start start 2024-03-07T06:54:01.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.599+0000 7f2543e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f253c0aeb70 0x7f253c0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:01.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.599+0000 7f2543e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c071310 con 0x7f253c0acf40 2024-03-07T06:54:01.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.599+0000 7f2543e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c15e950 con 0x7f253c15ada0 2024-03-07T06:54:01.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.599+0000 7f2543e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253c15f840 con 0x7f253c0aeb70 2024-03-07T06:54:01.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.599+0000 7f2541e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f253c0acf40 0x7f253c0ad390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46402/0 (socket says 172.21.15.133:46402) 2024-03-07T06:54:01.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.599+0000 7f2541e00640 1 -- 172.21.15.133:0/1496592939 learned_addr learned my addr 172.21.15.133:0/1496592939 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:54:01.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f2540a00640 1 --2- 172.21.15.133:0/1496592939 >> v2:172.21.15.161:3300/0 conn(0x7f253c0aeb70 0x7f253c0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:01.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 -- 172.21.15.133:0/1496592939 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592892658 0 0) 0x7f2534001490 con 0x7f253c0acf40 2024-03-07T06:54:01.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 -- 172.21.15.133:0/1496592939 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2524003520 con 0x7f253c0acf40 2024-03-07T06:54:01.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 -- 172.21.15.133:0/1496592939 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 247123104 0 0) 0x7f2524003520 con 0x7f253c0acf40 2024-03-07T06:54:01.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 -- 172.21.15.133:0/1496592939 >> v2:172.21.15.161:3300/0 conn(0x7f253c0aeb70 msgr2=0x7f253c0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:01.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 --2- 172.21.15.133:0/1496592939 >> v2:172.21.15.161:3300/0 conn(0x7f253c0aeb70 0x7f253c0b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:01.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 -- 172.21.15.133:0/1496592939 >> v1:172.21.15.161:6789/0 conn(0x7f253c15ada0 legacy=0x7f253c0ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:01.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 -- 172.21.15.133:0/1496592939 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c15ec30 con 0x7f253c0acf40 2024-03-07T06:54:01.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 -- 172.21.15.133:0/1496592939 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f2534001bc0 con 0x7f253c0acf40 2024-03-07T06:54:01.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f2540a00640 1 --2- 172.21.15.133:0/1496592939 >> v2:172.21.15.161:3300/0 conn(0x7f253c0aeb70 0x7f253c0b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-07T06:54:01.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 -- 172.21.15.133:0/1496592939 >> v1:172.21.15.133:6789/0 conn(0x7f253c0acf40 legacy=0x7f253c0ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:01.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 --2- 172.21.15.133:0/1496592939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25240039b0 0x7f2524003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:01.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 --2- 172.21.15.133:0/1496592939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2524004800 0x7f2524004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:01.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 --2- 172.21.15.133:0/1496592939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25240051c0 0x7f252400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:01.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 -- 172.21.15.133:0/1496592939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c15f840 con 0x7f25240039b0 2024-03-07T06:54:01.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 -- 172.21.15.133:0/1496592939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252400c110 con 0x7f2524004800 2024-03-07T06:54:01.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.600+0000 7f253be00640 1 -- 172.21.15.133:0/1496592939 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252400c310 con 0x7f25240051c0 2024-03-07T06:54:01.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.601+0000 7f2541400640 1 --2- 172.21.15.133:0/1496592939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25240039b0 0x7f2524003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:01.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.601+0000 7f2541400640 1 -- 172.21.15.133:0/1496592939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25240051c0 msgr2=0x7f252400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:01.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.601+0000 7f2541400640 1 --2- 172.21.15.133:0/1496592939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25240051c0 0x7f252400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:01.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.601+0000 7f2541400640 1 -- 172.21.15.133:0/1496592939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2524004800 msgr2=0x7f2524004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:01.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.601+0000 7f2541400640 1 --2- 172.21.15.133:0/1496592939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2524004800 0x7f2524004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:01.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.601+0000 7f2541400640 1 -- 172.21.15.133:0/1496592939 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2524003520 con 0x7f25240039b0 2024-03-07T06:54:01.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.601+0000 7f2541e00640 1 -- 172.21.15.133:0/1496592939 reap_dead start 2024-03-07T06:54:01.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.602+0000 7f2541400640 1 --2- 172.21.15.133:0/1496592939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25240039b0 0x7f2524003dd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2530001750 tx=0x7f2524005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186a23965ee4a7b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:01.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.602+0000 7f253be00640 1 -- 172.21.15.133:0/1496592939 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530001370 con 0x7f25240039b0 2024-03-07T06:54:01.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.602+0000 7f253be00640 1 -- 172.21.15.133:0/1496592939 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25300052d0 con 0x7f25240039b0 2024-03-07T06:54:01.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.602+0000 7f253be00640 1 -- 172.21.15.133:0/1496592939 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530003830 con 0x7f25240039b0 2024-03-07T06:54:01.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.603+0000 7f2543e00640 1 -- 172.21.15.133:0/1496592939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25240039b0 msgr2=0x7f2524003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:01.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.603+0000 7f2543e00640 1 --2- 172.21.15.133:0/1496592939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25240039b0 0x7f2524003dd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f2530001750 tx=0x7f2524005830 comp rx=0 tx=0).stop 2024-03-07T06:54:01.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.603+0000 7f2543e00640 1 -- 172.21.15.133:0/1496592939 shutdown_connections 2024-03-07T06:54:01.610 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.603+0000 7f2543e00640 1 --2- 172.21.15.133:0/1496592939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25240039b0 0x7f2524003dd0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:01.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.603+0000 7f2543e00640 1 -- 172.21.15.133:0/1496592939 >> 172.21.15.133:0/1496592939 conn(0x7f253c0cfb00 msgr2=0x7f253c0cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:01.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.603+0000 7f2543e00640 1 -- 172.21.15.133:0/1496592939 shutdown_connections 2024-03-07T06:54:01.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.603+0000 7f2543e00640 1 -- 172.21.15.133:0/1496592939 wait complete. 2024-03-07T06:54:01.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.605+0000 7f2543e00640 1 Processor -- start 2024-03-07T06:54:01.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.605+0000 7f2543e00640 1 -- start start 2024-03-07T06:54:01.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2543e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25240039b0 0x7f253c155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:01.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2543e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f253c0b3dd0 0x7f253c0b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:01.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2543e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f253c09ac30 0x7f253c09b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:01.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2543e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c15f4d0 con 0x7f253c09ac30 2024-03-07T06:54:01.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2543e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c15f1f0 con 0x7f25240039b0 2024-03-07T06:54:01.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2543e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c15b180 con 0x7f253c0b3dd0 2024-03-07T06:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2541e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f253c09ac30 0x7f253c09b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2541e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f253c09ac30 0x7f253c09b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50090/0 (socket says 172.21.15.133:50090) 2024-03-07T06:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2541e00640 1 -- 172.21.15.133:0/3438316190 learned_addr learned my addr 172.21.15.133:0/3438316190 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2541400640 1 --2- 172.21.15.133:0/3438316190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25240039b0 0x7f253c155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2541e00640 1 -- 172.21.15.133:0/3438316190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f253c0b3dd0 msgr2=0x7f253c0b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2541e00640 1 --2- 172.21.15.133:0/3438316190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f253c0b3dd0 0x7f253c0b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2541e00640 1 -- 172.21.15.133:0/3438316190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25240039b0 msgr2=0x7f253c155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2541e00640 1 --2- 172.21.15.133:0/3438316190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25240039b0 0x7f253c155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.606+0000 7f2541e00640 1 -- 172.21.15.133:0/3438316190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c09b8f0 con 0x7f253c09ac30 2024-03-07T06:54:01.612 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.607+0000 7f2541e00640 1 --2- 172.21.15.133:0/3438316190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f253c09ac30 0x7f253c09b0b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2534000f80 tx=0x7f253400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.607+0000 7f2539600640 1 -- 172.21.15.133:0/3438316190 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253400ce70 con 0x7f253c09ac30 2024-03-07T06:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.607+0000 7f2539600640 1 -- 172.21.15.133:0/3438316190 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253400a610 con 0x7f253c09ac30 2024-03-07T06:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.607+0000 7f2539600640 1 -- 172.21.15.133:0/3438316190 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253400aa00 con 0x7f253c09ac30 2024-03-07T06:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.607+0000 7f2543e00640 1 -- 172.21.15.133:0/3438316190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c09bbd0 con 0x7f253c09ac30 2024-03-07T06:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.607+0000 7f2543e00640 1 -- 172.21.15.133:0/3438316190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c09c190 con 0x7f253c09ac30 2024-03-07T06:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.608+0000 7f2543e00640 1 -- 172.21.15.133:0/3438316190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c062c90 con 0x7f253c09ac30 2024-03-07T06:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.609+0000 7f2539600640 1 -- 172.21.15.133:0/3438316190 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f253400aba0 con 0x7f253c09ac30 2024-03-07T06:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.610+0000 7f2539600640 1 --2- 172.21.15.133:0/3438316190 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f251c06b440 0x7f251c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.610+0000 7f2539600640 1 -- 172.21.15.133:0/3438316190 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2534089400 con 0x7f253c09ac30 2024-03-07T06:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.610+0000 7f2541400640 1 --2- 172.21.15.133:0/3438316190 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f251c06b440 0x7f251c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:01.613 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.612+0000 7f2541400640 1 --2- 172.21.15.133:0/3438316190 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f251c06b440 0x7f251c06d900 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2530033000 tx=0x7f2530009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:01.614 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.614+0000 7f2539600640 1 -- 172.21.15.133:0/3438316190 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f253401a030 con 0x7f253c09ac30 2024-03-07T06:54:01.807 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.808+0000 7f2543e00640 1 -- 172.21.15.133:0/3438316190 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f253c072960 con 0x7f253c09ac30 2024-03-07T06:54:01.812 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.813+0000 7f2539600640 1 -- 172.21.15.133:0/3438316190 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) v1 ==== 104+0+0 (secure 0 0 0) 0x7f253c071310 con 0x7f253c09ac30 2024-03-07T06:54:01.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.816+0000 7f2543e00640 1 -- 172.21.15.133:0/3438316190 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f251c06b440 msgr2=0x7f251c06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.816+0000 7f2543e00640 1 --2- 172.21.15.133:0/3438316190 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f251c06b440 0x7f251c06d900 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2530033000 tx=0x7f2530009030 comp rx=0 tx=0).stop 2024-03-07T06:54:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.816+0000 7f2543e00640 1 -- 172.21.15.133:0/3438316190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f253c09ac30 msgr2=0x7f253c09b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.816+0000 7f2543e00640 1 --2- 172.21.15.133:0/3438316190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f253c09ac30 0x7f253c09b0b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2534000f80 tx=0x7f253400cc20 comp rx=0 tx=0).stop 2024-03-07T06:54:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.816+0000 7f2543e00640 1 -- 172.21.15.133:0/3438316190 shutdown_connections 2024-03-07T06:54:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.816+0000 7f2543e00640 1 --2- 172.21.15.133:0/3438316190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f253c09ac30 0x7f253c09b0b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.817+0000 7f2543e00640 1 --2- 172.21.15.133:0/3438316190 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f253c0b3dd0 0x7f253c0b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.817+0000 7f2543e00640 1 --2- 172.21.15.133:0/3438316190 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f251c06b440 0x7f251c06d900 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.817+0000 7f2543e00640 1 --2- 172.21.15.133:0/3438316190 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25240039b0 0x7f253c155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.817+0000 7f2543e00640 1 -- 172.21.15.133:0/3438316190 >> 172.21.15.133:0/3438316190 conn(0x7f253c0cfb00 msgr2=0x7f253c06e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.817+0000 7f2543e00640 1 -- 172.21.15.133:0/3438316190 shutdown_connections 2024-03-07T06:54:01.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:01.817+0000 7f2543e00640 1 -- 172.21.15.133:0/3438316190 wait complete. 2024-03-07T06:54:01.834 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_bal_fragment_size_max 2024-03-07T06:54:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.032+0000 7faa13e00640 1 Processor -- start 2024-03-07T06:54:02.031 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.032+0000 7faa13e00640 1 -- start start 2024-03-07T06:54:02.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.034+0000 7faa13e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7faa1415ca90 0x7faa1415ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:02.032 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.034+0000 7faa13e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa140d24f0 0x7faa14141580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:02.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.034+0000 7faa13e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faa140d2930 0x7faa140b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:02.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.034+0000 7faa13e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1413f270 con 0x7faa140d2930 2024-03-07T06:54:02.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.034+0000 7faa13e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1415d3b0 con 0x7faa1415ca90 2024-03-07T06:54:02.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.034+0000 7faa13e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa141424b0 con 0x7faa140d24f0 2024-03-07T06:54:02.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.034+0000 7faa13400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faa140d2930 0x7faa140b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:02.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.034+0000 7faa13400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7faa140d2930 0x7faa140b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50104/0 (socket says 172.21.15.133:50104) 2024-03-07T06:54:02.033 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.034+0000 7faa13400640 1 -- 172.21.15.133:0/1797669711 learned_addr learned my addr 172.21.15.133:0/1797669711 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:02.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.034+0000 7faa13400640 1 -- 172.21.15.133:0/1797669711 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa140d24f0 msgr2=0x7faa14141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:02.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.034+0000 7faa13400640 1 --2- 172.21.15.133:0/1797669711 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa140d24f0 0x7faa14141580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.034+0000 7faa13400640 1 -- 172.21.15.133:0/1797669711 >> v2:172.21.15.161:3300/0 conn(0x7faa1415ca90 msgr2=0x7faa1415ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:02.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.034+0000 7faa13400640 1 --2- 172.21.15.133:0/1797669711 >> v2:172.21.15.161:3300/0 conn(0x7faa1415ca90 0x7faa1415ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.034+0000 7faa13400640 1 -- 172.21.15.133:0/1797669711 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa14141ef0 con 0x7faa140d2930 2024-03-07T06:54:02.034 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.036+0000 7faa13400640 1 --2- 172.21.15.133:0/1797669711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa140d2930 0x7faa140b3aa0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7faa140b46f0 tx=0x7faa0c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edcb3915bb0c476 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.036+0000 7faa11600640 1 -- 172.21.15.133:0/1797669711 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c02e070 con 0x7faa140d2930 2024-03-07T06:54:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.036+0000 7faa11600640 1 -- 172.21.15.133:0/1797669711 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c037070 con 0x7faa140d2930 2024-03-07T06:54:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.036+0000 7faa11600640 1 -- 172.21.15.133:0/1797669711 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c031a00 con 0x7faa140d2930 2024-03-07T06:54:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.036+0000 7faa13e00640 1 -- 172.21.15.133:0/1797669711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa140d2930 msgr2=0x7faa140b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.036+0000 7faa13e00640 1 --2- 172.21.15.133:0/1797669711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa140d2930 0x7faa140b3aa0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7faa140b46f0 tx=0x7faa0c02b880 comp rx=0 tx=0).stop 2024-03-07T06:54:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.037+0000 7faa13e00640 1 -- 172.21.15.133:0/1797669711 shutdown_connections 2024-03-07T06:54:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.037+0000 7faa13e00640 1 --2- 172.21.15.133:0/1797669711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa140d2930 0x7faa140b3aa0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.037+0000 7faa13e00640 1 --2- 172.21.15.133:0/1797669711 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa140d24f0 0x7faa14141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.037+0000 7faa13e00640 1 --2- 172.21.15.133:0/1797669711 >> v2:172.21.15.161:3300/0 conn(0x7faa1415ca90 0x7faa1415ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.037+0000 7faa13e00640 1 -- 172.21.15.133:0/1797669711 >> 172.21.15.133:0/1797669711 conn(0x7faa14070400 msgr2=0x7faa140707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.037+0000 7faa13e00640 1 -- 172.21.15.133:0/1797669711 shutdown_connections 2024-03-07T06:54:02.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.037+0000 7faa13e00640 1 -- 172.21.15.133:0/1797669711 wait complete. 2024-03-07T06:54:02.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.038+0000 7faa13e00640 1 Processor -- start 2024-03-07T06:54:02.037 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.039+0000 7faa13e00640 1 -- start start 2024-03-07T06:54:02.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.039+0000 7faa13e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa140d24f0 0x7faa140ad840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:02.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.039+0000 7faa13e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa140d2930 0x7faa1415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:02.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.039+0000 7faa13e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa1415ca90 0x7faa1415e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:02.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.039+0000 7faa13e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa140d1fd0 con 0x7faa140d24f0 2024-03-07T06:54:02.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.039+0000 7faa13e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa140d1cf0 con 0x7faa1415ca90 2024-03-07T06:54:02.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.039+0000 7faa13e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa14141ac0 con 0x7faa140d2930 2024-03-07T06:54:02.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.040+0000 7faa12a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa140d24f0 0x7faa140ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:02.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.040+0000 7faa12a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa140d24f0 0x7faa140ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50108/0 (socket says 172.21.15.133:50108) 2024-03-07T06:54:02.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.040+0000 7faa12a00640 1 -- 172.21.15.133:0/3059738880 learned_addr learned my addr 172.21.15.133:0/3059738880 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:02.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.040+0000 7faa13400640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa1415ca90 0x7faa1415e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:02.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.040+0000 7faa12a00640 1 -- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa140d2930 msgr2=0x7faa1415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:02.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.040+0000 7faa12000640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa140d2930 0x7faa1415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:02.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.040+0000 7faa12a00640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa140d2930 0x7faa1415b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.040+0000 7faa12a00640 1 -- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa1415ca90 msgr2=0x7faa1415e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:02.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.040+0000 7faa12a00640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa1415ca90 0x7faa1415e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.040+0000 7faa12a00640 1 -- 172.21.15.133:0/3059738880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1413f490 con 0x7faa140d24f0 2024-03-07T06:54:02.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.040+0000 7faa13400640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa1415ca90 0x7faa1415e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:02.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.040+0000 7faa12a00640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa140d24f0 0x7faa140ad840 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7faa0400b4a0 tx=0x7faa0400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:02.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.041+0000 7faa0aa00640 1 -- 172.21.15.133:0/3059738880 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04014070 con 0x7faa140d24f0 2024-03-07T06:54:02.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.041+0000 7faa0aa00640 1 -- 172.21.15.133:0/3059738880 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa04010610 con 0x7faa140d24f0 2024-03-07T06:54:02.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.041+0000 7faa0aa00640 1 -- 172.21.15.133:0/3059738880 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04010a00 con 0x7faa140d24f0 2024-03-07T06:54:02.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.041+0000 7faa13e00640 1 -- 172.21.15.133:0/3059738880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa14159300 con 0x7faa140d24f0 2024-03-07T06:54:02.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.041+0000 7faa13e00640 1 -- 172.21.15.133:0/3059738880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1413f700 con 0x7faa140d24f0 2024-03-07T06:54:02.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.042+0000 7faa13e00640 1 -- 172.21.15.133:0/3059738880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1413f900 con 0x7faa140d24f0 2024-03-07T06:54:02.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.043+0000 7faa0aa00640 1 -- 172.21.15.133:0/3059738880 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7faa04017070 con 0x7faa140d24f0 2024-03-07T06:54:02.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.044+0000 7faa0aa00640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa9f006b4d0 0x7fa9f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:02.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.044+0000 7faa0aa00640 1 -- 172.21.15.133:0/3059738880 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7faa040899c0 con 0x7faa140d24f0 2024-03-07T06:54:02.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.045+0000 7faa12000640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa9f006b4d0 0x7fa9f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:02.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.046+0000 7faa12000640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa9f006b4d0 0x7fa9f006d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faa140b0ab0 tx=0x7fa9fc0072f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:02.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.049+0000 7faa0aa00640 1 -- 172.21.15.133:0/3059738880 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7faa1413f900 con 0x7faa140d24f0 2024-03-07T06:54:02.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.245+0000 7faa13e00640 1 -- 172.21.15.133:0/3059738880 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_bal_fragment_size_max"} v 0) v1 -- 0x7faa14062c90 con 0x7faa140d24f0 2024-03-07T06:54:02.249 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.250+0000 7faa0aa00640 1 -- 172.21.15.133:0/3059738880 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_bal_fragment_size_max"}]=0 v15) v1 ==== 108+0+0 (secure 0 0 0) 0x7faa04058aa0 con 0x7faa140d24f0 2024-03-07T06:54:02.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.253+0000 7faa13e00640 1 -- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa9f006b4d0 msgr2=0x7fa9f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:02.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.253+0000 7faa13e00640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa9f006b4d0 0x7fa9f006d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faa140b0ab0 tx=0x7fa9fc0072f0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.253+0000 7faa13e00640 1 -- 172.21.15.133:0/3059738880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa140d24f0 msgr2=0x7faa140ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:02.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.253+0000 7faa13e00640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa140d24f0 0x7faa140ad840 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7faa0400b4a0 tx=0x7faa0400b960 comp rx=0 tx=0).stop 2024-03-07T06:54:02.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.253+0000 7faa13e00640 1 -- 172.21.15.133:0/3059738880 shutdown_connections 2024-03-07T06:54:02.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.253+0000 7faa13e00640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa9f006b4d0 0x7fa9f006d990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.254+0000 7faa13e00640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7faa1415ca90 0x7faa1415e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.254+0000 7faa13e00640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7faa140d2930 0x7faa1415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.254+0000 7faa13e00640 1 --2- 172.21.15.133:0/3059738880 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa140d24f0 0x7faa140ad840 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.254+0000 7faa13e00640 1 -- 172.21.15.133:0/3059738880 >> 172.21.15.133:0/3059738880 conn(0x7faa14070400 msgr2=0x7faa140b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:02.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.254+0000 7faa13e00640 1 -- 172.21.15.133:0/3059738880 shutdown_connections 2024-03-07T06:54:02.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.254+0000 7faa13e00640 1 -- 172.21.15.133:0/3059738880 wait complete. 2024-03-07T06:54:02.271 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_strays.TestStrays.test_dirfrag_limit' 2024-03-07T06:54:02.482 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.484+0000 7f7fa9400640 1 Processor -- start 2024-03-07T06:54:02.483 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.484+0000 7f7fa9400640 1 -- start start 2024-03-07T06:54:02.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.485+0000 7f7fa9400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fa40cf5d0 0x7f7fa4072830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:02.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.485+0000 7f7fa9400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7fa4072d70 0x7f7fa415c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:02.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.485+0000 7f7fa9400640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa4160760 con 0x7f7fa415bd30 2024-03-07T06:54:02.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.485+0000 7f7fa9400640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa40731e0 con 0x7f7fa4072d70 2024-03-07T06:54:02.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.485+0000 7f7fa9400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa415f9a0 con 0x7f7fa40cf5d0 2024-03-07T06:54:02.484 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.486+0000 7f7fa8a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7fa4072d70 0x7f7fa415c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:02.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.486+0000 7f7fa3e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f7fa415bd30 0x7f7fa406e8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46416/0 (socket says 172.21.15.133:46416) 2024-03-07T06:54:02.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.486+0000 7f7fa3e00640 1 -- 172.21.15.133:0/2140172832 learned_addr learned my addr 172.21.15.133:0/2140172832 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:54:02.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.486+0000 7f7f9be00640 1 --2- 172.21.15.133:0/2140172832 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fa40cf5d0 0x7f7fa4072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:02.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.486+0000 7f7fa8a00640 1 -- 172.21.15.133:0/2140172832 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fa40cf5d0 msgr2=0x7f7fa4072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:02.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.486+0000 7f7fa8a00640 1 --2- 172.21.15.133:0/2140172832 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fa40cf5d0 0x7f7fa4072830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.485 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.486+0000 7f7fa8a00640 1 -- 172.21.15.133:0/2140172832 >> v1:172.21.15.133:6789/0 conn(0x7f7fa415bd30 legacy=0x7f7fa406e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:02.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.486+0000 7f7fa8a00640 1 -- 172.21.15.133:0/2140172832 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa415fc80 con 0x7f7fa4072d70 2024-03-07T06:54:02.486 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.486+0000 7f7fa3400640 1 -- 172.21.15.133:0/2140172832 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526781890 0 0) 0x7f7fa4160760 con 0x7f7fa415bd30 2024-03-07T06:54:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.487+0000 7f7fa8a00640 1 --2- 172.21.15.133:0/2140172832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa4072d70 0x7f7fa415c110 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009d90 tx=0x7f7f9c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20124bc8a23d7c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.488+0000 7f7fa3400640 1 -- 172.21.15.133:0/2140172832 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c039070 con 0x7f7fa4072d70 2024-03-07T06:54:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.488+0000 7f7fa3400640 1 -- 172.21.15.133:0/2140172832 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9c02cc50 con 0x7f7fa4072d70 2024-03-07T06:54:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.488+0000 7f7fa3400640 1 -- 172.21.15.133:0/2140172832 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c034440 con 0x7f7fa4072d70 2024-03-07T06:54:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.488+0000 7f7fa9400640 1 -- 172.21.15.133:0/2140172832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa4072d70 msgr2=0x7f7fa415c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.488+0000 7f7fa9400640 1 --2- 172.21.15.133:0/2140172832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa4072d70 0x7f7fa415c110 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009d90 tx=0x7f7f9c02bea0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.488+0000 7f7fa9400640 1 -- 172.21.15.133:0/2140172832 shutdown_connections 2024-03-07T06:54:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.488+0000 7f7fa9400640 1 --2- 172.21.15.133:0/2140172832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa4072d70 0x7f7fa415c110 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.488+0000 7f7fa9400640 1 --2- 172.21.15.133:0/2140172832 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fa40cf5d0 0x7f7fa4072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.488+0000 7f7fa9400640 1 -- 172.21.15.133:0/2140172832 >> 172.21.15.133:0/2140172832 conn(0x7f7fa40b2e90 msgr2=0x7f7fa40b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.488+0000 7f7fa9400640 1 -- 172.21.15.133:0/2140172832 shutdown_connections 2024-03-07T06:54:02.487 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.488+0000 7f7fa9400640 1 -- 172.21.15.133:0/2140172832 wait complete. 2024-03-07T06:54:02.488 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.490+0000 7f7fa9400640 1 Processor -- start 2024-03-07T06:54:02.489 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.490+0000 7f7fa9400640 1 -- start start 2024-03-07T06:54:02.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.491+0000 7f7fa9400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fa4072d70 0x7f7fa4148060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:02.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.491+0000 7f7fa9400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fa40cf5d0 0x7f7fa41477a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:02.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.491+0000 7f7fa9400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa415bd30 0x7f7fa415c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:02.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.491+0000 7f7fa9400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa406f130 con 0x7f7fa415bd30 2024-03-07T06:54:02.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.491+0000 7f7fa9400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa4160240 con 0x7f7fa40cf5d0 2024-03-07T06:54:02.490 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.491+0000 7f7fa9400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa415c650 con 0x7f7fa4072d70 2024-03-07T06:54:02.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.491+0000 7f7fa8a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa415bd30 0x7f7fa415c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:02.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.491+0000 7f7fa8a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa415bd30 0x7f7fa415c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50124/0 (socket says 172.21.15.133:50124) 2024-03-07T06:54:02.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.491+0000 7f7fa8a00640 1 -- 172.21.15.133:0/3646303131 learned_addr learned my addr 172.21.15.133:0/3646303131 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:02.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.492+0000 7f7fa8a00640 1 -- 172.21.15.133:0/3646303131 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fa4072d70 msgr2=0x7f7fa4148060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:02.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.492+0000 7f7fa8a00640 1 --2- 172.21.15.133:0/3646303131 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fa4072d70 0x7f7fa4148060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.492+0000 7f7fa8a00640 1 -- 172.21.15.133:0/3646303131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fa40cf5d0 msgr2=0x7f7fa41477a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:02.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.492+0000 7f7fa8a00640 1 --2- 172.21.15.133:0/3646303131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fa40cf5d0 0x7f7fa41477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:02.491 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.492+0000 7f7fa8a00640 1 -- 172.21.15.133:0/3646303131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa40ccb20 con 0x7f7fa415bd30 2024-03-07T06:54:02.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.492+0000 7f7fa8a00640 1 --2- 172.21.15.133:0/3646303131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa415bd30 0x7f7fa415c9d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009d90 tx=0x7f7f9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:02.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.492+0000 7f7fa0c00640 1 -- 172.21.15.133:0/3646303131 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c044020 con 0x7f7fa415bd30 2024-03-07T06:54:02.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.492+0000 7f7fa0c00640 1 -- 172.21.15.133:0/3646303131 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9c03a040 con 0x7f7fa415bd30 2024-03-07T06:54:02.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.492+0000 7f7fa0c00640 1 -- 172.21.15.133:0/3646303131 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c039050 con 0x7f7fa415bd30 2024-03-07T06:54:02.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.492+0000 7f7fa9400640 1 -- 172.21.15.133:0/3646303131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa40cce00 con 0x7f7fa415bd30 2024-03-07T06:54:02.492 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.493+0000 7f7fa9400640 1 -- 172.21.15.133:0/3646303131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa40cd3c0 con 0x7f7fa415bd30 2024-03-07T06:54:02.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.494+0000 7f7fa9400640 1 -- 172.21.15.133:0/3646303131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa4062c90 con 0x7f7fa415bd30 2024-03-07T06:54:02.493 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.495+0000 7f7fa0c00640 1 -- 172.21.15.133:0/3646303131 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7f9c003990 con 0x7f7fa415bd30 2024-03-07T06:54:02.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.496+0000 7f7fa0c00640 1 --2- 172.21.15.133:0/3646303131 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7f8006b500 0x7f7f8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:02.494 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.496+0000 7f7fa0c00640 1 -- 172.21.15.133:0/3646303131 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7f9c0ac390 con 0x7f7fa415bd30 2024-03-07T06:54:02.495 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.497+0000 7f7fa3e00640 1 --2- 172.21.15.133:0/3646303131 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7f8006b500 0x7f7f8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:02.496 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.498+0000 7f7fa3e00640 1 --2- 172.21.15.133:0/3646303131 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7f8006b500 0x7f7f8006d9c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7f94007fd0 tx=0x7f7f94007960 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:02.499 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.501+0000 7f7fa0c00640 1 -- 172.21.15.133:0/3646303131 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7f9c04e060 con 0x7f7fa415bd30 2024-03-07T06:54:02.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:02.768+0000 7f7fa9400640 1 -- 172.21.15.133:0/3646303131 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_strays.TestStrays.test_dirfrag_limit"]} v 0) v1 -- 0x7f7fa40734a0 con 0x7f7fa415bd30 2024-03-07T06:54:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:03.344+0000 7f7fa0c00640 1 -- 172.21.15.133:0/3646303131 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_strays.TestStrays.test_dirfrag_limit"]}]=0 v171) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7f9c07b500 con 0x7f7fa415bd30 2024-03-07T06:54:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:03.347+0000 7f7fa9400640 1 -- 172.21.15.133:0/3646303131 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7f8006b500 msgr2=0x7f7f8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:03.347+0000 7f7fa9400640 1 --2- 172.21.15.133:0/3646303131 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7f8006b500 0x7f7f8006d9c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7f94007fd0 tx=0x7f7f94007960 comp rx=0 tx=0).stop 2024-03-07T06:54:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:03.347+0000 7f7fa9400640 1 -- 172.21.15.133:0/3646303131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa415bd30 msgr2=0x7f7fa415c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:03.346 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:03.347+0000 7f7fa9400640 1 --2- 172.21.15.133:0/3646303131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa415bd30 0x7f7fa415c9d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009d90 tx=0x7f7f9c004a40 comp rx=0 tx=0).stop 2024-03-07T06:54:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:03.347+0000 7f7fa9400640 1 -- 172.21.15.133:0/3646303131 shutdown_connections 2024-03-07T06:54:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:03.347+0000 7f7fa9400640 1 --2- 172.21.15.133:0/3646303131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fa415bd30 0x7f7fa415c9d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:03.347+0000 7f7fa9400640 1 --2- 172.21.15.133:0/3646303131 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7f8006b500 0x7f7f8006d9c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:03.347+0000 7f7fa9400640 1 --2- 172.21.15.133:0/3646303131 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7fa40cf5d0 0x7f7fa41477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:03.347+0000 7f7fa9400640 1 --2- 172.21.15.133:0/3646303131 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7fa4072d70 0x7f7fa4148060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:03.347 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:03.347+0000 7f7fa9400640 1 -- 172.21.15.133:0/3646303131 >> 172.21.15.133:0/3646303131 conn(0x7f7fa40b2e90 msgr2=0x7f7fa406cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:03.348+0000 7f7fa9400640 1 -- 172.21.15.133:0/3646303131 shutdown_connections 2024-03-07T06:54:03.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:03.348+0000 7f7fa9400640 1 -- 172.21.15.133:0/3646303131 wait complete. 2024-03-07T06:54:03.368 INFO:tasks.cephfs_test_runner:That the directory fragment size cannot exceed mds_bal_fragment_size_max (using a limit of 50 in all configurations). ... FAIL 2024-03-07T06:54:03.368 INFO:tasks.cephfs_test_runner: 2024-03-07T06:54:03.368 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-07T06:54:03.368 INFO:tasks.cephfs_test_runner:FAIL: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) 2024-03-07T06:54:03.368 INFO:tasks.cephfs_test_runner:That the directory fragment size cannot exceed mds_bal_fragment_size_max (using a limit of 50 in all configurations). 2024-03-07T06:54:03.368 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-07T06:54:03.369 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-07T06:54:03.369 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/cephfs/test_strays.py", line 932, in test_dirfrag_limit 2024-03-07T06:54:03.369 INFO:tasks.cephfs_test_runner: self.fail("fragment size exceeded") 2024-03-07T06:54:03.369 INFO:tasks.cephfs_test_runner:AssertionError: fragment size exceeded 2024-03-07T06:54:03.369 INFO:tasks.cephfs_test_runner: 2024-03-07T06:54:03.369 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-07T06:54:03.369 INFO:tasks.cephfs_test_runner:Ran 2 tests in 125.498s 2024-03-07T06:54:03.369 INFO:tasks.cephfs_test_runner: 2024-03-07T06:54:03.369 INFO:tasks.cephfs_test_runner:FAILED (failures=1) 2024-03-07T06:54:03.369 INFO:tasks.cephfs_test_runner: 2024-03-07T06:54:03.369 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-07T06:54:03.369 INFO:tasks.cephfs_test_runner:FAIL: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) 2024-03-07T06:54:03.369 INFO:tasks.cephfs_test_runner:That the directory fragment size cannot exceed mds_bal_fragment_size_max (using a limit of 50 in all configurations). 2024-03-07T06:54:03.369 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-07T06:54:03.370 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-07T06:54:03.370 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/cephfs/test_strays.py", line 932, in test_dirfrag_limit 2024-03-07T06:54:03.370 INFO:tasks.cephfs_test_runner: self.fail("fragment size exceeded") 2024-03-07T06:54:03.370 INFO:tasks.cephfs_test_runner:AssertionError: fragment size exceeded 2024-03-07T06:54:03.370 INFO:tasks.cephfs_test_runner: 2024-03-07T06:54:03.370 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) 2024-03-07T06:54:03.634 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=466416b0f1624b5eaeac4b2301dc182b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_8b896a9e145796119f0451201dc4d53ddac97db0/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays) 2024-03-07T06:54:03.636 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-07T06:54:03.645 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-07T06:54:03.656 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-07T06:54:03.656 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:54:03.656 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:54:03.715 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053417,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-07T06:54:03.715 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-03-07T06:54:03.715 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:54:03.763 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053417,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-07T06:54:03.763 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:54:03.763 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:54:03.820 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053429,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-07T06:54:03.820 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2024-03-07T06:54:03.820 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-07T06:54:03.871 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053429,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249180k,nr_inodes=812295,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498364k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-07T06:54:03.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:54:03.872 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-03-07T06:54:03.928 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-07T06:54:03.929 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-07T06:54:03.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:54:03.930 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-07T06:54:04.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:54:04.025 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-07T06:54:04.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:54:04.116 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-03-07T06:54:04.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:54:04.258 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-03-07T06:54:04.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:54:04.304 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-03-07T06:54:04.444 INFO:teuthology.orchestra.run.smithi133.stderr:Cannot find device "ceph-brx" 2024-03-07T06:54:04.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:54:04.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:54:04.447 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-03-07T06:54:04.505 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-07T06:54:04.505 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-07T06:54:04.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:54:04.506 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-07T06:54:04.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:54:04.609 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-07T06:54:04.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:54:04.698 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2024-03-07T06:54:04.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:54:04.814 DEBUG:teuthology.orchestra.run.smithi161:> ip netns list 2024-03-07T06:54:04.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-07T06:54:04.862 DEBUG:teuthology.orchestra.run.smithi161:> sudo ip link delete ceph-brx 2024-03-07T06:54:05.006 INFO:teuthology.orchestra.run.smithi161.stderr:Cannot find device "ceph-brx" 2024-03-07T06:54:05.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:54:05.013 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-07T06:54:05.025 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-07T06:54:05.026 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:54:05.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.252+0000 7f82dc600640 1 Processor -- start 2024-03-07T06:54:05.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.252+0000 7f82dc600640 1 -- start start 2024-03-07T06:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.253+0000 7f82dc600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f82d415b1b0 0x7f82d40b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.253+0000 7f82dc600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d406f140 con 0x7f82d40ce5f0 2024-03-07T06:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.253+0000 7f82dc600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d406f420 con 0x7f82d40af950 2024-03-07T06:54:05.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.253+0000 7f82dc600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d4071610 con 0x7f82d415b1b0 2024-03-07T06:54:05.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.254+0000 7f82d9c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f82d415b1b0 0x7f82d40b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:05.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.254+0000 7f82da600640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f82d40ce5f0 0x7f82d4071d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46432/0 (socket says 172.21.15.133:46432) 2024-03-07T06:54:05.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.254+0000 7f82da600640 1 -- 172.21.15.133:0/248225965 learned_addr learned my addr 172.21.15.133:0/248225965 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:54:05.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.254+0000 7f82d9c00640 1 -- 172.21.15.133:0/248225965 >> v1:172.21.15.161:6789/0 conn(0x7f82d40af950 legacy=0x7f82d40cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:05.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.254+0000 7f82d9c00640 1 -- 172.21.15.133:0/248225965 >> v1:172.21.15.133:6789/0 conn(0x7f82d40ce5f0 legacy=0x7f82d4071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:05.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.254+0000 7f82d9c00640 1 -- 172.21.15.133:0/248225965 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d4071860 con 0x7f82d415b1b0 2024-03-07T06:54:05.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.254+0000 7f82d3e00640 1 -- 172.21.15.133:0/248225965 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 946070808 0 0) 0x7f82d406f140 con 0x7f82d40ce5f0 2024-03-07T06:54:05.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.255+0000 7f82d9c00640 1 --2- 172.21.15.133:0/248225965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d415b1b0 0x7f82d40b3c10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f82c80097a0 tx=0x7f82c802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d0d8c84550070b server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:05.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.256+0000 7f82d3e00640 1 -- 172.21.15.133:0/248225965 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c80315a0 con 0x7f82d415b1b0 2024-03-07T06:54:05.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.256+0000 7f82d3e00640 1 -- 172.21.15.133:0/248225965 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c8037070 con 0x7f82d415b1b0 2024-03-07T06:54:05.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.256+0000 7f82d3e00640 1 -- 172.21.15.133:0/248225965 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c8031960 con 0x7f82d415b1b0 2024-03-07T06:54:05.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.256+0000 7f82dc600640 1 -- 172.21.15.133:0/248225965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d415b1b0 msgr2=0x7f82d40b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:05.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.256+0000 7f82dc600640 1 --2- 172.21.15.133:0/248225965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d415b1b0 0x7f82d40b3c10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f82c80097a0 tx=0x7f82c802b840 comp rx=0 tx=0).stop 2024-03-07T06:54:05.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.256+0000 7f82dc600640 1 -- 172.21.15.133:0/248225965 shutdown_connections 2024-03-07T06:54:05.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.256+0000 7f82dc600640 1 --2- 172.21.15.133:0/248225965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d415b1b0 0x7f82d40b3c10 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.256+0000 7f82dc600640 1 -- 172.21.15.133:0/248225965 >> 172.21.15.133:0/248225965 conn(0x7f82d40b3120 msgr2=0x7f82d40b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:05.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.256+0000 7f82dc600640 1 -- 172.21.15.133:0/248225965 shutdown_connections 2024-03-07T06:54:05.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.256+0000 7f82dc600640 1 -- 172.21.15.133:0/248225965 wait complete. 2024-03-07T06:54:05.256 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.258+0000 7f82dc600640 1 Processor -- start 2024-03-07T06:54:05.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.258+0000 7f82dc600640 1 -- start start 2024-03-07T06:54:05.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.259+0000 7f82dc600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82d40af950 0x7f82d415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:05.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.259+0000 7f82dc600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82d40ce5f0 0x7f82d40b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:05.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.259+0000 7f82dc600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d415b1b0 0x7f82d40add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:05.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.259+0000 7f82dc600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d415b590 con 0x7f82d415b1b0 2024-03-07T06:54:05.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.259+0000 7f82dc600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d406ee30 con 0x7f82d40af950 2024-03-07T06:54:05.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.259+0000 7f82dc600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d415b710 con 0x7f82d40ce5f0 2024-03-07T06:54:05.258 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.259+0000 7f82da600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d415b1b0 0x7f82d40add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:05.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.259+0000 7f82da600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d415b1b0 0x7f82d40add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50144/0 (socket says 172.21.15.133:50144) 2024-03-07T06:54:05.259 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.259+0000 7f82da600640 1 -- 172.21.15.133:0/1037173258 learned_addr learned my addr 172.21.15.133:0/1037173258 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:05.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.260+0000 7f82da600640 1 -- 172.21.15.133:0/1037173258 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82d40ce5f0 msgr2=0x7f82d40b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:05.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.260+0000 7f82da600640 1 --2- 172.21.15.133:0/1037173258 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82d40ce5f0 0x7f82d40b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.260+0000 7f82da600640 1 -- 172.21.15.133:0/1037173258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82d40af950 msgr2=0x7f82d415b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:05.260 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.260+0000 7f82da600640 1 --2- 172.21.15.133:0/1037173258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82d40af950 0x7f82d415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.260+0000 7f82da600640 1 -- 172.21.15.133:0/1037173258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d415c170 con 0x7f82d415b1b0 2024-03-07T06:54:05.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.260+0000 7f82da600640 1 --2- 172.21.15.133:0/1037173258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d415b1b0 0x7f82d40add50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00c7a0 tx=0x7f82cc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:05.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.260+0000 7f82d1600640 1 -- 172.21.15.133:0/1037173258 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc018070 con 0x7f82d415b1b0 2024-03-07T06:54:05.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.260+0000 7f82d1600640 1 -- 172.21.15.133:0/1037173258 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc00a640 con 0x7f82d415b1b0 2024-03-07T06:54:05.261 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.260+0000 7f82d1600640 1 -- 172.21.15.133:0/1037173258 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc00aab0 con 0x7f82d415b1b0 2024-03-07T06:54:05.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.260+0000 7f82dc600640 1 -- 172.21.15.133:0/1037173258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d415c450 con 0x7f82d415b1b0 2024-03-07T06:54:05.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.261+0000 7f82dc600640 1 -- 172.21.15.133:0/1037173258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d415ca10 con 0x7f82d415b1b0 2024-03-07T06:54:05.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.262+0000 7f82dc600640 1 -- 172.21.15.133:0/1037173258 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d4062c90 con 0x7f82d415b1b0 2024-03-07T06:54:05.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.262+0000 7f82d1600640 1 -- 172.21.15.133:0/1037173258 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f82cc00ac50 con 0x7f82d415b1b0 2024-03-07T06:54:05.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.263+0000 7f82d1600640 1 --2- 172.21.15.133:0/1037173258 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f82b406b4d0 0x7f82b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:05.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.263+0000 7f82d1600640 1 -- 172.21.15.133:0/1037173258 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f82cc089630 con 0x7f82d415b1b0 2024-03-07T06:54:05.263 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.264+0000 7f82d9c00640 1 --2- 172.21.15.133:0/1037173258 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f82b406b4d0 0x7f82b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:05.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.265+0000 7f82d9c00640 1 --2- 172.21.15.133:0/1037173258 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f82b406b4d0 0x7f82b406d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f82c8036000 tx=0x7f82c8030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:05.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.268+0000 7f82d1600640 1 -- 172.21.15.133:0/1037173258 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f82cc058710 con 0x7f82d415b1b0 2024-03-07T06:54:05.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.461+0000 7f82dc600640 1 -- 172.21.15.133:0/1037173258 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f82d40cc660 con 0x7f82b406b4d0 2024-03-07T06:54:05.462 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.463+0000 7f82d1600640 1 -- 172.21.15.133:0/1037173258 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13490 (secure 0 0 0) 0x7f82d40cc660 con 0x7f82b406b4d0 2024-03-07T06:54:05.462 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:54:05.463 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:54:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.466+0000 7f82dc600640 1 -- 172.21.15.133:0/1037173258 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f82b406b4d0 msgr2=0x7f82b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.466+0000 7f82dc600640 1 --2- 172.21.15.133:0/1037173258 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f82b406b4d0 0x7f82b406d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f82c8036000 tx=0x7f82c8030000 comp rx=0 tx=0).stop 2024-03-07T06:54:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.466+0000 7f82dc600640 1 -- 172.21.15.133:0/1037173258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d415b1b0 msgr2=0x7f82d40add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.466+0000 7f82dc600640 1 --2- 172.21.15.133:0/1037173258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d415b1b0 0x7f82d40add50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00c7a0 tx=0x7f82cc00cc60 comp rx=0 tx=0).stop 2024-03-07T06:54:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.466+0000 7f82dc600640 1 -- 172.21.15.133:0/1037173258 shutdown_connections 2024-03-07T06:54:05.465 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.466+0000 7f82dc600640 1 --2- 172.21.15.133:0/1037173258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82d415b1b0 0x7f82d40add50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.466+0000 7f82dc600640 1 --2- 172.21.15.133:0/1037173258 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82d40ce5f0 0x7f82d40b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.466+0000 7f82dc600640 1 --2- 172.21.15.133:0/1037173258 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f82b406b4d0 0x7f82b406d990 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.466+0000 7f82dc600640 1 --2- 172.21.15.133:0/1037173258 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82d40af950 0x7f82d415b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.466+0000 7f82dc600640 1 -- 172.21.15.133:0/1037173258 >> 172.21.15.133:0/1037173258 conn(0x7f82d40b3120 msgr2=0x7f82d4071df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.466+0000 7f82dc600640 1 -- 172.21.15.133:0/1037173258 shutdown_connections 2024-03-07T06:54:05.466 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.466+0000 7f82dc600640 1 -- 172.21.15.133:0/1037173258 wait complete. 2024-03-07T06:54:05.485 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-03-07T06:54:04.531782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":255376,"kb_used_data":5928,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719344,"statfs":{"total":773094113280,"available":772832608256,"internally_reserved":0,"allocated":6070272,"data_stored":3177544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-346680,"num_objects":-72,"num_object_clones":0,"num_object_copies":-144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-72,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-108,"num_write_kb":-410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.670663"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-07T06:54:01.852422+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:54:01.852422+0000","last_peered":"2024-03-07T06:54:01.852422+0000","last_clean":"2024-03-07T06:54:01.852422+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:54:01.852422+0000","last_undegraded":"2024-03-07T06:54:01.852422+0000","last_fullsized":"2024-03-07T06:54:01.852422+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":839969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":839969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:54:05.485 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:54:05.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.699+0000 7f159f600640 1 Processor -- start 2024-03-07T06:54:05.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.699+0000 7f159f600640 1 -- start start 2024-03-07T06:54:05.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.701+0000 7f159f600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1598147790 0x7f15980b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.701+0000 7f159f600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15980b4520 con 0x7f15980b55c0 2024-03-07T06:54:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.701+0000 7f159f600640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15980b4800 con 0x7f159815e0d0 2024-03-07T06:54:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.701+0000 7f159f600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15980b5f60 con 0x7f1598147790 2024-03-07T06:54:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.701+0000 7f1597e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f15980b55c0 0x7f1598147080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46446/0 (socket says 172.21.15.133:46446) 2024-03-07T06:54:05.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.701+0000 7f1597e00640 1 -- 172.21.15.133:0/131020862 learned_addr learned my addr 172.21.15.133:0/131020862 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:54:05.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f159d600640 1 --2- 172.21.15.133:0/131020862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1598147790 0x7f15980b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:05.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 -- 172.21.15.133:0/131020862 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 621615734 0 0) 0x7f15980b4520 con 0x7f15980b55c0 2024-03-07T06:54:05.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 -- 172.21.15.133:0/131020862 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f157c003520 con 0x7f15980b55c0 2024-03-07T06:54:05.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 -- 172.21.15.133:0/131020862 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3991056262 0 0) 0x7f157c003520 con 0x7f15980b55c0 2024-03-07T06:54:05.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 -- 172.21.15.133:0/131020862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1598147790 msgr2=0x7f15980b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:05.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 --2- 172.21.15.133:0/131020862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1598147790 0x7f15980b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 -- 172.21.15.133:0/131020862 >> v1:172.21.15.161:6789/0 conn(0x7f159815e0d0 legacy=0x7f15981417d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:05.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 -- 172.21.15.133:0/131020862 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15980b4ae0 con 0x7f15980b55c0 2024-03-07T06:54:05.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 -- 172.21.15.133:0/131020862 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7f1580001670 con 0x7f15980b55c0 2024-03-07T06:54:05.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 -- 172.21.15.133:0/131020862 >> v1:172.21.15.133:6789/0 conn(0x7f15980b55c0 legacy=0x7f1598147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:05.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 --2- 172.21.15.133:0/131020862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c0039e0 0x7f157c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:05.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 --2- 172.21.15.133:0/131020862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f157c004800 0x7f157c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:05.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 --2- 172.21.15.133:0/131020862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1598147790 0x7f157c005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 -- 172.21.15.133:0/131020862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15980b5f60 con 0x7f157c004800 2024-03-07T06:54:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 -- 172.21.15.133:0/131020862 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c00bbb0 con 0x7f157c0039e0 2024-03-07T06:54:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597400640 1 -- 172.21.15.133:0/131020862 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c00bdb0 con 0x7f1598147790 2024-03-07T06:54:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.702+0000 7f1597e00640 1 --2- 172.21.15.133:0/131020862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f157c004800 0x7f157c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.703+0000 7f159d600640 1 --2- 172.21.15.133:0/131020862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1598147790 0x7f157c005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.703+0000 7f159cc00640 1 --2- 172.21.15.133:0/131020862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c0039e0 0x7f157c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.703+0000 7f1597e00640 1 -- 172.21.15.133:0/131020862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1598147790 msgr2=0x7f157c005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:05.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.703+0000 7f1597e00640 1 --2- 172.21.15.133:0/131020862 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1598147790 0x7f157c005290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.703+0000 7f1597e00640 1 -- 172.21.15.133:0/131020862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c0039e0 msgr2=0x7f157c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:05.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.703+0000 7f1597e00640 1 --2- 172.21.15.133:0/131020862 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c0039e0 0x7f157c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.703+0000 7f1597e00640 1 -- 172.21.15.133:0/131020862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c003520 con 0x7f157c004800 2024-03-07T06:54:05.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.704+0000 7f1597e00640 1 --2- 172.21.15.133:0/131020862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f157c004800 0x7f157c004c80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f158000c210 tx=0x7f157c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2d19c24f3a1261 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:05.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.704+0000 7f1597400640 1 -- 172.21.15.133:0/131020862 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580007cf0 con 0x7f157c004800 2024-03-07T06:54:05.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.704+0000 7f1597400640 1 -- 172.21.15.133:0/131020862 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1580030070 con 0x7f157c004800 2024-03-07T06:54:05.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.704+0000 7f1597400640 1 -- 172.21.15.133:0/131020862 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580005d30 con 0x7f157c004800 2024-03-07T06:54:05.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.705+0000 7f159f600640 1 -- 172.21.15.133:0/131020862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f157c004800 msgr2=0x7f157c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:05.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.705+0000 7f159f600640 1 --2- 172.21.15.133:0/131020862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f157c004800 0x7f157c004c80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f158000c210 tx=0x7f157c005830 comp rx=0 tx=0).stop 2024-03-07T06:54:05.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.705+0000 7f159d600640 1 -- 172.21.15.133:0/131020862 reap_dead start 2024-03-07T06:54:05.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.705+0000 7f159f600640 1 -- 172.21.15.133:0/131020862 shutdown_connections 2024-03-07T06:54:05.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.705+0000 7f159f600640 1 -- 172.21.15.133:0/131020862 >> 172.21.15.133:0/131020862 conn(0x7f1598071300 msgr2=0x7f1598071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:05.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.705+0000 7f159f600640 1 -- 172.21.15.133:0/131020862 shutdown_connections 2024-03-07T06:54:05.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.705+0000 7f159f600640 1 -- 172.21.15.133:0/131020862 wait complete. 2024-03-07T06:54:05.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.707+0000 7f159f600640 1 Processor -- start 2024-03-07T06:54:05.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.707+0000 7f159f600640 1 -- start start 2024-03-07T06:54:05.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.708+0000 7f159f600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c004800 0x7f15981f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:05.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.708+0000 7f159f600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f159812c1a0 0x7f159812c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:05.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.708+0000 7f159f600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f159812e3e0 0x7f159812e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:05.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.708+0000 7f159f600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1598141ee0 con 0x7f159812e3e0 2024-03-07T06:54:05.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.708+0000 7f159f600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15980b50a0 con 0x7f157c004800 2024-03-07T06:54:05.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.708+0000 7f159f600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159815e4b0 con 0x7f159812c1a0 2024-03-07T06:54:05.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.708+0000 7f159d600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f159812e3e0 0x7f159812e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:05.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.708+0000 7f159d600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f159812e3e0 0x7f159812e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50164/0 (socket says 172.21.15.133:50164) 2024-03-07T06:54:05.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.708+0000 7f159d600640 1 -- 172.21.15.133:0/2646472392 learned_addr learned my addr 172.21.15.133:0/2646472392 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:05.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.708+0000 7f159d600640 1 -- 172.21.15.133:0/2646472392 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f159812c1a0 msgr2=0x7f159812c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:05.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.708+0000 7f159d600640 1 --2- 172.21.15.133:0/2646472392 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f159812c1a0 0x7f159812c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.708+0000 7f159d600640 1 -- 172.21.15.133:0/2646472392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c004800 msgr2=0x7f15981f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:05.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.708+0000 7f159d600640 1 --2- 172.21.15.133:0/2646472392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c004800 0x7f15981f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.708+0000 7f159d600640 1 -- 172.21.15.133:0/2646472392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159812f150 con 0x7f159812e3e0 2024-03-07T06:54:05.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.709+0000 7f159d600640 1 --2- 172.21.15.133:0/2646472392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f159812e3e0 0x7f159812e860 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f15980b6520 tx=0x7f1590011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:05.711 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.709+0000 7f1594c00640 1 -- 172.21.15.133:0/2646472392 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590003610 con 0x7f159812e3e0 2024-03-07T06:54:05.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.709+0000 7f1594c00640 1 -- 172.21.15.133:0/2646472392 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159001b070 con 0x7f159812e3e0 2024-03-07T06:54:05.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.709+0000 7f1594c00640 1 -- 172.21.15.133:0/2646472392 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1590003a10 con 0x7f159812e3e0 2024-03-07T06:54:05.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.709+0000 7f159f600640 1 -- 172.21.15.133:0/2646472392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159812f430 con 0x7f159812e3e0 2024-03-07T06:54:05.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.709+0000 7f159f600640 1 -- 172.21.15.133:0/2646472392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159812f9f0 con 0x7f159812e3e0 2024-03-07T06:54:05.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.711+0000 7f159f600640 1 -- 172.21.15.133:0/2646472392 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1598062c90 con 0x7f159812e3e0 2024-03-07T06:54:05.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.711+0000 7f1594c00640 1 -- 172.21.15.133:0/2646472392 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1590003bb0 con 0x7f159812e3e0 2024-03-07T06:54:05.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.712+0000 7f1594c00640 1 --2- 172.21.15.133:0/2646472392 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f157406b4d0 0x7f157406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:05.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.712+0000 7f1594c00640 1 -- 172.21.15.133:0/2646472392 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1590089d30 con 0x7f159812e3e0 2024-03-07T06:54:05.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.713+0000 7f159cc00640 1 --2- 172.21.15.133:0/2646472392 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f157406b4d0 0x7f157406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:05.713 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.714+0000 7f159cc00640 1 --2- 172.21.15.133:0/2646472392 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f157406b4d0 0x7f157406d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1588007fd0 tx=0x7f1588009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:05.715 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.716+0000 7f1594c00640 1 -- 172.21.15.133:0/2646472392 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f1590024020 con 0x7f159812e3e0 2024-03-07T06:54:05.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.909+0000 7f159f600640 1 -- 172.21.15.133:0/2646472392 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1598161a20 con 0x7f157406b4d0 2024-03-07T06:54:05.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.912+0000 7f1594c00640 1 -- 172.21.15.133:0/2646472392 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13490 (secure 0 0 0) 0x7f1598161a20 con 0x7f157406b4d0 2024-03-07T06:54:05.911 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:54:05.911 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:54:05.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.914+0000 7f159f600640 1 -- 172.21.15.133:0/2646472392 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f157406b4d0 msgr2=0x7f157406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:05.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.914+0000 7f159f600640 1 --2- 172.21.15.133:0/2646472392 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f157406b4d0 0x7f157406d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1588007fd0 tx=0x7f1588009040 comp rx=0 tx=0).stop 2024-03-07T06:54:05.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.914+0000 7f159f600640 1 -- 172.21.15.133:0/2646472392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f159812e3e0 msgr2=0x7f159812e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:05.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.914+0000 7f159f600640 1 --2- 172.21.15.133:0/2646472392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f159812e3e0 0x7f159812e860 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f15980b6520 tx=0x7f1590011920 comp rx=0 tx=0).stop 2024-03-07T06:54:05.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.915+0000 7f159f600640 1 -- 172.21.15.133:0/2646472392 shutdown_connections 2024-03-07T06:54:05.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.915+0000 7f159f600640 1 --2- 172.21.15.133:0/2646472392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f159812e3e0 0x7f159812e860 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.915+0000 7f159f600640 1 --2- 172.21.15.133:0/2646472392 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f159812c1a0 0x7f159812c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.915+0000 7f159f600640 1 --2- 172.21.15.133:0/2646472392 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f157406b4d0 0x7f157406d990 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.915+0000 7f159f600640 1 --2- 172.21.15.133:0/2646472392 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f157c004800 0x7f15981f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:05.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.915+0000 7f159f600640 1 -- 172.21.15.133:0/2646472392 >> 172.21.15.133:0/2646472392 conn(0x7f1598071300 msgr2=0x7f15981606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:05.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.915+0000 7f159f600640 1 -- 172.21.15.133:0/2646472392 shutdown_connections 2024-03-07T06:54:05.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:05.915+0000 7f159f600640 1 -- 172.21.15.133:0/2646472392 wait complete. 2024-03-07T06:54:05.931 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-03-07T06:54:04.531782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":255376,"kb_used_data":5928,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719344,"statfs":{"total":773094113280,"available":772832608256,"internally_reserved":0,"allocated":6070272,"data_stored":3177544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-346680,"num_objects":-72,"num_object_clones":0,"num_object_copies":-144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-72,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-108,"num_write_kb":-410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.670663"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-07T06:54:01.852422+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:54:01.852422+0000","last_peered":"2024-03-07T06:54:01.852422+0000","last_clean":"2024-03-07T06:54:01.852422+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:54:01.852422+0000","last_undegraded":"2024-03-07T06:54:01.852422+0000","last_fullsized":"2024-03-07T06:54:01.852422+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":839969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":839969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:54:05.932 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-07T06:54:05.932 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:54:06.143 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.145+0000 7f8479c00640 1 Processor -- start 2024-03-07T06:54:06.144 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.145+0000 7f8479c00640 1 -- start start 2024-03-07T06:54:06.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.146+0000 7f8479c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8474140ec0 0x7f84741416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:06.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.146+0000 7f8479c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8474161c90 0x7f8474162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:06.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.146+0000 7f8479c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847415f420 con 0x7f847415e6a0 2024-03-07T06:54:06.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.146+0000 7f8479c00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f847415f200 con 0x7f8474161c90 2024-03-07T06:54:06.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.146+0000 7f8479c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84740b4590 con 0x7f8474140ec0 2024-03-07T06:54:06.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.147+0000 7f8472000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8474161c90 0x7f8474162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:06.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.147+0000 7f8472000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8474161c90 0x7f8474162110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50174/0 (socket says 172.21.15.133:50174) 2024-03-07T06:54:06.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.147+0000 7f8472000640 1 -- 172.21.15.133:0/1562736321 learned_addr learned my addr 172.21.15.133:0/1562736321 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:06.146 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.147+0000 7f8472a00640 1 --2- 172.21.15.133:0/1562736321 >> v2:172.21.15.161:3300/0 conn(0x7f8474140ec0 0x7f84741416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.147+0000 7f8472000640 1 -- 172.21.15.133:0/1562736321 >> v2:172.21.15.161:3300/0 conn(0x7f8474140ec0 msgr2=0x7f84741416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.147+0000 7f8472000640 1 --2- 172.21.15.133:0/1562736321 >> v2:172.21.15.161:3300/0 conn(0x7f8474140ec0 0x7f84741416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.147+0000 7f8472000640 1 -- 172.21.15.133:0/1562736321 >> v1:172.21.15.161:6789/0 conn(0x7f847415e6a0 legacy=0x7f847415eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.147+0000 7f8472000640 1 -- 172.21.15.133:0/1562736321 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84740b4870 con 0x7f8474161c90 2024-03-07T06:54:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.149+0000 7f8472000640 1 --2- 172.21.15.133:0/1562736321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8474161c90 0x7f8474162110 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f84740d40b0 tx=0x7f845c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc19d688385b182 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.149+0000 7f8471600640 1 -- 172.21.15.133:0/1562736321 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c035020 con 0x7f8474161c90 2024-03-07T06:54:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.149+0000 7f8471600640 1 -- 172.21.15.133:0/1562736321 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845c031870 con 0x7f8474161c90 2024-03-07T06:54:06.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.149+0000 7f8471600640 1 -- 172.21.15.133:0/1562736321 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845c031c40 con 0x7f8474161c90 2024-03-07T06:54:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.150+0000 7f8479c00640 1 -- 172.21.15.133:0/1562736321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8474161c90 msgr2=0x7f8474162110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.150+0000 7f8479c00640 1 --2- 172.21.15.133:0/1562736321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8474161c90 0x7f8474162110 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f84740d40b0 tx=0x7f845c02b910 comp rx=0 tx=0).stop 2024-03-07T06:54:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.150+0000 7f8479c00640 1 -- 172.21.15.133:0/1562736321 shutdown_connections 2024-03-07T06:54:06.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.150+0000 7f8479c00640 1 --2- 172.21.15.133:0/1562736321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8474161c90 0x7f8474162110 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.150+0000 7f8479c00640 1 --2- 172.21.15.133:0/1562736321 >> v2:172.21.15.161:3300/0 conn(0x7f8474140ec0 0x7f84741416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.150+0000 7f8479c00640 1 -- 172.21.15.133:0/1562736321 >> 172.21.15.133:0/1562736321 conn(0x7f8474071340 msgr2=0x7f8474071720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.150+0000 7f8479c00640 1 -- 172.21.15.133:0/1562736321 shutdown_connections 2024-03-07T06:54:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.150+0000 7f8479c00640 1 -- 172.21.15.133:0/1562736321 wait complete. 2024-03-07T06:54:06.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.152+0000 7f8479c00640 1 Processor -- start 2024-03-07T06:54:06.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.152+0000 7f8479c00640 1 -- start start 2024-03-07T06:54:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.153+0000 7f8479c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8474140ec0 0x7f84740b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.153+0000 7f8479c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f847415e6a0 0x7f84740d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.153+0000 7f8479c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8474161c90 0x7f84740d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:06.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.153+0000 7f8479c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8474141420 con 0x7f8474140ec0 2024-03-07T06:54:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.153+0000 7f8479c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84741412a0 con 0x7f8474161c90 2024-03-07T06:54:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.153+0000 7f8479c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84740b5070 con 0x7f847415e6a0 2024-03-07T06:54:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.154+0000 7f8472a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8474140ec0 0x7f84740b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.154+0000 7f8472a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8474140ec0 0x7f84740b6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50186/0 (socket says 172.21.15.133:50186) 2024-03-07T06:54:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.154+0000 7f8472a00640 1 -- 172.21.15.133:0/1721109229 learned_addr learned my addr 172.21.15.133:0/1721109229 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.154+0000 7f8472a00640 1 -- 172.21.15.133:0/1721109229 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f847415e6a0 msgr2=0x7f84740d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.154+0000 7f8472a00640 1 --2- 172.21.15.133:0/1721109229 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f847415e6a0 0x7f84740d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.154+0000 7f8472a00640 1 -- 172.21.15.133:0/1721109229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8474161c90 msgr2=0x7f84740d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:06.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.154+0000 7f8472a00640 1 --2- 172.21.15.133:0/1721109229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8474161c90 0x7f84740d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.154+0000 7f8472a00640 1 -- 172.21.15.133:0/1721109229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84740b6760 con 0x7f8474140ec0 2024-03-07T06:54:06.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.154+0000 7f8472a00640 1 --2- 172.21.15.133:0/1721109229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8474140ec0 0x7f84740b6220 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f846400d800 tx=0x7f846400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:06.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.154+0000 7f846aa00640 1 -- 172.21.15.133:0/1721109229 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846400b6c0 con 0x7f8474140ec0 2024-03-07T06:54:06.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.155+0000 7f846aa00640 1 -- 172.21.15.133:0/1721109229 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8464019070 con 0x7f8474140ec0 2024-03-07T06:54:06.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.155+0000 7f846aa00640 1 -- 172.21.15.133:0/1721109229 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846400ba80 con 0x7f8474140ec0 2024-03-07T06:54:06.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.155+0000 7f8479c00640 1 -- 172.21.15.133:0/1721109229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84740d11b0 con 0x7f8474140ec0 2024-03-07T06:54:06.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.155+0000 7f8479c00640 1 -- 172.21.15.133:0/1721109229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847414cbe0 con 0x7f8474140ec0 2024-03-07T06:54:06.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.156+0000 7f8479c00640 1 -- 172.21.15.133:0/1721109229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8474062c90 con 0x7f8474140ec0 2024-03-07T06:54:06.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.157+0000 7f846aa00640 1 -- 172.21.15.133:0/1721109229 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f846400bc20 con 0x7f8474140ec0 2024-03-07T06:54:06.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.158+0000 7f846aa00640 1 --2- 172.21.15.133:0/1721109229 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f845006b4d0 0x7f845006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:06.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.158+0000 7f846aa00640 1 -- 172.21.15.133:0/1721109229 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f846408a450 con 0x7f8474140ec0 2024-03-07T06:54:06.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.158+0000 7f8472000640 1 --2- 172.21.15.133:0/1721109229 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f845006b4d0 0x7f845006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:06.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.159+0000 7f8472000640 1 --2- 172.21.15.133:0/1721109229 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f845006b4d0 0x7f845006d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f845c036000 tx=0x7f845c030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:06.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.162+0000 7f846aa00640 1 -- 172.21.15.133:0/1721109229 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f84740d11b0 con 0x7f8474140ec0 2024-03-07T06:54:06.353 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.355+0000 7f8479c00640 1 -- 172.21.15.133:0/1721109229 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84740b6000 con 0x7f845006b4d0 2024-03-07T06:54:06.355 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.357+0000 7f846aa00640 1 -- 172.21.15.133:0/1721109229 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13490 (secure 0 0 0) 0x7f84740b6000 con 0x7f845006b4d0 2024-03-07T06:54:06.356 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:54:06.356 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:54:06.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.360+0000 7f8479c00640 1 -- 172.21.15.133:0/1721109229 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f845006b4d0 msgr2=0x7f845006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:06.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.360+0000 7f8479c00640 1 --2- 172.21.15.133:0/1721109229 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f845006b4d0 0x7f845006d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f845c036000 tx=0x7f845c030000 comp rx=0 tx=0).stop 2024-03-07T06:54:06.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.360+0000 7f8479c00640 1 -- 172.21.15.133:0/1721109229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8474140ec0 msgr2=0x7f84740b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:06.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.360+0000 7f8479c00640 1 --2- 172.21.15.133:0/1721109229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8474140ec0 0x7f84740b6220 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f846400d800 tx=0x7f846400dcc0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.359 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.360+0000 7f8479c00640 1 -- 172.21.15.133:0/1721109229 shutdown_connections 2024-03-07T06:54:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.360+0000 7f8479c00640 1 --2- 172.21.15.133:0/1721109229 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f845006b4d0 0x7f845006d990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.360+0000 7f8479c00640 1 --2- 172.21.15.133:0/1721109229 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8474161c90 0x7f84740d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.360+0000 7f8479c00640 1 --2- 172.21.15.133:0/1721109229 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f847415e6a0 0x7f84740d3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.360+0000 7f8479c00640 1 --2- 172.21.15.133:0/1721109229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8474140ec0 0x7f84740b6220 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.360+0000 7f8479c00640 1 -- 172.21.15.133:0/1721109229 >> 172.21.15.133:0/1721109229 conn(0x7f8474071340 msgr2=0x7f847415d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.360+0000 7f8479c00640 1 -- 172.21.15.133:0/1721109229 shutdown_connections 2024-03-07T06:54:06.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.360+0000 7f8479c00640 1 -- 172.21.15.133:0/1721109229 wait complete. 2024-03-07T06:54:06.378 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-03-07T06:54:04.531782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":255376,"kb_used_data":5928,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719344,"statfs":{"total":773094113280,"available":772832608256,"internally_reserved":0,"allocated":6070272,"data_stored":3177544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-346680,"num_objects":-72,"num_object_clones":0,"num_object_copies":-144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-72,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-108,"num_write_kb":-410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.670663"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-07T06:54:01.852422+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:54:01.852422+0000","last_peered":"2024-03-07T06:54:01.852422+0000","last_clean":"2024-03-07T06:54:01.852422+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:54:01.852422+0000","last_undegraded":"2024-03-07T06:54:01.852422+0000","last_fullsized":"2024-03-07T06:54:01.852422+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":839969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":839969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:54:06.379 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-07T06:54:06.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.584+0000 7ffb45200640 1 Processor -- start 2024-03-07T06:54:06.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.584+0000 7ffb45200640 1 -- start start 2024-03-07T06:54:06.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.586+0000 7ffb45200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb4015fbb0 0x7ffb400cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:06.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.586+0000 7ffb45200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ffb40072950 0x7ffb400cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:06.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.586+0000 7ffb45200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb400731b0 con 0x7ffb4015ff90 2024-03-07T06:54:06.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.586+0000 7ffb45200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb40072f60 con 0x7ffb4015fbb0 2024-03-07T06:54:06.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.586+0000 7ffb45200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb40073490 con 0x7ffb40072950 2024-03-07T06:54:06.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.586+0000 7ffb3de00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb4015fbb0 0x7ffb400cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:06.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.586+0000 7ffb3de00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ffb4015fbb0 0x7ffb400cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50196/0 (socket says 172.21.15.133:50196) 2024-03-07T06:54:06.585 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.586+0000 7ffb3de00640 1 -- 172.21.15.133:0/4078360462 learned_addr learned my addr 172.21.15.133:0/4078360462 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:06.586 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.587+0000 7ffb3d400640 1 --2- 172.21.15.133:0/4078360462 >> v2:172.21.15.161:3300/0 conn(0x7ffb40072950 0x7ffb400cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:06.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.587+0000 7ffb3de00640 1 -- 172.21.15.133:0/4078360462 >> v2:172.21.15.161:3300/0 conn(0x7ffb40072950 msgr2=0x7ffb400cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:06.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.587+0000 7ffb3de00640 1 --2- 172.21.15.133:0/4078360462 >> v2:172.21.15.161:3300/0 conn(0x7ffb40072950 0x7ffb400cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.587+0000 7ffb3de00640 1 -- 172.21.15.133:0/4078360462 >> v1:172.21.15.161:6789/0 conn(0x7ffb4015ff90 legacy=0x7ffb400acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:06.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.587+0000 7ffb3de00640 1 -- 172.21.15.133:0/4078360462 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb40072150 con 0x7ffb4015fbb0 2024-03-07T06:54:06.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.588+0000 7ffb3de00640 1 --2- 172.21.15.133:0/4078360462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb4015fbb0 0x7ffb400cf7d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ffb400cdd10 tx=0x7ffb3002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750a1dd902346c06 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:06.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.588+0000 7ffb3ca00640 1 -- 172.21.15.133:0/4078360462 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3002e070 con 0x7ffb4015fbb0 2024-03-07T06:54:06.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.588+0000 7ffb3ca00640 1 -- 172.21.15.133:0/4078360462 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb30037070 con 0x7ffb4015fbb0 2024-03-07T06:54:06.587 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.588+0000 7ffb3ca00640 1 -- 172.21.15.133:0/4078360462 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30031a00 con 0x7ffb4015fbb0 2024-03-07T06:54:06.588 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.589+0000 7ffb45200640 1 -- 172.21.15.133:0/4078360462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb4015fbb0 msgr2=0x7ffb400cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:06.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.589+0000 7ffb45200640 1 --2- 172.21.15.133:0/4078360462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb4015fbb0 0x7ffb400cf7d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ffb400cdd10 tx=0x7ffb3002b880 comp rx=0 tx=0).stop 2024-03-07T06:54:06.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.589+0000 7ffb45200640 1 -- 172.21.15.133:0/4078360462 shutdown_connections 2024-03-07T06:54:06.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.589+0000 7ffb45200640 1 --2- 172.21.15.133:0/4078360462 >> v2:172.21.15.161:3300/0 conn(0x7ffb40072950 0x7ffb400cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.589+0000 7ffb45200640 1 --2- 172.21.15.133:0/4078360462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb4015fbb0 0x7ffb400cf7d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.589+0000 7ffb45200640 1 -- 172.21.15.133:0/4078360462 >> 172.21.15.133:0/4078360462 conn(0x7ffb400b3120 msgr2=0x7ffb400b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:06.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.589+0000 7ffb45200640 1 -- 172.21.15.133:0/4078360462 shutdown_connections 2024-03-07T06:54:06.589 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.589+0000 7ffb45200640 1 -- 172.21.15.133:0/4078360462 wait complete. 2024-03-07T06:54:06.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.591+0000 7ffb45200640 1 Processor -- start 2024-03-07T06:54:06.590 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.591+0000 7ffb45200640 1 -- start start 2024-03-07T06:54:06.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.592+0000 7ffb45200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb4015fbb0 0x7ffb400c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:06.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.592+0000 7ffb45200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb4015ff90 0x7ffb400c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:06.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.592+0000 7ffb45200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb4015af30 0x7ffb400c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:06.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.592+0000 7ffb45200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb400b1a00 con 0x7ffb4015ff90 2024-03-07T06:54:06.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.592+0000 7ffb45200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40160980 con 0x7ffb4015fbb0 2024-03-07T06:54:06.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.592+0000 7ffb45200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb401603b0 con 0x7ffb4015af30 2024-03-07T06:54:06.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.593+0000 7ffb3d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb4015ff90 0x7ffb400c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:06.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.593+0000 7ffb3d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb4015ff90 0x7ffb400c7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50212/0 (socket says 172.21.15.133:50212) 2024-03-07T06:54:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.593+0000 7ffb3d400640 1 -- 172.21.15.133:0/1348098565 learned_addr learned my addr 172.21.15.133:0/1348098565 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.593+0000 7ffb3d400640 1 -- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb4015af30 msgr2=0x7ffb400c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.593+0000 7ffb3e800640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb4015af30 0x7ffb400c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.593+0000 7ffb3de00640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb4015fbb0 0x7ffb400c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.593+0000 7ffb3d400640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb4015af30 0x7ffb400c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.593+0000 7ffb3d400640 1 -- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb4015fbb0 msgr2=0x7ffb400c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.593+0000 7ffb3d400640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb4015fbb0 0x7ffb400c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.593+0000 7ffb3d400640 1 -- 172.21.15.133:0/1348098565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb400c55d0 con 0x7ffb4015ff90 2024-03-07T06:54:06.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.593+0000 7ffb3e800640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb4015af30 0x7ffb400c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:06.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.593+0000 7ffb3d400640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb4015ff90 0x7ffb400c7610 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00c5a0 tx=0x7ffb2c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:06.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.594+0000 7ffb36000640 1 -- 172.21.15.133:0/1348098565 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c015070 con 0x7ffb4015ff90 2024-03-07T06:54:06.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.594+0000 7ffb36000640 1 -- 172.21.15.133:0/1348098565 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c004220 con 0x7ffb4015ff90 2024-03-07T06:54:06.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.594+0000 7ffb36000640 1 -- 172.21.15.133:0/1348098565 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c003610 con 0x7ffb4015ff90 2024-03-07T06:54:06.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.594+0000 7ffb45200640 1 -- 172.21.15.133:0/1348098565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb400c5850 con 0x7ffb4015ff90 2024-03-07T06:54:06.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.594+0000 7ffb45200640 1 -- 172.21.15.133:0/1348098565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb400c5e10 con 0x7ffb4015ff90 2024-03-07T06:54:06.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.595+0000 7ffb45200640 1 -- 172.21.15.133:0/1348098565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb40062c90 con 0x7ffb4015ff90 2024-03-07T06:54:06.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.596+0000 7ffb36000640 1 -- 172.21.15.133:0/1348098565 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ffb2c0037b0 con 0x7ffb4015ff90 2024-03-07T06:54:06.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.597+0000 7ffb36000640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ffb2006b4d0 0x7ffb2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:06.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.597+0000 7ffb36000640 1 -- 172.21.15.133:0/1348098565 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffb2c089ff0 con 0x7ffb4015ff90 2024-03-07T06:54:06.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.597+0000 7ffb3de00640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ffb2006b4d0 0x7ffb2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:06.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.598+0000 7ffb3de00640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ffb2006b4d0 0x7ffb2006d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffb30036000 tx=0x7ffb30008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:06.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.601+0000 7ffb36000640 1 -- 172.21.15.133:0/1348098565 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ffb400c55d0 con 0x7ffb4015ff90 2024-03-07T06:54:06.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.796+0000 7ffb45200640 1 -- 172.21.15.133:0/1348098565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb400719a0 con 0x7ffb4015ff90 2024-03-07T06:54:06.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.798+0000 7ffb36000640 1 -- 172.21.15.133:0/1348098565 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+13070 (secure 0 0 0) 0x7ffb2c059140 con 0x7ffb4015ff90 2024-03-07T06:54:06.796 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:54:06.797 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":51,"fsid":"1655279d-73c2-44d4-ba1b-3e8a15ae005a","created":"2024-03-07T06:50:49.322818+0000","modified":"2024-03-07T06:54:01.319730+0000","last_up_change":"2024-03-07T06:51:02.760983+0000","last_in_change":"2024-03-07T06:50:53.834716+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-07T06:51:04.544943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"467c9ca0-afeb-42a2-bdb7-d61ac09c0e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6819","nonce":316026862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6822","nonce":316026862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6827","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6828","nonce":316026862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":316026862},{"type":"v1","addr":"172.21.15.133:6825","nonce":316026862}]},"public_addr":"172.21.15.133:6819/316026862","cluster_addr":"172.21.15.133:6822/316026862","heartbeat_back_addr":"172.21.15.133:6828/316026862","heartbeat_front_addr":"172.21.15.133:6825/316026862","state":["exists","up"]},{"osd":1,"uuid":"5457d2a7-f1f3-4ff1-86a2-a5ed0a6cfe4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6801","nonce":1149391957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6803","nonce":1149391957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6807","nonce":1149391957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":1149391957},{"type":"v1","addr":"172.21.15.133:6805","nonce":1149391957}]},"public_addr":"172.21.15.133:6801/1149391957","cluster_addr":"172.21.15.133:6803/1149391957","heartbeat_back_addr":"172.21.15.133:6807/1149391957","heartbeat_front_addr":"172.21.15.133:6805/1149391957","state":["exists","up"]},{"osd":2,"uuid":"bf595492-1c7c-41bd-a9fa-a3d855305317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6809","nonce":3182454988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6811","nonce":3182454988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6815","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6818","nonce":3182454988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3182454988},{"type":"v1","addr":"172.21.15.133:6813","nonce":3182454988}]},"public_addr":"172.21.15.133:6809/3182454988","cluster_addr":"172.21.15.133:6811/3182454988","heartbeat_back_addr":"172.21.15.133:6818/3182454988","heartbeat_front_addr":"172.21.15.133:6813/3182454988","state":["exists","up"]},{"osd":3,"uuid":"3af6d127-f3e2-4465-b13e-914b7c50fb33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6817","nonce":2661528260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6821","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6823","nonce":2661528260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6831","nonce":2661528260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":2661528260},{"type":"v1","addr":"172.21.15.133:6829","nonce":2661528260}]},"public_addr":"172.21.15.133:6817/2661528260","cluster_addr":"172.21.15.133:6823/2661528260","heartbeat_back_addr":"172.21.15.133:6831/2661528260","heartbeat_front_addr":"172.21.15.133:6829/2661528260","state":["exists","up"]},{"osd":4,"uuid":"163f8723-31be-4a08-bd17-c9d68108167f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6827","nonce":3482024257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6829","nonce":3482024257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6833","nonce":3482024257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":3482024257},{"type":"v1","addr":"172.21.15.161:6831","nonce":3482024257}]},"public_addr":"172.21.15.161:6827/3482024257","cluster_addr":"172.21.15.161:6829/3482024257","heartbeat_back_addr":"172.21.15.161:6833/3482024257","heartbeat_front_addr":"172.21.15.161:6831/3482024257","state":["exists","up"]},{"osd":5,"uuid":"972747df-dbbf-4ae5-8101-be5d39fde1ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6819","nonce":1085452100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6821","nonce":1085452100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6825","nonce":1085452100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":1085452100},{"type":"v1","addr":"172.21.15.161:6823","nonce":1085452100}]},"public_addr":"172.21.15.161:6819/1085452100","cluster_addr":"172.21.15.161:6821/1085452100","heartbeat_back_addr":"172.21.15.161:6825/1085452100","heartbeat_front_addr":"172.21.15.161:6823/1085452100","state":["exists","up"]},{"osd":6,"uuid":"df89d7ba-8eab-4e1f-b03b-6e322d1a26d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6803","nonce":2969599292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6805","nonce":2969599292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6809","nonce":2969599292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":2969599292},{"type":"v1","addr":"172.21.15.161:6807","nonce":2969599292}]},"public_addr":"172.21.15.161:6803/2969599292","cluster_addr":"172.21.15.161:6805/2969599292","heartbeat_back_addr":"172.21.15.161:6809/2969599292","heartbeat_front_addr":"172.21.15.161:6807/2969599292","state":["exists","up"]},{"osd":7,"uuid":"a53c260c-9f9b-418c-b6d2-8aa034368372","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6811","nonce":3665789028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6813","nonce":3665789028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6817","nonce":3665789028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":3665789028},{"type":"v1","addr":"172.21.15.161:6815","nonce":3665789028}]},"public_addr":"172.21.15.161:6811/3665789028","cluster_addr":"172.21.15.161:6813/3665789028","heartbeat_back_addr":"172.21.15.161:6817/3665789028","heartbeat_front_addr":"172.21.15.161:6815/3665789028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.370818+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.107415+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.203497+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.260250+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.221985+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.222239+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.149609+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-07T06:50:59.166977+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6835/2951618124":"2024-03-08T06:53:57.854870+0000","172.21.15.133:6834/2951618124":"2024-03-08T06:53:57.854870+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-07T06:54:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.800+0000 7ffb45200640 1 -- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ffb2006b4d0 msgr2=0x7ffb2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.800+0000 7ffb45200640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ffb2006b4d0 0x7ffb2006d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffb30036000 tx=0x7ffb30008030 comp rx=0 tx=0).stop 2024-03-07T06:54:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.800+0000 7ffb45200640 1 -- 172.21.15.133:0/1348098565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb4015ff90 msgr2=0x7ffb400c7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.800+0000 7ffb45200640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb4015ff90 0x7ffb400c7610 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00c5a0 tx=0x7ffb2c00ca60 comp rx=0 tx=0).stop 2024-03-07T06:54:06.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.800+0000 7ffb45200640 1 -- 172.21.15.133:0/1348098565 shutdown_connections 2024-03-07T06:54:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.800+0000 7ffb45200640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ffb4015af30 0x7ffb400c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.800+0000 7ffb45200640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffb4015ff90 0x7ffb400c7610 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.800+0000 7ffb45200640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ffb2006b4d0 0x7ffb2006d990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.800+0000 7ffb45200640 1 --2- 172.21.15.133:0/1348098565 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ffb4015fbb0 0x7ffb400c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.800+0000 7ffb45200640 1 -- 172.21.15.133:0/1348098565 >> 172.21.15.133:0/1348098565 conn(0x7ffb400b3120 msgr2=0x7ffb400b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.800+0000 7ffb45200640 1 -- 172.21.15.133:0/1348098565 shutdown_connections 2024-03-07T06:54:06.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:06.801+0000 7ffb45200640 1 -- 172.21.15.133:0/1348098565 wait complete. 2024-03-07T06:54:07.820 INFO:tasks.ceph:Scrubbing osd.0 2024-03-07T06:54:07.820 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-07T06:54:08.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.040+0000 7fa968a00640 1 Processor -- start 2024-03-07T06:54:08.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.040+0000 7fa968a00640 1 -- start start 2024-03-07T06:54:08.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.042+0000 7fa968a00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa964157ec0 0x7fa96415b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.042+0000 7fa968a00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9640b2440 con 0x7fa9640cf9e0 2024-03-07T06:54:08.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.042+0000 7fa968a00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9640b2720 con 0x7fa9640b4970 2024-03-07T06:54:08.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.042+0000 7fa968a00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa96415add0 con 0x7fa964157ec0 2024-03-07T06:54:08.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.042+0000 7fa961400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fa9640cf9e0 0x7fa9640b1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46452/0 (socket says 172.21.15.133:46452) 2024-03-07T06:54:08.041 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.042+0000 7fa961400640 1 -- 172.21.15.133:0/1092762197 learned_addr learned my addr 172.21.15.133:0/1092762197 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:54:08.042 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.042+0000 7fa961e00640 1 --2- 172.21.15.133:0/1092762197 >> v2:172.21.15.161:3300/0 conn(0x7fa964157ec0 0x7fa96415b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.042+0000 7fa960a00640 1 -- 172.21.15.133:0/1092762197 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545594373 0 0) 0x7fa9640b2440 con 0x7fa9640cf9e0 2024-03-07T06:54:08.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.042+0000 7fa960a00640 1 -- 172.21.15.133:0/1092762197 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa944003520 con 0x7fa9640cf9e0 2024-03-07T06:54:08.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.043+0000 7fa960a00640 1 -- 172.21.15.133:0/1092762197 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3610689812 0 0) 0x7fa944003520 con 0x7fa9640cf9e0 2024-03-07T06:54:08.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.043+0000 7fa960a00640 1 -- 172.21.15.133:0/1092762197 >> v2:172.21.15.161:3300/0 conn(0x7fa964157ec0 msgr2=0x7fa96415b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:08.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.043+0000 7fa960a00640 1 --2- 172.21.15.133:0/1092762197 >> v2:172.21.15.161:3300/0 conn(0x7fa964157ec0 0x7fa96415b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.043+0000 7fa960a00640 1 -- 172.21.15.133:0/1092762197 >> v1:172.21.15.161:6789/0 conn(0x7fa9640b4970 legacy=0x7fa96406d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:08.043 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.043+0000 7fa960a00640 1 -- 172.21.15.133:0/1092762197 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96415b0b0 con 0x7fa9640cf9e0 2024-03-07T06:54:08.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.043+0000 7fa960a00640 1 -- 172.21.15.133:0/1092762197 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fa94c0019f0 con 0x7fa9640cf9e0 2024-03-07T06:54:08.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.043+0000 7fa960a00640 1 -- 172.21.15.133:0/1092762197 >> v1:172.21.15.133:6789/0 conn(0x7fa9640cf9e0 legacy=0x7fa9640b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.043+0000 7fa960a00640 1 --2- 172.21.15.133:0/1092762197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9440039b0 0x7fa944003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.043+0000 7fa960a00640 1 --2- 172.21.15.133:0/1092762197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa944004800 0x7fa944004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.043+0000 7fa960a00640 1 --2- 172.21.15.133:0/1092762197 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa9440051c0 0x7fa94400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.043+0000 7fa960a00640 1 -- 172.21.15.133:0/1092762197 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96415add0 con 0x7fa9440039b0 2024-03-07T06:54:08.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.043+0000 7fa960a00640 1 -- 172.21.15.133:0/1092762197 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94400d220 con 0x7fa944004800 2024-03-07T06:54:08.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.043+0000 7fa960a00640 1 -- 172.21.15.133:0/1092762197 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94400c160 con 0x7fa9440051c0 2024-03-07T06:54:08.044 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.043+0000 7fa961e00640 1 --2- 172.21.15.133:0/1092762197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9440039b0 0x7fa944003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.044+0000 7fa961e00640 1 -- 172.21.15.133:0/1092762197 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa9440051c0 msgr2=0x7fa94400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:08.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.044+0000 7fa961400640 1 --2- 172.21.15.133:0/1092762197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa944004800 0x7fa944004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.044+0000 7fa961e00640 1 --2- 172.21.15.133:0/1092762197 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa9440051c0 0x7fa94400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.044+0000 7fa961e00640 1 -- 172.21.15.133:0/1092762197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa944004800 msgr2=0x7fa944004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:08.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.044+0000 7fa961e00640 1 --2- 172.21.15.133:0/1092762197 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa944004800 0x7fa944004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.044+0000 7fa962800640 1 -- 172.21.15.133:0/1092762197 reap_dead start 2024-03-07T06:54:08.045 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.044+0000 7fa961e00640 1 -- 172.21.15.133:0/1092762197 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa944003520 con 0x7fa9440039b0 2024-03-07T06:54:08.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.045+0000 7fa961e00640 1 --2- 172.21.15.133:0/1092762197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9440039b0 0x7fa944003dd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa95400ba30 tx=0x7fa944005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c334c90a7025939c server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:08.046 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.046+0000 7fa960a00640 1 -- 172.21.15.133:0/1092762197 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954036020 con 0x7fa9440039b0 2024-03-07T06:54:08.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.046+0000 7fa960a00640 1 -- 172.21.15.133:0/1092762197 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa954032680 con 0x7fa9440039b0 2024-03-07T06:54:08.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.046+0000 7fa960a00640 1 -- 172.21.15.133:0/1092762197 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95400b600 con 0x7fa9440039b0 2024-03-07T06:54:08.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.046+0000 7fa968a00640 1 -- 172.21.15.133:0/1092762197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9440039b0 msgr2=0x7fa944003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.046+0000 7fa968a00640 1 --2- 172.21.15.133:0/1092762197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9440039b0 0x7fa944003dd0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa95400ba30 tx=0x7fa944005830 comp rx=0 tx=0).stop 2024-03-07T06:54:08.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.046+0000 7fa968a00640 1 -- 172.21.15.133:0/1092762197 shutdown_connections 2024-03-07T06:54:08.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.046+0000 7fa968a00640 1 --2- 172.21.15.133:0/1092762197 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9440039b0 0x7fa944003dd0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.046+0000 7fa968a00640 1 -- 172.21.15.133:0/1092762197 >> 172.21.15.133:0/1092762197 conn(0x7fa964066eb0 msgr2=0x7fa964067290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:08.047 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.046+0000 7fa968a00640 1 -- 172.21.15.133:0/1092762197 shutdown_connections 2024-03-07T06:54:08.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.046+0000 7fa968a00640 1 -- 172.21.15.133:0/1092762197 wait complete. 2024-03-07T06:54:08.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.048+0000 7fa968a00640 1 Processor -- start 2024-03-07T06:54:08.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.048+0000 7fa968a00640 1 -- start start 2024-03-07T06:54:08.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.049+0000 7fa968a00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa9440039b0 0x7fa96414a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.049+0000 7fa968a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9641504d0 0x7fa9640b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.049+0000 7fa968a00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa96415a1d0 0x7fa9640b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.049+0000 7fa968a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa964158420 con 0x7fa9641504d0 2024-03-07T06:54:08.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.049+0000 7fa968a00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9641582a0 con 0x7fa96415a1d0 2024-03-07T06:54:08.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.049+0000 7fa968a00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96415b5d0 con 0x7fa9440039b0 2024-03-07T06:54:08.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.050+0000 7fa961400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9641504d0 0x7fa9640b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.050+0000 7fa961400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9641504d0 0x7fa9640b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50218/0 (socket says 172.21.15.133:50218) 2024-03-07T06:54:08.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.050+0000 7fa961400640 1 -- 172.21.15.133:0/410452957 learned_addr learned my addr 172.21.15.133:0/410452957 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:08.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.050+0000 7fa961e00640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa9440039b0 0x7fa96414a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.050+0000 7fa962800640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa96415a1d0 0x7fa9640b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.050+0000 7fa961400640 1 -- 172.21.15.133:0/410452957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa9440039b0 msgr2=0x7fa96414a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.050+0000 7fa961400640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa9440039b0 0x7fa96414a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.050+0000 7fa961400640 1 -- 172.21.15.133:0/410452957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa96415a1d0 msgr2=0x7fa9640b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.050+0000 7fa961400640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa96415a1d0 0x7fa9640b3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.050+0000 7fa961400640 1 -- 172.21.15.133:0/410452957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96414df90 con 0x7fa9641504d0 2024-03-07T06:54:08.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.050+0000 7fa961e00640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa9440039b0 0x7fa96414a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:08.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.050+0000 7fa961400640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9641504d0 0x7fa9640b39c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa94c001580 tx=0x7fa94c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:08.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.051+0000 7fa95a000640 1 -- 172.21.15.133:0/410452957 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c019070 con 0x7fa9641504d0 2024-03-07T06:54:08.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.051+0000 7fa95a000640 1 -- 172.21.15.133:0/410452957 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c004d60 con 0x7fa9641504d0 2024-03-07T06:54:08.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.051+0000 7fa95a000640 1 -- 172.21.15.133:0/410452957 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0072f0 con 0x7fa9641504d0 2024-03-07T06:54:08.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.051+0000 7fa968a00640 1 -- 172.21.15.133:0/410452957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96414f9f0 con 0x7fa9641504d0 2024-03-07T06:54:08.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.051+0000 7fa968a00640 1 -- 172.21.15.133:0/410452957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96414ffb0 con 0x7fa9641504d0 2024-03-07T06:54:08.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.052+0000 7fa968a00640 1 -- 172.21.15.133:0/410452957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa964062c90 con 0x7fa9641504d0 2024-03-07T06:54:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.053+0000 7fa95a000640 1 -- 172.21.15.133:0/410452957 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa94c007490 con 0x7fa9641504d0 2024-03-07T06:54:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.054+0000 7fa95a000640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa94006b4d0 0x7fa94006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.054+0000 7fa95a000640 1 -- 172.21.15.133:0/410452957 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa94c089ff0 con 0x7fa9641504d0 2024-03-07T06:54:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.054+0000 7fa95a000640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.133:6814/316026862,v1:172.21.15.133:6819/316026862] conn(0x7fa940071390 0x7fa9400737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.054+0000 7fa95a000640 1 -- 172.21.15.133:0/410452957 --> [v2:172.21.15.133:6814/316026862,v1:172.21.15.133:6819/316026862] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa940073e60 con 0x7fa940071390 2024-03-07T06:54:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.054+0000 7fa95a000640 1 -- 172.21.15.133:0/410452957 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa94c091050 con 0x7fa9641504d0 2024-03-07T06:54:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.054+0000 7fa962800640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.133:6814/316026862,v1:172.21.15.133:6819/316026862] conn(0x7fa940071390 0x7fa9400737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.054+0000 7fa961e00640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa94006b4d0 0x7fa94006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.055+0000 7fa962800640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.133:6814/316026862,v1:172.21.15.133:6819/316026862] conn(0x7fa940071390 0x7fa9400737b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:08.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.055+0000 7fa961e00640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa94006b4d0 0x7fa94006d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa95400ba00 tx=0x7fa954030000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:08.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.056+0000 7fa95a000640 1 -- 172.21.15.133:0/410452957 <== osd.0 v2:172.21.15.133:6814/316026862 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa940073e60 con 0x7fa940071390 2024-03-07T06:54:08.068 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.070+0000 7fa968a00640 1 -- 172.21.15.133:0/410452957 --> [v2:172.21.15.133:6814/316026862,v1:172.21.15.133:6819/316026862] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa96406e8e0 con 0x7fa940071390 2024-03-07T06:54:08.069 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.070+0000 7fa95a000640 1 -- 172.21.15.133:0/410452957 <== osd.0 v2:172.21.15.133:6814/316026862 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa96406e8e0 con 0x7fa940071390 2024-03-07T06:54:08.069 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:54:08.069 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T06:54:08.070 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:54:08.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.071+0000 7fa968a00640 1 -- 172.21.15.133:0/410452957 >> [v2:172.21.15.133:6814/316026862,v1:172.21.15.133:6819/316026862] conn(0x7fa940071390 msgr2=0x7fa9400737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.071+0000 7fa968a00640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.133:6814/316026862,v1:172.21.15.133:6819/316026862] conn(0x7fa940071390 0x7fa9400737b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.071+0000 7fa968a00640 1 -- 172.21.15.133:0/410452957 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa94006b4d0 msgr2=0x7fa94006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.071+0000 7fa968a00640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa94006b4d0 0x7fa94006d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa95400ba00 tx=0x7fa954030000 comp rx=0 tx=0).stop 2024-03-07T06:54:08.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.071+0000 7fa968a00640 1 -- 172.21.15.133:0/410452957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9641504d0 msgr2=0x7fa9640b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.071+0000 7fa968a00640 1 --2- 172.21.15.133:0/410452957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9641504d0 0x7fa9640b39c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa94c001580 tx=0x7fa94c00b4a0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.071+0000 7fa962800640 1 -- 172.21.15.133:0/410452957 reap_dead start 2024-03-07T06:54:08.071 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.071+0000 7fa968a00640 1 -- 172.21.15.133:0/410452957 shutdown_connections 2024-03-07T06:54:08.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.071+0000 7fa968a00640 1 -- 172.21.15.133:0/410452957 >> 172.21.15.133:0/410452957 conn(0x7fa964066eb0 msgr2=0x7fa9640b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:08.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.071+0000 7fa968a00640 1 -- 172.21.15.133:0/410452957 shutdown_connections 2024-03-07T06:54:08.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.071+0000 7fa968a00640 1 -- 172.21.15.133:0/410452957 wait complete. 2024-03-07T06:54:08.088 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-07T06:54:08.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.291+0000 7f6d6fe00640 1 Processor -- start 2024-03-07T06:54:08.290 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.291+0000 7f6d6fe00640 1 -- start start 2024-03-07T06:54:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.293+0000 7f6d6fe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f6d700b7560 0x7f6d700d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.293+0000 7f6d6fe00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d70143810 con 0x7f6d70140cb0 2024-03-07T06:54:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.293+0000 7f6d6fe00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d70143af0 con 0x7f6d700b70d0 2024-03-07T06:54:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.293+0000 7f6d6fe00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d701435c0 con 0x7f6d700b7560 2024-03-07T06:54:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.293+0000 7f6d6ea00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f6d70140cb0 0x7f6d70162290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46460/0 (socket says 172.21.15.133:46460) 2024-03-07T06:54:08.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.293+0000 7f6d6ea00640 1 -- 172.21.15.133:0/1839842333 learned_addr learned my addr 172.21.15.133:0/1839842333 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:54:08.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.293+0000 7f6d6f400640 1 --2- 172.21.15.133:0/1839842333 >> v2:172.21.15.133:3300/0 conn(0x7f6d700b7560 0x7f6d700d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.294+0000 7f6d6d600640 1 -- 172.21.15.133:0/1839842333 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740608597 0 0) 0x7f6d70143810 con 0x7f6d70140cb0 2024-03-07T06:54:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.294+0000 7f6d6d600640 1 -- 172.21.15.133:0/1839842333 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d54003520 con 0x7f6d70140cb0 2024-03-07T06:54:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.294+0000 7f6d6f400640 1 -- 172.21.15.133:0/1839842333 >> v1:172.21.15.161:6789/0 conn(0x7f6d700b70d0 legacy=0x7f6d700d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.294+0000 7f6d6f400640 1 -- 172.21.15.133:0/1839842333 >> v1:172.21.15.133:6789/0 conn(0x7f6d70140cb0 legacy=0x7f6d70162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.294+0000 7f6d6f400640 1 -- 172.21.15.133:0/1839842333 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d70143dd0 con 0x7f6d700b7560 2024-03-07T06:54:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.295+0000 7f6d6f400640 1 --2- 172.21.15.133:0/1839842333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d700b7560 0x7f6d700d2200 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009d90 tx=0x7f6d6802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad60787a7b8d06f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.295+0000 7f6d6d600640 1 -- 172.21.15.133:0/1839842333 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d68039070 con 0x7f6d700b7560 2024-03-07T06:54:08.294 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.295+0000 7f6d6d600640 1 -- 172.21.15.133:0/1839842333 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6802cc60 con 0x7f6d700b7560 2024-03-07T06:54:08.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.295+0000 7f6d6d600640 1 -- 172.21.15.133:0/1839842333 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d68034660 con 0x7f6d700b7560 2024-03-07T06:54:08.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.296+0000 7f6d6fe00640 1 -- 172.21.15.133:0/1839842333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d700b7560 msgr2=0x7f6d700d2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.296+0000 7f6d6fe00640 1 --2- 172.21.15.133:0/1839842333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d700b7560 0x7f6d700d2200 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6d68009d90 tx=0x7f6d6802beb0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.296+0000 7f6d6fe00640 1 -- 172.21.15.133:0/1839842333 shutdown_connections 2024-03-07T06:54:08.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.296+0000 7f6d6fe00640 1 --2- 172.21.15.133:0/1839842333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d700b7560 0x7f6d700d2200 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.296+0000 7f6d6fe00640 1 -- 172.21.15.133:0/1839842333 >> 172.21.15.133:0/1839842333 conn(0x7f6d700674e0 msgr2=0x7f6d700d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:08.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.296+0000 7f6d6fe00640 1 -- 172.21.15.133:0/1839842333 shutdown_connections 2024-03-07T06:54:08.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.296+0000 7f6d6fe00640 1 -- 172.21.15.133:0/1839842333 wait complete. 2024-03-07T06:54:08.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.298+0000 7f6d6fe00640 1 Processor -- start 2024-03-07T06:54:08.297 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.298+0000 7f6d6fe00640 1 -- start start 2024-03-07T06:54:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.299+0000 7f6d6fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d700b70d0 0x7f6d70156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.299+0000 7f6d6fe00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d700b7560 0x7f6d70134400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.299+0000 7f6d6fe00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d70140cb0 0x7f6d70134940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.299+0000 7f6d6fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d700b7990 con 0x7f6d700b70d0 2024-03-07T06:54:08.299 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.299+0000 7f6d6fe00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d70144420 con 0x7f6d70140cb0 2024-03-07T06:54:08.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.299+0000 7f6d6fe00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d70161bb0 con 0x7f6d700b7560 2024-03-07T06:54:08.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.299+0000 7f6d6ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d700b70d0 0x7f6d70156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.299+0000 7f6d6ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d700b70d0 0x7f6d70156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50246/0 (socket says 172.21.15.133:50246) 2024-03-07T06:54:08.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.299+0000 7f6d6ea00640 1 -- 172.21.15.133:0/2654401693 learned_addr learned my addr 172.21.15.133:0/2654401693 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:08.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.300+0000 7f6d6f400640 1 --2- 172.21.15.133:0/2654401693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d70140cb0 0x7f6d70134940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.300+0000 7f6d6e000640 1 --2- 172.21.15.133:0/2654401693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d700b7560 0x7f6d70134400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.300+0000 7f6d6ea00640 1 -- 172.21.15.133:0/2654401693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d700b7560 msgr2=0x7f6d70134400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.300+0000 7f6d6ea00640 1 --2- 172.21.15.133:0/2654401693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d700b7560 0x7f6d70134400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.300+0000 7f6d6ea00640 1 -- 172.21.15.133:0/2654401693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d70140cb0 msgr2=0x7f6d70134940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.300+0000 7f6d6ea00640 1 --2- 172.21.15.133:0/2654401693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d70140cb0 0x7f6d70134940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.300+0000 7f6d6ea00640 1 -- 172.21.15.133:0/2654401693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d70157990 con 0x7f6d700b70d0 2024-03-07T06:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.300+0000 7f6d6ea00640 1 --2- 172.21.15.133:0/2654401693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d700b70d0 0x7f6d70156ed0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000c3c0 tx=0x7f6d6000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.300+0000 7f6d67400640 1 -- 172.21.15.133:0/2654401693 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60017070 con 0x7f6d700b70d0 2024-03-07T06:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.300+0000 7f6d67400640 1 -- 172.21.15.133:0/2654401693 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d60005ce0 con 0x7f6d700b70d0 2024-03-07T06:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.300+0000 7f6d67400640 1 -- 172.21.15.133:0/2654401693 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d600072f0 con 0x7f6d700b70d0 2024-03-07T06:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.300+0000 7f6d6fe00640 1 -- 172.21.15.133:0/2654401693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d70157c70 con 0x7f6d700b70d0 2024-03-07T06:54:08.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.300+0000 7f6d6fe00640 1 -- 172.21.15.133:0/2654401693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7012d230 con 0x7f6d700b70d0 2024-03-07T06:54:08.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.302+0000 7f6d6fe00640 1 -- 172.21.15.133:0/2654401693 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d70141a30 con 0x7f6d700b70d0 2024-03-07T06:54:08.302 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.302+0000 7f6d67400640 1 -- 172.21.15.133:0/2654401693 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f6d6001a070 con 0x7f6d700b70d0 2024-03-07T06:54:08.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.303+0000 7f6d67400640 1 --2- 172.21.15.133:0/2654401693 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6d5006b500 0x7f6d5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.303+0000 7f6d67400640 1 -- 172.21.15.133:0/2654401693 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6d60089300 con 0x7f6d700b70d0 2024-03-07T06:54:08.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.304+0000 7f6d6e000640 1 --2- 172.21.15.133:0/2654401693 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6d5006b500 0x7f6d5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.303 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.305+0000 7f6d6e000640 1 --2- 172.21.15.133:0/2654401693 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6d5006b500 0x7f6d5006d9c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6d7015de80 tx=0x7f6d5c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:08.306 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.308+0000 7f6d67400640 1 -- 172.21.15.133:0/2654401693 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f6d70141a30 con 0x7f6d700b70d0 2024-03-07T06:54:08.504 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.505+0000 7f6d6fe00640 1 -- 172.21.15.133:0/2654401693 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d70062c90 con 0x7f6d5006b500 2024-03-07T06:54:08.505 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.506+0000 7f6d67400640 1 -- 172.21.15.133:0/2654401693 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6d70062c90 con 0x7f6d5006b500 2024-03-07T06:54:08.505 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 0 to deep-scrub 2024-03-07T06:54:08.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.508+0000 7f6d6fe00640 1 -- 172.21.15.133:0/2654401693 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6d5006b500 msgr2=0x7f6d5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.507 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.509+0000 7f6d6fe00640 1 --2- 172.21.15.133:0/2654401693 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6d5006b500 0x7f6d5006d9c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6d7015de80 tx=0x7f6d5c009040 comp rx=0 tx=0).stop 2024-03-07T06:54:08.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.509+0000 7f6d6fe00640 1 -- 172.21.15.133:0/2654401693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d700b70d0 msgr2=0x7f6d70156ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.509+0000 7f6d6fe00640 1 --2- 172.21.15.133:0/2654401693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d700b70d0 0x7f6d70156ed0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000c3c0 tx=0x7f6d6000c880 comp rx=0 tx=0).stop 2024-03-07T06:54:08.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.509+0000 7f6d6fe00640 1 -- 172.21.15.133:0/2654401693 shutdown_connections 2024-03-07T06:54:08.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.509+0000 7f6d6fe00640 1 --2- 172.21.15.133:0/2654401693 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f6d5006b500 0x7f6d5006d9c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.509+0000 7f6d6fe00640 1 --2- 172.21.15.133:0/2654401693 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d70140cb0 0x7f6d70134940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.509+0000 7f6d6fe00640 1 --2- 172.21.15.133:0/2654401693 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d700b7560 0x7f6d70134400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.509+0000 7f6d6fe00640 1 --2- 172.21.15.133:0/2654401693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d700b70d0 0x7f6d70156ed0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.508 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.509+0000 7f6d6fe00640 1 -- 172.21.15.133:0/2654401693 >> 172.21.15.133:0/2654401693 conn(0x7f6d700674e0 msgr2=0x7f6d700d1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:08.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.509+0000 7f6d6fe00640 1 -- 172.21.15.133:0/2654401693 shutdown_connections 2024-03-07T06:54:08.509 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.509+0000 7f6d6fe00640 1 -- 172.21.15.133:0/2654401693 wait complete. 2024-03-07T06:54:08.526 INFO:tasks.ceph:Scrubbing osd.1 2024-03-07T06:54:08.527 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-07T06:54:08.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.736+0000 7f1665000640 1 Processor -- start 2024-03-07T06:54:08.735 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.736+0000 7f1665000640 1 -- start start 2024-03-07T06:54:08.736 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.738+0000 7f1665000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16600cc6f0 0x7f16600b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.738+0000 7f1665000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1660070820 0x7f1660070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.738+0000 7f1665000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16600b2190 con 0x7f16600cdf00 2024-03-07T06:54:08.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.738+0000 7f1665000640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16600b2430 con 0x7f1660070820 2024-03-07T06:54:08.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.738+0000 7f1665000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16600b2710 con 0x7f16600cc6f0 2024-03-07T06:54:08.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.738+0000 7f165e800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1660070820 0x7f1660070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.738+0000 7f165e800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1660070820 0x7f1660070cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50254/0 (socket says 172.21.15.133:50254) 2024-03-07T06:54:08.737 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.738+0000 7f165e800640 1 -- 172.21.15.133:0/1753214481 learned_addr learned my addr 172.21.15.133:0/1753214481 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:08.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.738+0000 7f165de00640 1 --2- 172.21.15.133:0/1753214481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16600cc6f0 0x7f16600b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.738 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.738+0000 7f165e800640 1 -- 172.21.15.133:0/1753214481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16600cc6f0 msgr2=0x7f16600b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:08.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.738+0000 7f165e800640 1 --2- 172.21.15.133:0/1753214481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16600cc6f0 0x7f16600b1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.738+0000 7f165e800640 1 -- 172.21.15.133:0/1753214481 >> v1:172.21.15.161:6789/0 conn(0x7f16600cdf00 legacy=0x7f166015ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:08.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.738+0000 7f165e800640 1 -- 172.21.15.133:0/1753214481 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16600b29f0 con 0x7f1660070820 2024-03-07T06:54:08.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.740+0000 7f165e800640 1 --2- 172.21.15.133:0/1753214481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1660070820 0x7f1660070cb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1658009e50 tx=0x7f165802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9361aaaf949587f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:08.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.740+0000 7f165ca00640 1 -- 172.21.15.133:0/1753214481 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165802ca70 con 0x7f1660070820 2024-03-07T06:54:08.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.740+0000 7f165ca00640 1 -- 172.21.15.133:0/1753214481 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165802cc10 con 0x7f1660070820 2024-03-07T06:54:08.739 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.740+0000 7f165ca00640 1 -- 172.21.15.133:0/1753214481 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16580343f0 con 0x7f1660070820 2024-03-07T06:54:08.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.741+0000 7f1665000640 1 -- 172.21.15.133:0/1753214481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1660070820 msgr2=0x7f1660070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.741+0000 7f1665000640 1 --2- 172.21.15.133:0/1753214481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1660070820 0x7f1660070cb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1658009e50 tx=0x7f165802bed0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.741+0000 7f1665000640 1 -- 172.21.15.133:0/1753214481 shutdown_connections 2024-03-07T06:54:08.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.741+0000 7f1665000640 1 --2- 172.21.15.133:0/1753214481 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1660070820 0x7f1660070cb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.741+0000 7f1665000640 1 --2- 172.21.15.133:0/1753214481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f16600cc6f0 0x7f16600b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.741+0000 7f1665000640 1 -- 172.21.15.133:0/1753214481 >> 172.21.15.133:0/1753214481 conn(0x7f1660066eb0 msgr2=0x7f1660067290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:08.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.741+0000 7f1665000640 1 -- 172.21.15.133:0/1753214481 shutdown_connections 2024-03-07T06:54:08.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.741+0000 7f1665000640 1 -- 172.21.15.133:0/1753214481 wait complete. 2024-03-07T06:54:08.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.743+0000 7f1665000640 1 Processor -- start 2024-03-07T06:54:08.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.743+0000 7f1665000640 1 -- start start 2024-03-07T06:54:08.743 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.744+0000 7f1665000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1660070820 0x7f166013b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.744+0000 7f1665000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16600cc6f0 0x7f166014d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.744+0000 7f1665000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16600cdf00 0x7f166014dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.744+0000 7f1665000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166015fa50 con 0x7f16600cdf00 2024-03-07T06:54:08.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.744+0000 7f1665000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166015c460 con 0x7f16600cc6f0 2024-03-07T06:54:08.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.744+0000 7f1665000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166015fbd0 con 0x7f1660070820 2024-03-07T06:54:08.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.744+0000 7f165e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16600cdf00 0x7f166014dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.744 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.744+0000 7f165e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16600cdf00 0x7f166014dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50256/0 (socket says 172.21.15.133:50256) 2024-03-07T06:54:08.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.744+0000 7f165e800640 1 -- 172.21.15.133:0/985529922 learned_addr learned my addr 172.21.15.133:0/985529922 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:08.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.744+0000 7f165d400640 1 --2- 172.21.15.133:0/985529922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16600cc6f0 0x7f166014d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.744+0000 7f165de00640 1 --2- 172.21.15.133:0/985529922 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1660070820 0x7f166013b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.744+0000 7f165e800640 1 -- 172.21.15.133:0/985529922 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1660070820 msgr2=0x7f166013b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.744+0000 7f165e800640 1 --2- 172.21.15.133:0/985529922 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1660070820 0x7f166013b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.744+0000 7f165e800640 1 -- 172.21.15.133:0/985529922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16600cc6f0 msgr2=0x7f166014d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.745+0000 7f165e800640 1 --2- 172.21.15.133:0/985529922 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16600cc6f0 0x7f166014d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.745+0000 7f165e800640 1 -- 172.21.15.133:0/985529922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16600b39c0 con 0x7f16600cdf00 2024-03-07T06:54:08.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.745+0000 7f165e800640 1 --2- 172.21.15.133:0/985529922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16600cdf00 0x7f166014dc90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1658009e50 tx=0x7f1658004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.745+0000 7f1656000640 1 -- 172.21.15.133:0/985529922 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1658043020 con 0x7f16600cdf00 2024-03-07T06:54:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.745+0000 7f1656000640 1 -- 172.21.15.133:0/985529922 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165803e450 con 0x7f16600cdf00 2024-03-07T06:54:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.745+0000 7f1656000640 1 -- 172.21.15.133:0/985529922 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1658009510 con 0x7f16600cdf00 2024-03-07T06:54:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.745+0000 7f1665000640 1 -- 172.21.15.133:0/985529922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16600b3ca0 con 0x7f16600cdf00 2024-03-07T06:54:08.748 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.745+0000 7f1665000640 1 -- 172.21.15.133:0/985529922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16600b4260 con 0x7f16600cdf00 2024-03-07T06:54:08.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.746+0000 7f1665000640 1 -- 172.21.15.133:0/985529922 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f16600cc230 con 0x7f16600cdf00 2024-03-07T06:54:08.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.747+0000 7f1656000640 1 -- 172.21.15.133:0/985529922 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1658005d10 con 0x7f16600cdf00 2024-03-07T06:54:08.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.748+0000 7f1656000640 1 --2- 172.21.15.133:0/985529922 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f164006b4d0 0x7f164006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.748+0000 7f1656000640 1 -- 172.21.15.133:0/985529922 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f16580ac960 con 0x7f16600cdf00 2024-03-07T06:54:08.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.749+0000 7f1656000640 1 --2- 172.21.15.133:0/985529922 >> [v2:172.21.15.133:6800/1149391957,v1:172.21.15.133:6801/1149391957] conn(0x7f1640071390 0x7f16400737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.749+0000 7f1656000640 1 -- 172.21.15.133:0/985529922 --> [v2:172.21.15.133:6800/1149391957,v1:172.21.15.133:6801/1149391957] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f165800ae60 con 0x7f1640071390 2024-03-07T06:54:08.749 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.749+0000 7f1656000640 1 -- 172.21.15.133:0/985529922 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16580b12c0 con 0x7f16600cdf00 2024-03-07T06:54:08.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.749+0000 7f165de00640 1 --2- 172.21.15.133:0/985529922 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f164006b4d0 0x7f164006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.749+0000 7f165d400640 1 --2- 172.21.15.133:0/985529922 >> [v2:172.21.15.133:6800/1149391957,v1:172.21.15.133:6801/1149391957] conn(0x7f1640071390 0x7f16400737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.750+0000 7f165d400640 1 --2- 172.21.15.133:0/985529922 >> [v2:172.21.15.133:6800/1149391957,v1:172.21.15.133:6801/1149391957] conn(0x7f1640071390 0x7f16400737b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:08.750 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.750+0000 7f165de00640 1 --2- 172.21.15.133:0/985529922 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f164006b4d0 0x7f164006d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f165000a970 tx=0x7f1650008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:08.751 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.751+0000 7f1656000640 1 -- 172.21.15.133:0/985529922 <== osd.1 v2:172.21.15.133:6800/1149391957 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f165800ae60 con 0x7f1640071390 2024-03-07T06:54:08.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.765+0000 7f1665000640 1 -- 172.21.15.133:0/985529922 --> [v2:172.21.15.133:6800/1149391957,v1:172.21.15.133:6801/1149391957] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1660062c90 con 0x7f1640071390 2024-03-07T06:54:08.764 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.766+0000 7f1656000640 1 -- 172.21.15.133:0/985529922 <== osd.1 v2:172.21.15.133:6800/1149391957 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1660062c90 con 0x7f1640071390 2024-03-07T06:54:08.765 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:54:08.765 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T06:54:08.765 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:54:08.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.766+0000 7f1665000640 1 -- 172.21.15.133:0/985529922 >> [v2:172.21.15.133:6800/1149391957,v1:172.21.15.133:6801/1149391957] conn(0x7f1640071390 msgr2=0x7f16400737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.766+0000 7f1665000640 1 --2- 172.21.15.133:0/985529922 >> [v2:172.21.15.133:6800/1149391957,v1:172.21.15.133:6801/1149391957] conn(0x7f1640071390 0x7f16400737b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.767+0000 7f1665000640 1 -- 172.21.15.133:0/985529922 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f164006b4d0 msgr2=0x7f164006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.767+0000 7f1665000640 1 --2- 172.21.15.133:0/985529922 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f164006b4d0 0x7f164006d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f165000a970 tx=0x7f1650008040 comp rx=0 tx=0).stop 2024-03-07T06:54:08.766 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.767+0000 7f1665000640 1 -- 172.21.15.133:0/985529922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16600cdf00 msgr2=0x7f166014dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.767+0000 7f1665000640 1 --2- 172.21.15.133:0/985529922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16600cdf00 0x7f166014dc90 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1658009e50 tx=0x7f1658004a40 comp rx=0 tx=0).stop 2024-03-07T06:54:08.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.767+0000 7f165e800640 1 -- 172.21.15.133:0/985529922 reap_dead start 2024-03-07T06:54:08.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.767+0000 7f1665000640 1 -- 172.21.15.133:0/985529922 shutdown_connections 2024-03-07T06:54:08.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.767+0000 7f1665000640 1 -- 172.21.15.133:0/985529922 >> 172.21.15.133:0/985529922 conn(0x7f1660066eb0 msgr2=0x7f166015ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:08.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.767+0000 7f1665000640 1 -- 172.21.15.133:0/985529922 shutdown_connections 2024-03-07T06:54:08.767 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.767+0000 7f1665000640 1 -- 172.21.15.133:0/985529922 wait complete. 2024-03-07T06:54:08.783 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-07T06:54:08.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.993+0000 7f0d36e00640 1 Processor -- start 2024-03-07T06:54:08.993 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.993+0000 7f0d36e00640 1 -- start start 2024-03-07T06:54:08.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.995+0000 7f0d36e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0d30140cb0 0x7f0d30162290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.995+0000 7f0d36e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0d300b70d0 0x7f0d300b7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:08.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.995+0000 7f0d36e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d30143810 con 0x7f0d300b9300 2024-03-07T06:54:08.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.995+0000 7f0d36e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d301435c0 con 0x7f0d30140cb0 2024-03-07T06:54:08.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.995+0000 7f0d36e00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d30144420 con 0x7f0d300b70d0 2024-03-07T06:54:08.995 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.995+0000 7f0d35a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0d30140cb0 0x7f0d30162290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.995+0000 7f0d35a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f0d30140cb0 0x7f0d30162290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50264/0 (socket says 172.21.15.133:50264) 2024-03-07T06:54:08.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.995+0000 7f0d35a00640 1 -- 172.21.15.133:0/2296850234 learned_addr learned my addr 172.21.15.133:0/2296850234 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:08.996 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.995+0000 7f0d36400640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f0d300b9300 0x7f0d300b9730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46464/0 (socket says 172.21.15.133:46464) 2024-03-07T06:54:08.997 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.996+0000 7f0d35a00640 1 -- 172.21.15.133:0/2296850234 >> v2:172.21.15.161:3300/0 conn(0x7f0d300b70d0 msgr2=0x7f0d300b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.996+0000 7f0d35000640 1 --2- 172.21.15.133:0/2296850234 >> v2:172.21.15.161:3300/0 conn(0x7f0d300b70d0 0x7f0d300b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.996+0000 7f0d35a00640 1 --2- 172.21.15.133:0/2296850234 >> v2:172.21.15.161:3300/0 conn(0x7f0d300b70d0 0x7f0d300b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.996+0000 7f0d35a00640 1 -- 172.21.15.133:0/2296850234 >> v1:172.21.15.133:6789/0 conn(0x7f0d300b9300 legacy=0x7f0d300b9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.996+0000 7f0d35a00640 1 -- 172.21.15.133:0/2296850234 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d30143af0 con 0x7f0d30140cb0 2024-03-07T06:54:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.996+0000 7f0d35000640 1 --2- 172.21.15.133:0/2296850234 >> v2:172.21.15.161:3300/0 conn(0x7f0d300b70d0 0x7f0d300b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.997+0000 7f0d35a00640 1 --2- 172.21.15.133:0/2296850234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d30140cb0 0x7f0d30162290 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0d30147b80 tx=0x7f0d2402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcc363828c70469 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.997+0000 7f0d2fe00640 1 -- 172.21.15.133:0/2296850234 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24034070 con 0x7f0d30140cb0 2024-03-07T06:54:08.998 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.997+0000 7f0d2fe00640 1 -- 172.21.15.133:0/2296850234 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d24030870 con 0x7f0d30140cb0 2024-03-07T06:54:08.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.997+0000 7f0d2fe00640 1 -- 172.21.15.133:0/2296850234 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24030c40 con 0x7f0d30140cb0 2024-03-07T06:54:08.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.998+0000 7f0d36e00640 1 -- 172.21.15.133:0/2296850234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d30140cb0 msgr2=0x7f0d30162290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:08.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.998+0000 7f0d36e00640 1 --2- 172.21.15.133:0/2296850234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d30140cb0 0x7f0d30162290 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f0d30147b80 tx=0x7f0d2402b9b0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.998+0000 7f0d36e00640 1 -- 172.21.15.133:0/2296850234 shutdown_connections 2024-03-07T06:54:08.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.998+0000 7f0d36e00640 1 --2- 172.21.15.133:0/2296850234 >> v2:172.21.15.161:3300/0 conn(0x7f0d300b70d0 0x7f0d300b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.998+0000 7f0d36e00640 1 --2- 172.21.15.133:0/2296850234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d30140cb0 0x7f0d30162290 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:08.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.998+0000 7f0d36e00640 1 -- 172.21.15.133:0/2296850234 >> 172.21.15.133:0/2296850234 conn(0x7f0d300674e0 msgr2=0x7f0d300d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:08.999 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.998+0000 7f0d36e00640 1 -- 172.21.15.133:0/2296850234 shutdown_connections 2024-03-07T06:54:09.000 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:08.998+0000 7f0d36e00640 1 -- 172.21.15.133:0/2296850234 wait complete. 2024-03-07T06:54:09.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.000+0000 7f0d36e00640 1 Processor -- start 2024-03-07T06:54:09.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.000+0000 7f0d36e00640 1 -- start start 2024-03-07T06:54:09.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.001+0000 7f0d36e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d300b70d0 0x7f0d300cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.001+0000 7f0d36e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d300b9300 0x7f0d300b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.001+0000 7f0d36e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d30140cb0 0x7f0d300b6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.002 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.001+0000 7f0d36e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d301627d0 con 0x7f0d300b70d0 2024-03-07T06:54:09.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.001+0000 7f0d36e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d301440b0 con 0x7f0d300b9300 2024-03-07T06:54:09.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.001+0000 7f0d36e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d30161bb0 con 0x7f0d30140cb0 2024-03-07T06:54:09.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.001+0000 7f0d35a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d300b70d0 0x7f0d300cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.001+0000 7f0d35a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d300b70d0 0x7f0d300cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50274/0 (socket says 172.21.15.133:50274) 2024-03-07T06:54:09.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.001+0000 7f0d35a00640 1 -- 172.21.15.133:0/1109890320 learned_addr learned my addr 172.21.15.133:0/1109890320 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:09.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.001+0000 7f0d35a00640 1 -- 172.21.15.133:0/1109890320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d30140cb0 msgr2=0x7f0d300b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.001+0000 7f0d35a00640 1 --2- 172.21.15.133:0/1109890320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d30140cb0 0x7f0d300b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.001+0000 7f0d35a00640 1 -- 172.21.15.133:0/1109890320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d300b9300 msgr2=0x7f0d300b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.003 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.001+0000 7f0d35a00640 1 --2- 172.21.15.133:0/1109890320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d300b9300 0x7f0d300b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.001+0000 7f0d35a00640 1 -- 172.21.15.133:0/1109890320 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d300cfe50 con 0x7f0d300b70d0 2024-03-07T06:54:09.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.002+0000 7f0d35a00640 1 --2- 172.21.15.133:0/1109890320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d300b70d0 0x7f0d300cd590 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0d2402b9f0 tx=0x7f0d240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:09.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.002+0000 7f0d2d600640 1 -- 172.21.15.133:0/1109890320 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d24034040 con 0x7f0d300b70d0 2024-03-07T06:54:09.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.002+0000 7f0d2d600640 1 -- 172.21.15.133:0/1109890320 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d240433f0 con 0x7f0d300b70d0 2024-03-07T06:54:09.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.002+0000 7f0d2d600640 1 -- 172.21.15.133:0/1109890320 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d240437e0 con 0x7f0d300b70d0 2024-03-07T06:54:09.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.002+0000 7f0d36e00640 1 -- 172.21.15.133:0/1109890320 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d301f7800 con 0x7f0d300b70d0 2024-03-07T06:54:09.004 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.002+0000 7f0d36e00640 1 -- 172.21.15.133:0/1109890320 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d301f7dc0 con 0x7f0d300b70d0 2024-03-07T06:54:09.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.003+0000 7f0d36e00640 1 -- 172.21.15.133:0/1109890320 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d30062c90 con 0x7f0d300b70d0 2024-03-07T06:54:09.005 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.004+0000 7f0d2d600640 1 -- 172.21.15.133:0/1109890320 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f0d2404a060 con 0x7f0d300b70d0 2024-03-07T06:54:09.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.006+0000 7f0d2d600640 1 --2- 172.21.15.133:0/1109890320 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0d1406b4d0 0x7f0d1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.006+0000 7f0d2d600640 1 -- 172.21.15.133:0/1109890320 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0d240ac180 con 0x7f0d300b70d0 2024-03-07T06:54:09.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.007+0000 7f0d35000640 1 --2- 172.21.15.133:0/1109890320 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0d1406b4d0 0x7f0d1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.008+0000 7f0d35000640 1 --2- 172.21.15.133:0/1109890320 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0d1406b4d0 0x7f0d1406d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0d20006fd0 tx=0x7f0d20008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:09.010 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.010+0000 7f0d2d600640 1 -- 172.21.15.133:0/1109890320 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f0d2403e070 con 0x7f0d300b70d0 2024-03-07T06:54:09.206 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.206+0000 7f0d36e00640 1 -- 172.21.15.133:0/1109890320 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d301f8000 con 0x7f0d1406b4d0 2024-03-07T06:54:09.207 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.207+0000 7f0d2d600640 1 -- 172.21.15.133:0/1109890320 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0d2000ae40 con 0x7f0d1406b4d0 2024-03-07T06:54:09.207 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 1 to deep-scrub 2024-03-07T06:54:09.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.210+0000 7f0d36e00640 1 -- 172.21.15.133:0/1109890320 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0d1406b4d0 msgr2=0x7f0d1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.210+0000 7f0d36e00640 1 --2- 172.21.15.133:0/1109890320 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0d1406b4d0 0x7f0d1406d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f0d20006fd0 tx=0x7f0d20008040 comp rx=0 tx=0).stop 2024-03-07T06:54:09.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.210+0000 7f0d36e00640 1 -- 172.21.15.133:0/1109890320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d300b70d0 msgr2=0x7f0d300cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.210+0000 7f0d36e00640 1 --2- 172.21.15.133:0/1109890320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d300b70d0 0x7f0d300cd590 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0d2402b9f0 tx=0x7f0d240046c0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.210 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.210+0000 7f0d36e00640 1 -- 172.21.15.133:0/1109890320 shutdown_connections 2024-03-07T06:54:09.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.210+0000 7f0d36e00640 1 --2- 172.21.15.133:0/1109890320 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0d30140cb0 0x7f0d300b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.210+0000 7f0d36e00640 1 --2- 172.21.15.133:0/1109890320 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f0d1406b4d0 0x7f0d1406d990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.210+0000 7f0d36e00640 1 --2- 172.21.15.133:0/1109890320 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0d300b9300 0x7f0d300b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.210+0000 7f0d36e00640 1 --2- 172.21.15.133:0/1109890320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d300b70d0 0x7f0d300cd590 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.210+0000 7f0d36e00640 1 -- 172.21.15.133:0/1109890320 >> 172.21.15.133:0/1109890320 conn(0x7f0d300674e0 msgr2=0x7f0d3015e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:09.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.210+0000 7f0d36e00640 1 -- 172.21.15.133:0/1109890320 shutdown_connections 2024-03-07T06:54:09.211 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.211+0000 7f0d36e00640 1 -- 172.21.15.133:0/1109890320 wait complete. 2024-03-07T06:54:09.228 INFO:tasks.ceph:Scrubbing osd.2 2024-03-07T06:54:09.228 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-07T06:54:09.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.442+0000 7eff0d200640 1 Processor -- start 2024-03-07T06:54:09.442 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.442+0000 7eff0d200640 1 -- start start 2024-03-07T06:54:09.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.444+0000 7eff0d200640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7eff0815bd40 0x7eff0806dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.444+0000 7eff0d200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff080b5690 0x7eff0806e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.444 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.444+0000 7eff0d200640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0806ef90 con 0x7eff080cf560 2024-03-07T06:54:09.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.444+0000 7eff0d200640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0806ed30 con 0x7eff0815bd40 2024-03-07T06:54:09.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.444+0000 7eff0d200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0815d130 con 0x7eff080b5690 2024-03-07T06:54:09.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.444+0000 7efefea00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7eff080cf560 0x7eff080b4f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46476/0 (socket says 172.21.15.133:46476) 2024-03-07T06:54:09.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.444+0000 7efefea00640 1 -- 172.21.15.133:0/3495967502 learned_addr learned my addr 172.21.15.133:0/3495967502 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:54:09.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff07400640 1 --2- 172.21.15.133:0/3495967502 >> v2:172.21.15.161:3300/0 conn(0x7eff0815bd40 0x7eff0806dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff07e00640 1 --2- 172.21.15.133:0/3495967502 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff080b5690 0x7eff0806e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.445 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 -- 172.21.15.133:0/3495967502 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927679473 0 0) 0x7eff0806ef90 con 0x7eff080cf560 2024-03-07T06:54:09.446 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 -- 172.21.15.133:0/3495967502 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeec003520 con 0x7eff080cf560 2024-03-07T06:54:09.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 -- 172.21.15.133:0/3495967502 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3998830839 0 0) 0x7efeec003520 con 0x7eff080cf560 2024-03-07T06:54:09.448 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 -- 172.21.15.133:0/3495967502 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff080b5690 msgr2=0x7eff0806e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:09.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 --2- 172.21.15.133:0/3495967502 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff080b5690 0x7eff0806e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 -- 172.21.15.133:0/3495967502 >> v2:172.21.15.161:3300/0 conn(0x7eff0815bd40 msgr2=0x7eff0806dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:09.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 --2- 172.21.15.133:0/3495967502 >> v2:172.21.15.161:3300/0 conn(0x7eff0815bd40 0x7eff0806dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 -- 172.21.15.133:0/3495967502 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0806d0d0 con 0x7eff080cf560 2024-03-07T06:54:09.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 -- 172.21.15.133:0/3495967502 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7efef0001690 con 0x7eff080cf560 2024-03-07T06:54:09.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 -- 172.21.15.133:0/3495967502 >> v1:172.21.15.133:6789/0 conn(0x7eff080cf560 legacy=0x7eff080b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 --2- 172.21.15.133:0/3495967502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeec0039b0 0x7efeec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 --2- 172.21.15.133:0/3495967502 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff080b5690 0x7efeec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 --2- 172.21.15.133:0/3495967502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeec004d40 0x7efeec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 -- 172.21.15.133:0/3495967502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0806ed30 con 0x7efeec004d40 2024-03-07T06:54:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 -- 172.21.15.133:0/3495967502 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0815d130 con 0x7efeec0039b0 2024-03-07T06:54:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.445+0000 7eff06a00640 1 -- 172.21.15.133:0/3495967502 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec00b9c0 con 0x7eff080b5690 2024-03-07T06:54:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.446+0000 7eff07e00640 1 --2- 172.21.15.133:0/3495967502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeec004d40 0x7efeec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.446+0000 7efefea00640 1 --2- 172.21.15.133:0/3495967502 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff080b5690 0x7efeec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.446+0000 7eff07e00640 1 -- 172.21.15.133:0/3495967502 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff080b5690 msgr2=0x7efeec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:09.450 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.446+0000 7eff07400640 1 --2- 172.21.15.133:0/3495967502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeec0039b0 0x7efeec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.446+0000 7eff07e00640 1 --2- 172.21.15.133:0/3495967502 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff080b5690 0x7efeec004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.446+0000 7eff07e00640 1 -- 172.21.15.133:0/3495967502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeec0039b0 msgr2=0x7efeec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.446+0000 7eff07e00640 1 --2- 172.21.15.133:0/3495967502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeec0039b0 0x7efeec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.446+0000 7eff07e00640 1 -- 172.21.15.133:0/3495967502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec003520 con 0x7efeec004d40 2024-03-07T06:54:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.446+0000 7eff07400640 1 --2- 172.21.15.133:0/3495967502 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeec0039b0 0x7efeec003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.447+0000 7eff07e00640 1 --2- 172.21.15.133:0/3495967502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeec004d40 0x7efeec0051c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7eff00009ac0 tx=0x7efeec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bcb9cd6fa54baf server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.447+0000 7eff06a00640 1 -- 172.21.15.133:0/3495967502 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00039070 con 0x7efeec004d40 2024-03-07T06:54:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.447+0000 7eff06a00640 1 -- 172.21.15.133:0/3495967502 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0002c680 con 0x7efeec004d40 2024-03-07T06:54:09.451 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.447+0000 7eff06a00640 1 -- 172.21.15.133:0/3495967502 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00009690 con 0x7efeec004d40 2024-03-07T06:54:09.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.448+0000 7eff0d200640 1 -- 172.21.15.133:0/3495967502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeec004d40 msgr2=0x7efeec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.448+0000 7eff0d200640 1 --2- 172.21.15.133:0/3495967502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efeec004d40 0x7efeec0051c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7eff00009ac0 tx=0x7efeec005830 comp rx=0 tx=0).stop 2024-03-07T06:54:09.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.448+0000 7eff07e00640 1 -- 172.21.15.133:0/3495967502 reap_dead start 2024-03-07T06:54:09.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.448+0000 7eff0d200640 1 -- 172.21.15.133:0/3495967502 shutdown_connections 2024-03-07T06:54:09.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.448+0000 7eff0d200640 1 -- 172.21.15.133:0/3495967502 >> 172.21.15.133:0/3495967502 conn(0x7eff08071850 msgr2=0x7eff08071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:09.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.448+0000 7eff0d200640 1 -- 172.21.15.133:0/3495967502 shutdown_connections 2024-03-07T06:54:09.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.448+0000 7eff0d200640 1 -- 172.21.15.133:0/3495967502 wait complete. 2024-03-07T06:54:09.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.450+0000 7eff0d200640 1 Processor -- start 2024-03-07T06:54:09.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.450+0000 7eff0d200640 1 -- start start 2024-03-07T06:54:09.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.451+0000 7eff0d200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeec004d40 0x7eff08154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.451+0000 7eff0d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff0815e290 0x7eff0815e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.451+0000 7eff0d200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff080ae150 0x7eff080ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.451+0000 7eff0d200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0806f270 con 0x7eff0815e290 2024-03-07T06:54:09.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.451+0000 7eff0d200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0806d690 con 0x7efeec004d40 2024-03-07T06:54:09.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.451+0000 7eff0d200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0806e7f0 con 0x7eff080ae150 2024-03-07T06:54:09.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.451+0000 7efefea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff0815e290 0x7eff0815e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.451+0000 7efefea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff0815e290 0x7eff0815e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50282/0 (socket says 172.21.15.133:50282) 2024-03-07T06:54:09.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.451+0000 7efefea00640 1 -- 172.21.15.133:0/282938731 learned_addr learned my addr 172.21.15.133:0/282938731 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:09.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.452+0000 7eff07400640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeec004d40 0x7eff08154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.452+0000 7efefea00640 1 -- 172.21.15.133:0/282938731 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff080ae150 msgr2=0x7eff080ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.452+0000 7eff07e00640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff080ae150 0x7eff080ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.452+0000 7efefea00640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff080ae150 0x7eff080ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.452+0000 7efefea00640 1 -- 172.21.15.133:0/282938731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeec004d40 msgr2=0x7eff08154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.457 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.452+0000 7efefea00640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeec004d40 0x7eff08154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.452+0000 7efefea00640 1 -- 172.21.15.133:0/282938731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff080aa6d0 con 0x7eff0815e290 2024-03-07T06:54:09.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.452+0000 7eff07400640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efeec004d40 0x7eff08154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:09.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.452+0000 7efefea00640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff0815e290 0x7eff0815e710 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7efef0000f60 tx=0x7efef000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:09.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.452+0000 7efeffe00640 1 -- 172.21.15.133:0/282938731 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef0019070 con 0x7eff0815e290 2024-03-07T06:54:09.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.452+0000 7efeffe00640 1 -- 172.21.15.133:0/282938731 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef00074e0 con 0x7eff0815e290 2024-03-07T06:54:09.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.452+0000 7efeffe00640 1 -- 172.21.15.133:0/282938731 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef00049e0 con 0x7eff0815e290 2024-03-07T06:54:09.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.452+0000 7eff0d200640 1 -- 172.21.15.133:0/282938731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff080aa9b0 con 0x7eff0815e290 2024-03-07T06:54:09.458 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.452+0000 7eff0d200640 1 -- 172.21.15.133:0/282938731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff080aeb10 con 0x7eff0815e290 2024-03-07T06:54:09.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.453+0000 7eff0d200640 1 -- 172.21.15.133:0/282938731 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7eff080aa480 con 0x7eff0815e290 2024-03-07T06:54:09.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.454+0000 7efeffe00640 1 -- 172.21.15.133:0/282938731 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7efef001c070 con 0x7eff0815e290 2024-03-07T06:54:09.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.455+0000 7efeffe00640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7efee406b4d0 0x7efee406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.455+0000 7efeffe00640 1 -- 172.21.15.133:0/282938731 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efef00899b0 con 0x7eff0815e290 2024-03-07T06:54:09.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.456+0000 7efeffe00640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.133:6808/3182454988,v1:172.21.15.133:6809/3182454988] conn(0x7efee4071390 0x7efee40737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.456+0000 7efeffe00640 1 -- 172.21.15.133:0/282938731 --> [v2:172.21.15.133:6808/3182454988,v1:172.21.15.133:6809/3182454988] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efee4073e60 con 0x7efee4071390 2024-03-07T06:54:09.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.456+0000 7efeffe00640 1 -- 172.21.15.133:0/282938731 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7efef008e2a0 con 0x7eff0815e290 2024-03-07T06:54:09.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.456+0000 7eff07e00640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.133:6808/3182454988,v1:172.21.15.133:6809/3182454988] conn(0x7efee4071390 0x7efee40737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.456+0000 7eff07400640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7efee406b4d0 0x7efee406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.456+0000 7eff07e00640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.133:6808/3182454988,v1:172.21.15.133:6809/3182454988] conn(0x7efee4071390 0x7efee40737b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:09.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.457+0000 7eff07400640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7efee406b4d0 0x7efee406d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7eff080b0450 tx=0x7efef800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:09.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.457+0000 7efeffe00640 1 -- 172.21.15.133:0/282938731 <== osd.2 v2:172.21.15.133:6808/3182454988 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7efee4073e60 con 0x7efee4071390 2024-03-07T06:54:09.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.471+0000 7eff0d200640 1 -- 172.21.15.133:0/282938731 --> [v2:172.21.15.133:6808/3182454988,v1:172.21.15.133:6809/3182454988] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7eff08062c90 con 0x7efee4071390 2024-03-07T06:54:09.471 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.472+0000 7efeffe00640 1 -- 172.21.15.133:0/282938731 <== osd.2 v2:172.21.15.133:6808/3182454988 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7eff08062c90 con 0x7efee4071390 2024-03-07T06:54:09.472 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:54:09.472 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T06:54:09.472 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:54:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.472+0000 7eff0d200640 1 -- 172.21.15.133:0/282938731 >> [v2:172.21.15.133:6808/3182454988,v1:172.21.15.133:6809/3182454988] conn(0x7efee4071390 msgr2=0x7efee40737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.472+0000 7eff0d200640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.133:6808/3182454988,v1:172.21.15.133:6809/3182454988] conn(0x7efee4071390 0x7efee40737b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.472+0000 7eff0d200640 1 -- 172.21.15.133:0/282938731 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7efee406b4d0 msgr2=0x7efee406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.472+0000 7eff0d200640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7efee406b4d0 0x7efee406d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7eff080b0450 tx=0x7efef800b040 comp rx=0 tx=0).stop 2024-03-07T06:54:09.473 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.472+0000 7eff0d200640 1 -- 172.21.15.133:0/282938731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff0815e290 msgr2=0x7eff0815e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.473+0000 7eff0d200640 1 --2- 172.21.15.133:0/282938731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff0815e290 0x7eff0815e710 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7efef0000f60 tx=0x7efef000dc90 comp rx=0 tx=0).stop 2024-03-07T06:54:09.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.473+0000 7eff07e00640 1 -- 172.21.15.133:0/282938731 reap_dead start 2024-03-07T06:54:09.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.473+0000 7eff0d200640 1 -- 172.21.15.133:0/282938731 shutdown_connections 2024-03-07T06:54:09.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.473+0000 7eff0d200640 1 -- 172.21.15.133:0/282938731 >> 172.21.15.133:0/282938731 conn(0x7eff08071850 msgr2=0x7eff080b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:09.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.473+0000 7eff0d200640 1 -- 172.21.15.133:0/282938731 shutdown_connections 2024-03-07T06:54:09.474 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.473+0000 7eff0d200640 1 -- 172.21.15.133:0/282938731 wait complete. 2024-03-07T06:54:09.491 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-07T06:54:09.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.696+0000 7f562f400640 1 Processor -- start 2024-03-07T06:54:09.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.696+0000 7f562f400640 1 -- start start 2024-03-07T06:54:09.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.698+0000 7f562f400640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f562806dab0 0x7f5628071a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.698+0000 7f562f400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5628070400 0x7f562806de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.698+0000 7f562f400640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562815e650 con 0x7f56280a9430 2024-03-07T06:54:09.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.698+0000 7f562f400640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562806e3d0 con 0x7f5628070400 2024-03-07T06:54:09.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.698+0000 7f562f400640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5628070970 con 0x7f562806dab0 2024-03-07T06:54:09.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.698+0000 7f5627e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5628070400 0x7f562806de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.698+0000 7f5627e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f5628070400 0x7f562806de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50290/0 (socket says 172.21.15.133:50290) 2024-03-07T06:54:09.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.698+0000 7f5627e00640 1 -- 172.21.15.133:0/1703363441 learned_addr learned my addr 172.21.15.133:0/1703363441 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:09.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.698+0000 7f562ca00640 1 --2- 172.21.15.133:0/1703363441 >> v2:172.21.15.161:3300/0 conn(0x7f562806dab0 0x7f5628071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.698+0000 7f5627e00640 1 -- 172.21.15.133:0/1703363441 >> v2:172.21.15.161:3300/0 conn(0x7f562806dab0 msgr2=0x7f5628071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:09.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.698+0000 7f5627e00640 1 --2- 172.21.15.133:0/1703363441 >> v2:172.21.15.161:3300/0 conn(0x7f562806dab0 0x7f5628071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.698+0000 7f5627e00640 1 -- 172.21.15.133:0/1703363441 >> v1:172.21.15.161:6789/0 conn(0x7f56280a9430 legacy=0x7f5628161d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:09.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.698+0000 7f5627e00640 1 -- 172.21.15.133:0/1703363441 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5628070c50 con 0x7f5628070400 2024-03-07T06:54:09.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.699+0000 7f5627e00640 1 --2- 172.21.15.133:0/1703363441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5628070400 0x7f562806de90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5628069170 tx=0x7f561802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5fb6dff83fdc88 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:09.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.700+0000 7f5627400640 1 -- 172.21.15.133:0/1703363441 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561802ca80 con 0x7f5628070400 2024-03-07T06:54:09.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.700+0000 7f5627400640 1 -- 172.21.15.133:0/1703363441 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561802cc20 con 0x7f5628070400 2024-03-07T06:54:09.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.700+0000 7f5627400640 1 -- 172.21.15.133:0/1703363441 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56180343f0 con 0x7f5628070400 2024-03-07T06:54:09.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.700+0000 7f562f400640 1 -- 172.21.15.133:0/1703363441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5628070400 msgr2=0x7f562806de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.700+0000 7f562f400640 1 --2- 172.21.15.133:0/1703363441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5628070400 0x7f562806de90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5628069170 tx=0x7f561802bee0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.700+0000 7f562f400640 1 -- 172.21.15.133:0/1703363441 shutdown_connections 2024-03-07T06:54:09.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.700+0000 7f562f400640 1 --2- 172.21.15.133:0/1703363441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5628070400 0x7f562806de90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.701 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.700+0000 7f562f400640 1 --2- 172.21.15.133:0/1703363441 >> v2:172.21.15.161:3300/0 conn(0x7f562806dab0 0x7f5628071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.700+0000 7f562f400640 1 -- 172.21.15.133:0/1703363441 >> 172.21.15.133:0/1703363441 conn(0x7f56280cd160 msgr2=0x7f56280cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:09.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.701+0000 7f562f400640 1 -- 172.21.15.133:0/1703363441 shutdown_connections 2024-03-07T06:54:09.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.701+0000 7f562f400640 1 -- 172.21.15.133:0/1703363441 wait complete. 2024-03-07T06:54:09.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.702+0000 7f562f400640 1 Processor -- start 2024-03-07T06:54:09.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.702+0000 7f562f400640 1 -- start start 2024-03-07T06:54:09.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.703+0000 7f562f400640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f562806dab0 0x7f5628145360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.703+0000 7f562f400640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5628070400 0x7f5628144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.703+0000 7f562f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56280a9430 0x7f562814dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.703+0000 7f562f400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56280714f0 con 0x7f56280a9430 2024-03-07T06:54:09.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.703+0000 7f562f400640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5628071210 con 0x7f562806dab0 2024-03-07T06:54:09.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.703+0000 7f562f400640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56280673b0 con 0x7f5628070400 2024-03-07T06:54:09.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.704+0000 7f562d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56280a9430 0x7f562814dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.704+0000 7f562d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56280a9430 0x7f562814dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50294/0 (socket says 172.21.15.133:50294) 2024-03-07T06:54:09.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.704+0000 7f562d400640 1 -- 172.21.15.133:0/2449924146 learned_addr learned my addr 172.21.15.133:0/2449924146 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:09.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.704+0000 7f5627e00640 1 --2- 172.21.15.133:0/2449924146 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5628070400 0x7f5628144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.704+0000 7f562ca00640 1 --2- 172.21.15.133:0/2449924146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f562806dab0 0x7f5628145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.704+0000 7f562d400640 1 -- 172.21.15.133:0/2449924146 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5628070400 msgr2=0x7f5628144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.704+0000 7f562d400640 1 --2- 172.21.15.133:0/2449924146 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5628070400 0x7f5628144b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.704+0000 7f562d400640 1 -- 172.21.15.133:0/2449924146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f562806dab0 msgr2=0x7f5628145360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.704+0000 7f562d400640 1 --2- 172.21.15.133:0/2449924146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f562806dab0 0x7f5628145360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.704+0000 7f562d400640 1 -- 172.21.15.133:0/2449924146 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56281f7f60 con 0x7f56280a9430 2024-03-07T06:54:09.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.704+0000 7f562d400640 1 --2- 172.21.15.133:0/2449924146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56280a9430 0x7f562814dd90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f562000b3e0 tx=0x7f562000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:09.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.705+0000 7f5624c00640 1 -- 172.21.15.133:0/2449924146 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620015020 con 0x7f56280a9430 2024-03-07T06:54:09.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.705+0000 7f5624c00640 1 -- 172.21.15.133:0/2449924146 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5620011830 con 0x7f56280a9430 2024-03-07T06:54:09.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.705+0000 7f562f400640 1 -- 172.21.15.133:0/2449924146 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56281f8240 con 0x7f56280a9430 2024-03-07T06:54:09.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.705+0000 7f5624c00640 1 -- 172.21.15.133:0/2449924146 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620011c20 con 0x7f56280a9430 2024-03-07T06:54:09.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.705+0000 7f562f400640 1 -- 172.21.15.133:0/2449924146 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56281f8800 con 0x7f56280a9430 2024-03-07T06:54:09.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.706+0000 7f562f400640 1 -- 172.21.15.133:0/2449924146 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562815e880 con 0x7f56280a9430 2024-03-07T06:54:09.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.707+0000 7f5624c00640 1 -- 172.21.15.133:0/2449924146 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f5620018020 con 0x7f56280a9430 2024-03-07T06:54:09.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.708+0000 7f5624c00640 1 --2- 172.21.15.133:0/2449924146 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f560406b4d0 0x7f560406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:09.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.708+0000 7f5624c00640 1 -- 172.21.15.133:0/2449924146 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5620089d60 con 0x7f56280a9430 2024-03-07T06:54:09.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.709+0000 7f562ca00640 1 --2- 172.21.15.133:0/2449924146 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f560406b4d0 0x7f560406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:09.709 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.709+0000 7f562ca00640 1 --2- 172.21.15.133:0/2449924146 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f560406b4d0 0x7f560406d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5610006fd0 tx=0x7f5610008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:09.712 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.712+0000 7f5624c00640 1 -- 172.21.15.133:0/2449924146 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f562001f020 con 0x7f56280a9430 2024-03-07T06:54:09.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.907+0000 7f562f400640 1 -- 172.21.15.133:0/2449924146 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f56281f8a40 con 0x7f560406b4d0 2024-03-07T06:54:09.908 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.908+0000 7f5624c00640 1 -- 172.21.15.133:0/2449924146 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f561000b200 con 0x7f560406b4d0 2024-03-07T06:54:09.908 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 2 to deep-scrub 2024-03-07T06:54:09.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.910+0000 7f562f400640 1 -- 172.21.15.133:0/2449924146 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f560406b4d0 msgr2=0x7f560406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.910+0000 7f562f400640 1 --2- 172.21.15.133:0/2449924146 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f560406b4d0 0x7f560406d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5610006fd0 tx=0x7f5610008040 comp rx=0 tx=0).stop 2024-03-07T06:54:09.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.910+0000 7f562f400640 1 -- 172.21.15.133:0/2449924146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56280a9430 msgr2=0x7f562814dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:09.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.910+0000 7f562f400640 1 --2- 172.21.15.133:0/2449924146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56280a9430 0x7f562814dd90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f562000b3e0 tx=0x7f562000b8a0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.911+0000 7f562f400640 1 -- 172.21.15.133:0/2449924146 shutdown_connections 2024-03-07T06:54:09.911 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.911+0000 7f562f400640 1 --2- 172.21.15.133:0/2449924146 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56280a9430 0x7f562814dd90 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.911+0000 7f562f400640 1 --2- 172.21.15.133:0/2449924146 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5628070400 0x7f5628144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.911+0000 7f562f400640 1 --2- 172.21.15.133:0/2449924146 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f560406b4d0 0x7f560406d990 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.911+0000 7f562f400640 1 --2- 172.21.15.133:0/2449924146 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f562806dab0 0x7f5628145360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:09.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.911+0000 7f562f400640 1 -- 172.21.15.133:0/2449924146 >> 172.21.15.133:0/2449924146 conn(0x7f56280cd160 msgr2=0x7f562815c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:09.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.911+0000 7f562f400640 1 -- 172.21.15.133:0/2449924146 shutdown_connections 2024-03-07T06:54:09.912 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:09.911+0000 7f562f400640 1 -- 172.21.15.133:0/2449924146 wait complete. 2024-03-07T06:54:09.929 INFO:tasks.ceph:Scrubbing osd.3 2024-03-07T06:54:09.929 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-07T06:54:10.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.145+0000 7fc220000640 1 Processor -- start 2024-03-07T06:54:10.145 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.145+0000 7fc220000640 1 -- start start 2024-03-07T06:54:10.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.147+0000 7fc220000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc2180cc6f0 0x7fc2180b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.147+0000 7fc220000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2180b2190 con 0x7fc218070820 2024-03-07T06:54:10.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.147+0000 7fc220000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2180b2430 con 0x7fc2180cdf00 2024-03-07T06:54:10.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.147+0000 7fc220000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2180b2710 con 0x7fc2180cc6f0 2024-03-07T06:54:10.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.147+0000 7fc21e000640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc218070820 0x7fc21815fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46492/0 (socket says 172.21.15.133:46492) 2024-03-07T06:54:10.147 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.147+0000 7fc21e000640 1 -- 172.21.15.133:0/1750488345 learned_addr learned my addr 172.21.15.133:0/1750488345 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:54:10.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc21d600640 1 --2- 172.21.15.133:0/1750488345 >> v2:172.21.15.161:3300/0 conn(0x7fc2180cc6f0 0x7fc2180b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 -- 172.21.15.133:0/1750488345 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409830647 0 0) 0x7fc2180b2190 con 0x7fc218070820 2024-03-07T06:54:10.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 -- 172.21.15.133:0/1750488345 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc200003520 con 0x7fc218070820 2024-03-07T06:54:10.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 -- 172.21.15.133:0/1750488345 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3087932841 0 0) 0x7fc200003520 con 0x7fc218070820 2024-03-07T06:54:10.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 -- 172.21.15.133:0/1750488345 >> v2:172.21.15.161:3300/0 conn(0x7fc2180cc6f0 msgr2=0x7fc2180b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:10.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 --2- 172.21.15.133:0/1750488345 >> v2:172.21.15.161:3300/0 conn(0x7fc2180cc6f0 0x7fc2180b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 -- 172.21.15.133:0/1750488345 >> v1:172.21.15.161:6789/0 conn(0x7fc2180cdf00 legacy=0x7fc21815ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 -- 172.21.15.133:0/1750488345 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180b29f0 con 0x7fc218070820 2024-03-07T06:54:10.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 -- 172.21.15.133:0/1750488345 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fc210001a10 con 0x7fc218070820 2024-03-07T06:54:10.150 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 -- 172.21.15.133:0/1750488345 >> v1:172.21.15.133:6789/0 conn(0x7fc218070820 legacy=0x7fc21815fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 --2- 172.21.15.133:0/1750488345 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2000039b0 0x7fc200003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 --2- 172.21.15.133:0/1750488345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200004800 0x7fc200004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 --2- 172.21.15.133:0/1750488345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2000051c0 0x7fc20000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 -- 172.21.15.133:0/1750488345 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180b2710 con 0x7fc2000051c0 2024-03-07T06:54:10.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 -- 172.21.15.133:0/1750488345 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20000c060 con 0x7fc200004800 2024-03-07T06:54:10.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.148+0000 7fc217e00640 1 -- 172.21.15.133:0/1750488345 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20000c260 con 0x7fc2000039b0 2024-03-07T06:54:10.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.149+0000 7fc21e000640 1 --2- 172.21.15.133:0/1750488345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2000051c0 0x7fc20000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.149+0000 7fc21d600640 1 --2- 172.21.15.133:0/1750488345 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2000039b0 0x7fc200003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.149+0000 7fc21cc00640 1 --2- 172.21.15.133:0/1750488345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200004800 0x7fc200004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.149+0000 7fc21e000640 1 -- 172.21.15.133:0/1750488345 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2000039b0 msgr2=0x7fc200003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:10.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.149+0000 7fc21e000640 1 --2- 172.21.15.133:0/1750488345 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2000039b0 0x7fc200003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.149+0000 7fc21e000640 1 -- 172.21.15.133:0/1750488345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200004800 msgr2=0x7fc200004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:10.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.149+0000 7fc21e000640 1 --2- 172.21.15.133:0/1750488345 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc200004800 0x7fc200004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.149+0000 7fc21e000640 1 -- 172.21.15.133:0/1750488345 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc200003520 con 0x7fc2000051c0 2024-03-07T06:54:10.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.150+0000 7fc21e000640 1 -- 172.21.15.133:0/1750488345 reap_dead start 2024-03-07T06:54:10.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.150+0000 7fc21e000640 1 --2- 172.21.15.133:0/1750488345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2000051c0 0x7fc20000b9c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc21000a850 tx=0x7fc200005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d07ef7213d44509 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:10.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.150+0000 7fc217e00640 1 -- 172.21.15.133:0/1750488345 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21000a4a0 con 0x7fc2000051c0 2024-03-07T06:54:10.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.150+0000 7fc217e00640 1 -- 172.21.15.133:0/1750488345 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc210034650 con 0x7fc2000051c0 2024-03-07T06:54:10.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.150+0000 7fc217e00640 1 -- 172.21.15.133:0/1750488345 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210034a40 con 0x7fc2000051c0 2024-03-07T06:54:10.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.151+0000 7fc220000640 1 -- 172.21.15.133:0/1750488345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2000051c0 msgr2=0x7fc20000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.151+0000 7fc220000640 1 --2- 172.21.15.133:0/1750488345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2000051c0 0x7fc20000b9c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc21000a850 tx=0x7fc200005830 comp rx=0 tx=0).stop 2024-03-07T06:54:10.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.151+0000 7fc220000640 1 -- 172.21.15.133:0/1750488345 shutdown_connections 2024-03-07T06:54:10.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.151+0000 7fc220000640 1 --2- 172.21.15.133:0/1750488345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2000051c0 0x7fc20000b9c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.151+0000 7fc220000640 1 -- 172.21.15.133:0/1750488345 >> 172.21.15.133:0/1750488345 conn(0x7fc218066eb0 msgr2=0x7fc218067290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:10.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.151+0000 7fc220000640 1 -- 172.21.15.133:0/1750488345 shutdown_connections 2024-03-07T06:54:10.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.151+0000 7fc220000640 1 -- 172.21.15.133:0/1750488345 wait complete. 2024-03-07T06:54:10.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.153+0000 7fc220000640 1 Processor -- start 2024-03-07T06:54:10.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.153+0000 7fc220000640 1 -- start start 2024-03-07T06:54:10.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.154+0000 7fc220000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2000051c0 0x7fc21813f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.154+0000 7fc220000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2181532f0 0x7fc218155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.154+0000 7fc220000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2180b3e70 0x7fc21813e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.154+0000 7fc220000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180b1c50 con 0x7fc2181532f0 2024-03-07T06:54:10.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.154+0000 7fc220000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21815c460 con 0x7fc2000051c0 2024-03-07T06:54:10.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.154+0000 7fc220000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180b1dd0 con 0x7fc2180b3e70 2024-03-07T06:54:10.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.154+0000 7fc21cc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2181532f0 0x7fc218155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.157 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.154+0000 7fc21cc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2181532f0 0x7fc218155cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50320/0 (socket says 172.21.15.133:50320) 2024-03-07T06:54:10.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.154+0000 7fc21cc00640 1 -- 172.21.15.133:0/1311504645 learned_addr learned my addr 172.21.15.133:0/1311504645 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:10.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.155+0000 7fc21d600640 1 --2- 172.21.15.133:0/1311504645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2000051c0 0x7fc21813f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.155+0000 7fc21cc00640 1 -- 172.21.15.133:0/1311504645 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2180b3e70 msgr2=0x7fc21813e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.155+0000 7fc21cc00640 1 --2- 172.21.15.133:0/1311504645 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc2180b3e70 0x7fc21813e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.155+0000 7fc21cc00640 1 -- 172.21.15.133:0/1311504645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2000051c0 msgr2=0x7fc21813f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.155+0000 7fc21cc00640 1 --2- 172.21.15.133:0/1311504645 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc2000051c0 0x7fc21813f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.155+0000 7fc21cc00640 1 -- 172.21.15.133:0/1311504645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21813c9f0 con 0x7fc2181532f0 2024-03-07T06:54:10.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.155+0000 7fc21cc00640 1 --2- 172.21.15.133:0/1311504645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2181532f0 0x7fc218155cb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc20400e480 tx=0x7fc20400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:10.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.155+0000 7fc215600640 1 -- 172.21.15.133:0/1311504645 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2040052e0 con 0x7fc2181532f0 2024-03-07T06:54:10.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.155+0000 7fc215600640 1 -- 172.21.15.133:0/1311504645 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc204018070 con 0x7fc2181532f0 2024-03-07T06:54:10.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.155+0000 7fc215600640 1 -- 172.21.15.133:0/1311504645 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204003830 con 0x7fc2181532f0 2024-03-07T06:54:10.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.155+0000 7fc220000640 1 -- 172.21.15.133:0/1311504645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21813ccd0 con 0x7fc2181532f0 2024-03-07T06:54:10.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.155+0000 7fc220000640 1 -- 172.21.15.133:0/1311504645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2181561f0 con 0x7fc2181532f0 2024-03-07T06:54:10.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.156+0000 7fc220000640 1 -- 172.21.15.133:0/1311504645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc218062c90 con 0x7fc2181532f0 2024-03-07T06:54:10.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.157+0000 7fc215600640 1 -- 172.21.15.133:0/1311504645 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc2040039d0 con 0x7fc2181532f0 2024-03-07T06:54:10.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.158+0000 7fc215600640 1 --2- 172.21.15.133:0/1311504645 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc1f806b4d0 0x7fc1f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.158+0000 7fc215600640 1 -- 172.21.15.133:0/1311504645 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc204089c70 con 0x7fc2181532f0 2024-03-07T06:54:10.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.159+0000 7fc215600640 1 --2- 172.21.15.133:0/1311504645 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x7fc1f8071390 0x7fc1f80737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.159+0000 7fc215600640 1 -- 172.21.15.133:0/1311504645 --> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc1f8073e60 con 0x7fc1f8071390 2024-03-07T06:54:10.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.159+0000 7fc215600640 1 -- 172.21.15.133:0/1311504645 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc20408c250 con 0x7fc2181532f0 2024-03-07T06:54:10.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.159+0000 7fc21e000640 1 --2- 172.21.15.133:0/1311504645 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x7fc1f8071390 0x7fc1f80737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.159+0000 7fc21d600640 1 --2- 172.21.15.133:0/1311504645 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc1f806b4d0 0x7fc1f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.159+0000 7fc21e000640 1 --2- 172.21.15.133:0/1311504645 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x7fc1f8071390 0x7fc1f80737b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:10.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.160+0000 7fc21d600640 1 --2- 172.21.15.133:0/1311504645 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc1f806b4d0 0x7fc1f806d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0098d0 tx=0x7fc20c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:10.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.160+0000 7fc215600640 1 -- 172.21.15.133:0/1311504645 <== osd.3 v2:172.21.15.133:6816/2661528260 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc1f8073e60 con 0x7fc1f8071390 2024-03-07T06:54:10.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.175+0000 7fc220000640 1 -- 172.21.15.133:0/1311504645 --> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc21815ebb0 con 0x7fc1f8071390 2024-03-07T06:54:10.174 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.175+0000 7fc215600640 1 -- 172.21.15.133:0/1311504645 <== osd.3 v2:172.21.15.133:6816/2661528260 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc21815ebb0 con 0x7fc1f8071390 2024-03-07T06:54:10.175 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:54:10.175 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T06:54:10.175 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:54:10.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.175+0000 7fc220000640 1 -- 172.21.15.133:0/1311504645 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x7fc1f8071390 msgr2=0x7fc1f80737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.176 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.175+0000 7fc220000640 1 --2- 172.21.15.133:0/1311504645 >> [v2:172.21.15.133:6816/2661528260,v1:172.21.15.133:6817/2661528260] conn(0x7fc1f8071390 0x7fc1f80737b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.176+0000 7fc220000640 1 -- 172.21.15.133:0/1311504645 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc1f806b4d0 msgr2=0x7fc1f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.176+0000 7fc220000640 1 --2- 172.21.15.133:0/1311504645 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc1f806b4d0 0x7fc1f806d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0098d0 tx=0x7fc20c008040 comp rx=0 tx=0).stop 2024-03-07T06:54:10.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.176+0000 7fc220000640 1 -- 172.21.15.133:0/1311504645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2181532f0 msgr2=0x7fc218155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.176+0000 7fc220000640 1 --2- 172.21.15.133:0/1311504645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2181532f0 0x7fc218155cb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc20400e480 tx=0x7fc20400e940 comp rx=0 tx=0).stop 2024-03-07T06:54:10.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.176+0000 7fc21e000640 1 -- 172.21.15.133:0/1311504645 reap_dead start 2024-03-07T06:54:10.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.176+0000 7fc220000640 1 -- 172.21.15.133:0/1311504645 shutdown_connections 2024-03-07T06:54:10.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.176+0000 7fc220000640 1 -- 172.21.15.133:0/1311504645 >> 172.21.15.133:0/1311504645 conn(0x7fc218066eb0 msgr2=0x7fc218071bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:10.177 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.176+0000 7fc220000640 1 -- 172.21.15.133:0/1311504645 shutdown_connections 2024-03-07T06:54:10.178 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.176+0000 7fc220000640 1 -- 172.21.15.133:0/1311504645 wait complete. 2024-03-07T06:54:10.195 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-07T06:54:10.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.404+0000 7fc49a000640 1 Processor -- start 2024-03-07T06:54:10.405 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.405+0000 7fc49a000640 1 -- start start 2024-03-07T06:54:10.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.406+0000 7fc49a000640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc4940b5dd0 0x7fc494144540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.406+0000 7fc49a000640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc494162c80 con 0x7fc494071410 2024-03-07T06:54:10.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.406+0000 7fc49a000640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4940b4bd0 con 0x7fc4940b43f0 2024-03-07T06:54:10.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.406+0000 7fc49a000640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc494141700 con 0x7fc4940b5dd0 2024-03-07T06:54:10.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.407+0000 7fc493e00640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fc494071410 0x7fc49415e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:46500/0 (socket says 172.21.15.133:46500) 2024-03-07T06:54:10.407 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.407+0000 7fc493e00640 1 -- 172.21.15.133:0/1355460595 learned_addr learned my addr 172.21.15.133:0/1355460595 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:54:10.410 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.407+0000 7fc493400640 1 --2- 172.21.15.133:0/1355460595 >> v2:172.21.15.161:3300/0 conn(0x7fc4940b5dd0 0x7fc494144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.407+0000 7fc492000640 1 -- 172.21.15.133:0/1355460595 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076175383 0 0) 0x7fc494162c80 con 0x7fc494071410 2024-03-07T06:54:10.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.407+0000 7fc492000640 1 -- 172.21.15.133:0/1355460595 --> v1:172.21.15.133:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc478003520 con 0x7fc494071410 2024-03-07T06:54:10.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.407+0000 7fc492000640 1 -- 172.21.15.133:0/1355460595 <== mon.0 v1:172.21.15.133:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2550840108 0 0) 0x7fc478003520 con 0x7fc494071410 2024-03-07T06:54:10.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.407+0000 7fc492000640 1 -- 172.21.15.133:0/1355460595 >> v2:172.21.15.161:3300/0 conn(0x7fc4940b5dd0 msgr2=0x7fc494144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:10.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc492000640 1 --2- 172.21.15.133:0/1355460595 >> v2:172.21.15.161:3300/0 conn(0x7fc4940b5dd0 0x7fc494144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc492000640 1 -- 172.21.15.133:0/1355460595 >> v1:172.21.15.161:6789/0 conn(0x7fc4940b43f0 legacy=0x7fc494162570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.411 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc492000640 1 -- 172.21.15.133:0/1355460595 --> v1:172.21.15.133:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4940b4eb0 con 0x7fc494071410 2024-03-07T06:54:10.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc492000640 1 -- 172.21.15.133:0/1355460595 <== mon.0 v1:172.21.15.133:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1043527667 0 0) 0x7fc48c0019f0 con 0x7fc494071410 2024-03-07T06:54:10.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc492000640 1 -- 172.21.15.133:0/1355460595 >> v1:172.21.15.133:6789/0 conn(0x7fc494071410 legacy=0x7fc49415e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc492000640 1 --2- 172.21.15.133:0/1355460595 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc478003a00 0x7fc478003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc492000640 1 --2- 172.21.15.133:0/1355460595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc478004800 0x7fc478004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc492000640 1 --2- 172.21.15.133:0/1355460595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc478005210 0x7fc47800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc492000640 1 -- 172.21.15.133:0/1355460595 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc494141700 con 0x7fc478005210 2024-03-07T06:54:10.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc492000640 1 -- 172.21.15.133:0/1355460595 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47800c160 con 0x7fc478004800 2024-03-07T06:54:10.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc492000640 1 -- 172.21.15.133:0/1355460595 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47800c360 con 0x7fc478003a00 2024-03-07T06:54:10.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc493e00640 1 --2- 172.21.15.133:0/1355460595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc478005210 0x7fc47800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc493400640 1 --2- 172.21.15.133:0/1355460595 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc478003a00 0x7fc478003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc492a00640 1 --2- 172.21.15.133:0/1355460595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc478004800 0x7fc478004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.413 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc493e00640 1 -- 172.21.15.133:0/1355460595 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc478003a00 msgr2=0x7fc478003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc493e00640 1 --2- 172.21.15.133:0/1355460595 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc478003a00 0x7fc478003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc493e00640 1 -- 172.21.15.133:0/1355460595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc478004800 msgr2=0x7fc478004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc493e00640 1 --2- 172.21.15.133:0/1355460595 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc478004800 0x7fc478004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.408+0000 7fc493e00640 1 -- 172.21.15.133:0/1355460595 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc478003520 con 0x7fc478005210 2024-03-07T06:54:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.409+0000 7fc493e00640 1 -- 172.21.15.133:0/1355460595 reap_dead start 2024-03-07T06:54:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.410+0000 7fc493e00640 1 --2- 172.21.15.133:0/1355460595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc478005210 0x7fc47800b9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00a740 tx=0x7fc478005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b7db0f1c41cd84 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:10.414 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.410+0000 7fc492000640 1 -- 172.21.15.133:0/1355460595 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c00a390 con 0x7fc478005210 2024-03-07T06:54:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.410+0000 7fc492000640 1 -- 172.21.15.133:0/1355460595 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c034650 con 0x7fc478005210 2024-03-07T06:54:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.410+0000 7fc492000640 1 -- 172.21.15.133:0/1355460595 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c034a40 con 0x7fc478005210 2024-03-07T06:54:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.410+0000 7fc49a000640 1 -- 172.21.15.133:0/1355460595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc478005210 msgr2=0x7fc47800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.410+0000 7fc49a000640 1 --2- 172.21.15.133:0/1355460595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc478005210 0x7fc47800b9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00a740 tx=0x7fc478005830 comp rx=0 tx=0).stop 2024-03-07T06:54:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.410+0000 7fc49a000640 1 -- 172.21.15.133:0/1355460595 shutdown_connections 2024-03-07T06:54:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.410+0000 7fc49a000640 1 --2- 172.21.15.133:0/1355460595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc478005210 0x7fc47800b9c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.415 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.410+0000 7fc49a000640 1 -- 172.21.15.133:0/1355460595 >> 172.21.15.133:0/1355460595 conn(0x7fc4940d0e70 msgr2=0x7fc4940d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:10.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.410+0000 7fc49a000640 1 -- 172.21.15.133:0/1355460595 shutdown_connections 2024-03-07T06:54:10.416 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.411+0000 7fc49a000640 1 -- 172.21.15.133:0/1355460595 wait complete. 2024-03-07T06:54:10.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.412+0000 7fc49a000640 1 Processor -- start 2024-03-07T06:54:10.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.412+0000 7fc49a000640 1 -- start start 2024-03-07T06:54:10.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.413+0000 7fc49a000640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc478005210 0x7fc49413ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.413+0000 7fc49a000640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc494141880 0x7fc494141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.413+0000 7fc49a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc494156450 0x7fc4941568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.413+0000 7fc49a000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc494162050 con 0x7fc494156450 2024-03-07T06:54:10.419 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.413+0000 7fc49a000640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc494161e00 con 0x7fc494141880 2024-03-07T06:54:10.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.413+0000 7fc49a000640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4940b61b0 con 0x7fc478005210 2024-03-07T06:54:10.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.413+0000 7fc493e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc494156450 0x7fc4941568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.414+0000 7fc493e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc494156450 0x7fc4941568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50338/0 (socket says 172.21.15.133:50338) 2024-03-07T06:54:10.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.414+0000 7fc493e00640 1 -- 172.21.15.133:0/1829744391 learned_addr learned my addr 172.21.15.133:0/1829744391 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:10.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.414+0000 7fc492a00640 1 --2- 172.21.15.133:0/1829744391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc494141880 0x7fc494141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.414+0000 7fc493e00640 1 -- 172.21.15.133:0/1829744391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc478005210 msgr2=0x7fc49413ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.420 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.414+0000 7fc493e00640 1 --2- 172.21.15.133:0/1829744391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc478005210 0x7fc49413ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.414+0000 7fc493e00640 1 -- 172.21.15.133:0/1829744391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc494141880 msgr2=0x7fc494141d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.414+0000 7fc493e00640 1 --2- 172.21.15.133:0/1829744391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc494141880 0x7fc494141d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.414+0000 7fc493e00640 1 -- 172.21.15.133:0/1829744391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4941570f0 con 0x7fc494156450 2024-03-07T06:54:10.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.414+0000 7fc492a00640 1 --2- 172.21.15.133:0/1829744391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc494141880 0x7fc494141d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:10.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.414+0000 7fc493e00640 1 --2- 172.21.15.133:0/1829744391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc494156450 0x7fc4941568d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc48c005c20 tx=0x7fc4940b5080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:10.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.414+0000 7fc48b400640 1 -- 172.21.15.133:0/1829744391 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c034a00 con 0x7fc494156450 2024-03-07T06:54:10.421 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.414+0000 7fc48b400640 1 -- 172.21.15.133:0/1829744391 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c045070 con 0x7fc494156450 2024-03-07T06:54:10.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.414+0000 7fc49a000640 1 -- 172.21.15.133:0/1829744391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4941573d0 con 0x7fc494156450 2024-03-07T06:54:10.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.415+0000 7fc48b400640 1 -- 172.21.15.133:0/1829744391 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c034da0 con 0x7fc494156450 2024-03-07T06:54:10.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.415+0000 7fc49a000640 1 -- 172.21.15.133:0/1829744391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc494157990 con 0x7fc494156450 2024-03-07T06:54:10.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.416+0000 7fc49a000640 1 -- 172.21.15.133:0/1829744391 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc494062c90 con 0x7fc494156450 2024-03-07T06:54:10.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.417+0000 7fc48b400640 1 -- 172.21.15.133:0/1829744391 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fc48c04a460 con 0x7fc494156450 2024-03-07T06:54:10.422 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.417+0000 7fc48b400640 1 --2- 172.21.15.133:0/1829744391 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc47406b500 0x7fc47406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.418+0000 7fc48b400640 1 -- 172.21.15.133:0/1829744391 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc48c0ac430 con 0x7fc494156450 2024-03-07T06:54:10.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.418+0000 7fc493400640 1 --2- 172.21.15.133:0/1829744391 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc47406b500 0x7fc47406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.419+0000 7fc493400640 1 --2- 172.21.15.133:0/1829744391 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc47406b500 0x7fc47406d9c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc484009950 tx=0x7fc484008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:10.423 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.422+0000 7fc48b400640 1 -- 172.21.15.133:0/1829744391 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fc48c032380 con 0x7fc494156450 2024-03-07T06:54:10.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.619+0000 7fc49a000640 1 -- 172.21.15.133:0/1829744391 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4940b8a10 con 0x7fc47406b500 2024-03-07T06:54:10.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.620+0000 7fc48b400640 1 -- 172.21.15.133:0/1829744391 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc4940b8a10 con 0x7fc47406b500 2024-03-07T06:54:10.620 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 3 to deep-scrub 2024-03-07T06:54:10.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.623+0000 7fc49a000640 1 -- 172.21.15.133:0/1829744391 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc47406b500 msgr2=0x7fc47406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.623+0000 7fc49a000640 1 --2- 172.21.15.133:0/1829744391 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc47406b500 0x7fc47406d9c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc484009950 tx=0x7fc484008040 comp rx=0 tx=0).stop 2024-03-07T06:54:10.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.623+0000 7fc49a000640 1 -- 172.21.15.133:0/1829744391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc494156450 msgr2=0x7fc4941568d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.623+0000 7fc49a000640 1 --2- 172.21.15.133:0/1829744391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc494156450 0x7fc4941568d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc48c005c20 tx=0x7fc4940b5080 comp rx=0 tx=0).stop 2024-03-07T06:54:10.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.623+0000 7fc49a000640 1 -- 172.21.15.133:0/1829744391 shutdown_connections 2024-03-07T06:54:10.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.623+0000 7fc49a000640 1 --2- 172.21.15.133:0/1829744391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc494156450 0x7fc4941568d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.623+0000 7fc49a000640 1 --2- 172.21.15.133:0/1829744391 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fc47406b500 0x7fc47406d9c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.623 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.623+0000 7fc49a000640 1 --2- 172.21.15.133:0/1829744391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc494141880 0x7fc494141d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.623+0000 7fc49a000640 1 --2- 172.21.15.133:0/1829744391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc478005210 0x7fc49413ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.623+0000 7fc49a000640 1 -- 172.21.15.133:0/1829744391 >> 172.21.15.133:0/1829744391 conn(0x7fc4940d0e70 msgr2=0x7fc4940d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:10.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.623+0000 7fc49a000640 1 -- 172.21.15.133:0/1829744391 shutdown_connections 2024-03-07T06:54:10.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.623+0000 7fc49a000640 1 -- 172.21.15.133:0/1829744391 wait complete. 2024-03-07T06:54:10.643 INFO:tasks.ceph:Scrubbing osd.4 2024-03-07T06:54:10.643 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-07T06:54:10.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.857+0000 7f1e26800640 1 Processor -- start 2024-03-07T06:54:10.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.857+0000 7f1e26800640 1 -- start start 2024-03-07T06:54:10.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.859+0000 7f1e26800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1e2015be10 0x7f1e200b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.859+0000 7f1e26800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e2015c1f0 0x7f1e2006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.859+0000 7f1e26800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e2015fbf0 0x7f1e2006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.859+0000 7f1e26800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e200b64f0 con 0x7f1e2015fbf0 2024-03-07T06:54:10.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.859+0000 7f1e26800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e200b67d0 con 0x7f1e2015be10 2024-03-07T06:54:10.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.859+0000 7f1e26800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e200b6ab0 con 0x7f1e2015c1f0 2024-03-07T06:54:10.859 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.859+0000 7f1e1fe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e2015fbf0 0x7f1e2006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.859+0000 7f1e1fe00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f1e2015fbf0 0x7f1e2006ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60814/0 (socket says 172.21.15.133:60814) 2024-03-07T06:54:10.860 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.859+0000 7f1e1fe00640 1 -- 172.21.15.133:0/1830894507 learned_addr learned my addr 172.21.15.133:0/1830894507 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.860+0000 7f1e1ea00640 1 --2- 172.21.15.133:0/1830894507 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e2015c1f0 0x7f1e2006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.860+0000 7f1e1f400640 1 --2- 172.21.15.133:0/1830894507 >> v2:172.21.15.161:3300/0 conn(0x7f1e2015be10 0x7f1e200b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.860+0000 7f1e1fe00640 1 -- 172.21.15.133:0/1830894507 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e2015c1f0 msgr2=0x7f1e2006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.860+0000 7f1e1fe00640 1 --2- 172.21.15.133:0/1830894507 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e2015c1f0 0x7f1e2006b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.861 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.860+0000 7f1e1fe00640 1 -- 172.21.15.133:0/1830894507 >> v2:172.21.15.161:3300/0 conn(0x7f1e2015be10 msgr2=0x7f1e200b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:10.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.860+0000 7f1e1fe00640 1 --2- 172.21.15.133:0/1830894507 >> v2:172.21.15.161:3300/0 conn(0x7f1e2015be10 0x7f1e200b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.860+0000 7f1e1fe00640 1 -- 172.21.15.133:0/1830894507 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e200b6d90 con 0x7f1e2015fbf0 2024-03-07T06:54:10.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.860+0000 7f1e1f400640 1 --2- 172.21.15.133:0/1830894507 >> v2:172.21.15.161:3300/0 conn(0x7f1e2015be10 0x7f1e200b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:10.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.861+0000 7f1e1fe00640 1 --2- 172.21.15.133:0/1830894507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e2015fbf0 0x7f1e2006ba20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1e200d2f30 tx=0x7f1e1802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c49179eb438a6a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:10.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.861+0000 7f1e1e000640 1 -- 172.21.15.133:0/1830894507 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e18035020 con 0x7f1e2015fbf0 2024-03-07T06:54:10.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.861+0000 7f1e1e000640 1 -- 172.21.15.133:0/1830894507 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e18031870 con 0x7f1e2015fbf0 2024-03-07T06:54:10.862 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.861+0000 7f1e1e000640 1 -- 172.21.15.133:0/1830894507 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e18031c40 con 0x7f1e2015fbf0 2024-03-07T06:54:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.862+0000 7f1e26800640 1 -- 172.21.15.133:0/1830894507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e2015fbf0 msgr2=0x7f1e2006ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.862+0000 7f1e26800640 1 --2- 172.21.15.133:0/1830894507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e2015fbf0 0x7f1e2006ba20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1e200d2f30 tx=0x7f1e1802b910 comp rx=0 tx=0).stop 2024-03-07T06:54:10.863 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.862+0000 7f1e26800640 1 -- 172.21.15.133:0/1830894507 shutdown_connections 2024-03-07T06:54:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.862+0000 7f1e26800640 1 --2- 172.21.15.133:0/1830894507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e2015fbf0 0x7f1e2006ba20 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.862+0000 7f1e26800640 1 --2- 172.21.15.133:0/1830894507 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e2015c1f0 0x7f1e2006b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.862+0000 7f1e26800640 1 --2- 172.21.15.133:0/1830894507 >> v2:172.21.15.161:3300/0 conn(0x7f1e2015be10 0x7f1e200b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.862+0000 7f1e26800640 1 -- 172.21.15.133:0/1830894507 >> 172.21.15.133:0/1830894507 conn(0x7f1e2006d930 msgr2=0x7f1e20066eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.862+0000 7f1e26800640 1 -- 172.21.15.133:0/1830894507 shutdown_connections 2024-03-07T06:54:10.864 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.862+0000 7f1e26800640 1 -- 172.21.15.133:0/1830894507 wait complete. 2024-03-07T06:54:10.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.864+0000 7f1e26800640 1 Processor -- start 2024-03-07T06:54:10.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.864+0000 7f1e26800640 1 -- start start 2024-03-07T06:54:10.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e26800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e2015be10 0x7f1e201504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e26800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e2015c1f0 0x7f1e20152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e26800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e2015fbf0 0x7f1e201453d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e26800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e200b3a90 con 0x7f1e2015fbf0 2024-03-07T06:54:10.867 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e26800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e200d2cb0 con 0x7f1e2015c1f0 2024-03-07T06:54:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e26800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e200b3c10 con 0x7f1e2015be10 2024-03-07T06:54:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e1fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e2015fbf0 0x7f1e201453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e1fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e2015fbf0 0x7f1e201453d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60824/0 (socket says 172.21.15.133:60824) 2024-03-07T06:54:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e1fe00640 1 -- 172.21.15.133:0/3291602141 learned_addr learned my addr 172.21.15.133:0/3291602141 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e1ea00640 1 --2- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e2015c1f0 0x7f1e20152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e1f400640 1 --2- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e2015be10 0x7f1e201504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e1fe00640 1 -- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e2015be10 msgr2=0x7f1e201504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e1fe00640 1 --2- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e2015be10 0x7f1e201504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.868 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e1fe00640 1 -- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e2015c1f0 msgr2=0x7f1e20152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e1fe00640 1 --2- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e2015c1f0 0x7f1e20152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.866+0000 7f1e1fe00640 1 -- 172.21.15.133:0/3291602141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e20152e80 con 0x7f1e2015fbf0 2024-03-07T06:54:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.867+0000 7f1e1fe00640 1 --2- 172.21.15.133:0/3291602141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e2015fbf0 0x7f1e201453d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1e180090b0 tx=0x7f1e180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.867+0000 7f1e17400640 1 -- 172.21.15.133:0/3291602141 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e18040070 con 0x7f1e2015fbf0 2024-03-07T06:54:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.867+0000 7f1e17400640 1 -- 172.21.15.133:0/3291602141 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e18030c50 con 0x7f1e2015fbf0 2024-03-07T06:54:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.867+0000 7f1e17400640 1 -- 172.21.15.133:0/3291602141 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e18035040 con 0x7f1e2015fbf0 2024-03-07T06:54:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.867+0000 7f1e26800640 1 -- 172.21.15.133:0/3291602141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e20153160 con 0x7f1e2015fbf0 2024-03-07T06:54:10.869 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.867+0000 7f1e26800640 1 -- 172.21.15.133:0/3291602141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e20153720 con 0x7f1e2015fbf0 2024-03-07T06:54:10.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.868+0000 7f1e26800640 1 -- 172.21.15.133:0/3291602141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e2015ca30 con 0x7f1e2015fbf0 2024-03-07T06:54:10.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.869+0000 7f1e17400640 1 -- 172.21.15.133:0/3291602141 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f1e1804b060 con 0x7f1e2015fbf0 2024-03-07T06:54:10.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.870+0000 7f1e17400640 1 --2- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1dfc06b4d0 0x7f1dfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.870+0000 7f1e17400640 1 -- 172.21.15.133:0/3291602141 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1e180abd50 con 0x7f1e2015fbf0 2024-03-07T06:54:10.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.871+0000 7f1e17400640 1 --2- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:6826/3482024257,v1:172.21.15.161:6827/3482024257] conn(0x7f1dfc071390 0x7f1dfc0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:10.871 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.871+0000 7f1e17400640 1 -- 172.21.15.133:0/3291602141 --> [v2:172.21.15.161:6826/3482024257,v1:172.21.15.161:6827/3482024257] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e1804ee50 con 0x7f1dfc071390 2024-03-07T06:54:10.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.871+0000 7f1e17400640 1 -- 172.21.15.133:0/3291602141 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e180ae370 con 0x7f1e2015fbf0 2024-03-07T06:54:10.872 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.871+0000 7f1e1f400640 1 --2- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1dfc06b4d0 0x7f1dfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.871+0000 7f1e1ea00640 1 --2- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:6826/3482024257,v1:172.21.15.161:6827/3482024257] conn(0x7f1dfc071390 0x7f1dfc0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:10.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.872+0000 7f1e1f400640 1 --2- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1dfc06b4d0 0x7f1dfc06d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1e10008f10 tx=0x7f1e10008460 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:10.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.872+0000 7f1e1ea00640 1 --2- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:6826/3482024257,v1:172.21.15.161:6827/3482024257] conn(0x7f1dfc071390 0x7f1dfc0737b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:10.874 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.874+0000 7f1e17400640 1 -- 172.21.15.133:0/3291602141 <== osd.4 v2:172.21.15.161:6826/3482024257 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1e1804ee50 con 0x7f1dfc071390 2024-03-07T06:54:10.888 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.889+0000 7f1e26800640 1 -- 172.21.15.133:0/3291602141 --> [v2:172.21.15.161:6826/3482024257,v1:172.21.15.161:6827/3482024257] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1e20062c90 con 0x7f1dfc071390 2024-03-07T06:54:10.890 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.890+0000 7f1e17400640 1 -- 172.21.15.133:0/3291602141 <== osd.4 v2:172.21.15.161:6826/3482024257 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1e20062c90 con 0x7f1dfc071390 2024-03-07T06:54:10.890 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:54:10.890 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T06:54:10.890 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:54:10.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.891+0000 7f1e26800640 1 -- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:6826/3482024257,v1:172.21.15.161:6827/3482024257] conn(0x7f1dfc071390 msgr2=0x7f1dfc0737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.891 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.891+0000 7f1e26800640 1 --2- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:6826/3482024257,v1:172.21.15.161:6827/3482024257] conn(0x7f1dfc071390 0x7f1dfc0737b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.891+0000 7f1e26800640 1 -- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1dfc06b4d0 msgr2=0x7f1dfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.891+0000 7f1e26800640 1 --2- 172.21.15.133:0/3291602141 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f1dfc06b4d0 0x7f1dfc06d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1e10008f10 tx=0x7f1e10008460 comp rx=0 tx=0).stop 2024-03-07T06:54:10.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.891+0000 7f1e26800640 1 -- 172.21.15.133:0/3291602141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e2015fbf0 msgr2=0x7f1e201453d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:10.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.891+0000 7f1e26800640 1 --2- 172.21.15.133:0/3291602141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e2015fbf0 0x7f1e201453d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1e180090b0 tx=0x7f1e180046c0 comp rx=0 tx=0).stop 2024-03-07T06:54:10.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.891+0000 7f1e1fe00640 1 -- 172.21.15.133:0/3291602141 reap_dead start 2024-03-07T06:54:10.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.891+0000 7f1e26800640 1 -- 172.21.15.133:0/3291602141 shutdown_connections 2024-03-07T06:54:10.892 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.891+0000 7f1e26800640 1 -- 172.21.15.133:0/3291602141 >> 172.21.15.133:0/3291602141 conn(0x7f1e2006d930 msgr2=0x7f1e201600d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:10.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.891+0000 7f1e26800640 1 -- 172.21.15.133:0/3291602141 shutdown_connections 2024-03-07T06:54:10.893 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:10.891+0000 7f1e26800640 1 -- 172.21.15.133:0/3291602141 wait complete. 2024-03-07T06:54:10.908 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-07T06:54:11.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.118+0000 7fbef9600640 1 Processor -- start 2024-03-07T06:54:11.118 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.118+0000 7fbef9600640 1 -- start start 2024-03-07T06:54:11.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.120+0000 7fbef9600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbef4165ed0 0x7fbef406ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.120+0000 7fbef9600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbef4071d20 0x7fbef4162f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.120+0000 7fbef9600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef40ce540 con 0x7fbef40b3c10 2024-03-07T06:54:11.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.120+0000 7fbef9600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef40734a0 con 0x7fbef4071d20 2024-03-07T06:54:11.120 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.120+0000 7fbef9600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef406f200 con 0x7fbef4165ed0 2024-03-07T06:54:11.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.120+0000 7fbef3400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fbef4071d20 0x7fbef4162f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.120+0000 7fbeea000640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7fbef40b3c10 0x7fbef4071610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33984/0 (socket says 172.21.15.133:33984) 2024-03-07T06:54:11.121 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.120+0000 7fbeea000640 1 -- 172.21.15.133:0/3842182814 learned_addr learned my addr 172.21.15.133:0/3842182814 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:54:11.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.121+0000 7fbef2a00640 1 --2- 172.21.15.133:0/3842182814 >> v2:172.21.15.161:3300/0 conn(0x7fbef4165ed0 0x7fbef406ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.121+0000 7fbef2000640 1 -- 172.21.15.133:0/3842182814 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738838065 0 0) 0x7fbef40ce540 con 0x7fbef40b3c10 2024-03-07T06:54:11.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.121+0000 7fbef3400640 1 -- 172.21.15.133:0/3842182814 >> v2:172.21.15.161:3300/0 conn(0x7fbef4165ed0 msgr2=0x7fbef406ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:11.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.121+0000 7fbef3400640 1 --2- 172.21.15.133:0/3842182814 >> v2:172.21.15.161:3300/0 conn(0x7fbef4165ed0 0x7fbef406ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.121+0000 7fbef3400640 1 -- 172.21.15.133:0/3842182814 >> v1:172.21.15.133:6789/0 conn(0x7fbef40b3c10 legacy=0x7fbef4071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.121+0000 7fbef3400640 1 -- 172.21.15.133:0/3842182814 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40cdf80 con 0x7fbef4071d20 2024-03-07T06:54:11.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.122+0000 7fbef3400640 1 --2- 172.21.15.133:0/3842182814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef4071d20 0x7fbef4162f90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbef40693e0 tx=0x7fbeec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7d643bfc8b0160 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:11.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.122+0000 7fbef2000640 1 -- 172.21.15.133:0/3842182814 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec035020 con 0x7fbef4071d20 2024-03-07T06:54:11.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.122+0000 7fbef2000640 1 -- 172.21.15.133:0/3842182814 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec031830 con 0x7fbef4071d20 2024-03-07T06:54:11.123 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.122+0000 7fbef2000640 1 -- 172.21.15.133:0/3842182814 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec031c00 con 0x7fbef4071d20 2024-03-07T06:54:11.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.123+0000 7fbef9600640 1 -- 172.21.15.133:0/3842182814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef4071d20 msgr2=0x7fbef4162f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.123+0000 7fbef9600640 1 --2- 172.21.15.133:0/3842182814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef4071d20 0x7fbef4162f90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fbef40693e0 tx=0x7fbeec02b8d0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.124 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.123+0000 7fbef9600640 1 -- 172.21.15.133:0/3842182814 shutdown_connections 2024-03-07T06:54:11.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.123+0000 7fbef9600640 1 --2- 172.21.15.133:0/3842182814 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef4071d20 0x7fbef4162f90 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.123+0000 7fbef9600640 1 --2- 172.21.15.133:0/3842182814 >> v2:172.21.15.161:3300/0 conn(0x7fbef4165ed0 0x7fbef406ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.123+0000 7fbef9600640 1 -- 172.21.15.133:0/3842182814 >> 172.21.15.133:0/3842182814 conn(0x7fbef40b3120 msgr2=0x7fbef40b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:11.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.123+0000 7fbef9600640 1 -- 172.21.15.133:0/3842182814 shutdown_connections 2024-03-07T06:54:11.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.123+0000 7fbef9600640 1 -- 172.21.15.133:0/3842182814 wait complete. 2024-03-07T06:54:11.125 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.125+0000 7fbef9600640 1 Processor -- start 2024-03-07T06:54:11.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.125+0000 7fbef9600640 1 -- start start 2024-03-07T06:54:11.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.126+0000 7fbef9600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbef4071d20 0x7fbef415ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.127 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.126+0000 7fbef9600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbef40b3c10 0x7fbef415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.126+0000 7fbef9600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef4165ed0 0x7fbef415ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.126+0000 7fbef9600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4072550 con 0x7fbef4165ed0 2024-03-07T06:54:11.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.126+0000 7fbef9600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4072270 con 0x7fbef40b3c10 2024-03-07T06:54:11.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.126+0000 7fbef9600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef41634d0 con 0x7fbef4071d20 2024-03-07T06:54:11.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.126+0000 7fbef3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef4165ed0 0x7fbef415ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.126+0000 7fbef3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef4165ed0 0x7fbef415ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60852/0 (socket says 172.21.15.133:60852) 2024-03-07T06:54:11.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.126+0000 7fbef3400640 1 -- 172.21.15.133:0/1703020846 learned_addr learned my addr 172.21.15.133:0/1703020846 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:11.128 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.127+0000 7fbeea000640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbef40b3c10 0x7fbef415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.127+0000 7fbef2a00640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbef4071d20 0x7fbef415ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.127+0000 7fbef3400640 1 -- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbef4071d20 msgr2=0x7fbef415ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.127+0000 7fbef3400640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbef4071d20 0x7fbef415ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.127+0000 7fbef3400640 1 -- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbef40b3c10 msgr2=0x7fbef415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.127+0000 7fbef3400640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbef40b3c10 0x7fbef415bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.127+0000 7fbef3400640 1 -- 172.21.15.133:0/1703020846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef41ecb50 con 0x7fbef4165ed0 2024-03-07T06:54:11.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.127+0000 7fbeea000640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbef40b3c10 0x7fbef415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:11.129 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.127+0000 7fbef2a00640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbef4071d20 0x7fbef415ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:11.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.127+0000 7fbef3400640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef4165ed0 0x7fbef415ae90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009890 tx=0x7fbeec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:11.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.127+0000 7fbeeb400640 1 -- 172.21.15.133:0/1703020846 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec035070 con 0x7fbef4165ed0 2024-03-07T06:54:11.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.128+0000 7fbeeb400640 1 -- 172.21.15.133:0/1703020846 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec004230 con 0x7fbef4165ed0 2024-03-07T06:54:11.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.128+0000 7fbeeb400640 1 -- 172.21.15.133:0/1703020846 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec03a6f0 con 0x7fbef4165ed0 2024-03-07T06:54:11.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.128+0000 7fbef9600640 1 -- 172.21.15.133:0/1703020846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef415c230 con 0x7fbef4165ed0 2024-03-07T06:54:11.130 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.128+0000 7fbef9600640 1 -- 172.21.15.133:0/1703020846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef41ed0d0 con 0x7fbef4165ed0 2024-03-07T06:54:11.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.129+0000 7fbef9600640 1 -- 172.21.15.133:0/1703020846 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef4062c90 con 0x7fbef4165ed0 2024-03-07T06:54:11.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.130+0000 7fbeeb400640 1 -- 172.21.15.133:0/1703020846 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fbeec038020 con 0x7fbef4165ed0 2024-03-07T06:54:11.131 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.131+0000 7fbeeb400640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbed006b4d0 0x7fbed006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.131+0000 7fbeeb400640 1 -- 172.21.15.133:0/1703020846 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbeec0abb90 con 0x7fbef4165ed0 2024-03-07T06:54:11.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.131+0000 7fbef2a00640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbed006b4d0 0x7fbed006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.132 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.132+0000 7fbef2a00640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbed006b4d0 0x7fbed006d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009f50 tx=0x7fbee400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:11.135 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.135+0000 7fbeeb400640 1 -- 172.21.15.133:0/1703020846 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fbeec040070 con 0x7fbef4165ed0 2024-03-07T06:54:11.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.330+0000 7fbef9600640 1 -- 172.21.15.133:0/1703020846 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbef41627f0 con 0x7fbed006b4d0 2024-03-07T06:54:11.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.332+0000 7fbeeb400640 1 -- 172.21.15.133:0/1703020846 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbef41627f0 con 0x7fbed006b4d0 2024-03-07T06:54:11.331 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 4 to deep-scrub 2024-03-07T06:54:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.334+0000 7fbef9600640 1 -- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbed006b4d0 msgr2=0x7fbed006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.334+0000 7fbef9600640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbed006b4d0 0x7fbed006d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009f50 tx=0x7fbee400b040 comp rx=0 tx=0).stop 2024-03-07T06:54:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.334+0000 7fbef9600640 1 -- 172.21.15.133:0/1703020846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef4165ed0 msgr2=0x7fbef415ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.334 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.334+0000 7fbef9600640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef4165ed0 0x7fbef415ae90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbeec009890 tx=0x7fbeec0046c0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.334+0000 7fbef9600640 1 -- 172.21.15.133:0/1703020846 shutdown_connections 2024-03-07T06:54:11.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.334+0000 7fbef9600640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbef4165ed0 0x7fbef415ae90 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.334+0000 7fbef9600640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fbed006b4d0 0x7fbed006d990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.334+0000 7fbef9600640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbef40b3c10 0x7fbef415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.334+0000 7fbef9600640 1 --2- 172.21.15.133:0/1703020846 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbef4071d20 0x7fbef415ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.334+0000 7fbef9600640 1 -- 172.21.15.133:0/1703020846 >> 172.21.15.133:0/1703020846 conn(0x7fbef40b3120 msgr2=0x7fbef406e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:11.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.334+0000 7fbef9600640 1 -- 172.21.15.133:0/1703020846 shutdown_connections 2024-03-07T06:54:11.335 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.335+0000 7fbef9600640 1 -- 172.21.15.133:0/1703020846 wait complete. 2024-03-07T06:54:11.353 INFO:tasks.ceph:Scrubbing osd.5 2024-03-07T06:54:11.354 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-07T06:54:11.567 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.568+0000 7f8aa4600640 1 Processor -- start 2024-03-07T06:54:11.568 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.568+0000 7f8aa4600640 1 -- start start 2024-03-07T06:54:11.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.569+0000 7f8aa4600640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8a9c0b1c50 0x7f8a9c0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.569+0000 7f8aa4600640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8a9c15e8f0 0x7f8a9c0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.569 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.569+0000 7f8aa4600640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a9c0b2570 con 0x7f8a9c0b52a0 2024-03-07T06:54:11.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.569+0000 7f8aa4600640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9c15f800 con 0x7f8a9c0b1c50 2024-03-07T06:54:11.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.569+0000 7f8aa4600640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9c15f240 con 0x7f8a9c15e8f0 2024-03-07T06:54:11.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.570+0000 7f8aa1c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8a9c0b1c50 0x7f8a9c0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.570+0000 7f8aa1c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f8a9c0b1c50 0x7f8a9c0b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60860/0 (socket says 172.21.15.133:60860) 2024-03-07T06:54:11.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.570+0000 7f8aa1200640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f8a9c0b52a0 0x7f8a9c15bc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:33994/0 (socket says 172.21.15.133:33994) 2024-03-07T06:54:11.570 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.570+0000 7f8aa1c00640 1 -- 172.21.15.133:0/1876556061 learned_addr learned my addr 172.21.15.133:0/1876556061 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:11.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.570+0000 7f8aa1c00640 1 -- 172.21.15.133:0/1876556061 >> v2:172.21.15.161:3300/0 conn(0x7f8a9c15e8f0 msgr2=0x7f8a9c0b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:11.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.570+0000 7f8aa2600640 1 --2- 172.21.15.133:0/1876556061 >> v2:172.21.15.161:3300/0 conn(0x7f8a9c15e8f0 0x7f8a9c0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.571 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.570+0000 7f8aa1c00640 1 --2- 172.21.15.133:0/1876556061 >> v2:172.21.15.161:3300/0 conn(0x7f8a9c15e8f0 0x7f8a9c0b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.570+0000 7f8aa1c00640 1 -- 172.21.15.133:0/1876556061 >> v1:172.21.15.133:6789/0 conn(0x7f8a9c0b52a0 legacy=0x7f8a9c15bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.570+0000 7f8aa1c00640 1 -- 172.21.15.133:0/1876556061 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c15f520 con 0x7f8a9c0b1c50 2024-03-07T06:54:11.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.570+0000 7f8aa2600640 1 --2- 172.21.15.133:0/1876556061 >> v2:172.21.15.161:3300/0 conn(0x7f8a9c15e8f0 0x7f8a9c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:11.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.571+0000 7f8aa1c00640 1 --2- 172.21.15.133:0/1876556061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9c0b1c50 0x7f8a9c0b2030 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0cd3d0 tx=0x7f8a9002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9793a398dd66fc40 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:11.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.572+0000 7f8a9be00640 1 -- 172.21.15.133:0/1876556061 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90034070 con 0x7f8a9c0b1c50 2024-03-07T06:54:11.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.572+0000 7f8a9be00640 1 -- 172.21.15.133:0/1876556061 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a90030860 con 0x7f8a9c0b1c50 2024-03-07T06:54:11.572 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.572+0000 7f8a9be00640 1 -- 172.21.15.133:0/1876556061 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90030c30 con 0x7f8a9c0b1c50 2024-03-07T06:54:11.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.572+0000 7f8aa4600640 1 -- 172.21.15.133:0/1876556061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9c0b1c50 msgr2=0x7f8a9c0b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.572+0000 7f8aa4600640 1 --2- 172.21.15.133:0/1876556061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9c0b1c50 0x7f8a9c0b2030 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0cd3d0 tx=0x7f8a9002b9a0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.573+0000 7f8aa4600640 1 -- 172.21.15.133:0/1876556061 shutdown_connections 2024-03-07T06:54:11.573 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.573+0000 7f8aa4600640 1 --2- 172.21.15.133:0/1876556061 >> v2:172.21.15.161:3300/0 conn(0x7f8a9c15e8f0 0x7f8a9c0b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.573+0000 7f8aa4600640 1 --2- 172.21.15.133:0/1876556061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9c0b1c50 0x7f8a9c0b2030 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.573+0000 7f8aa4600640 1 -- 172.21.15.133:0/1876556061 >> 172.21.15.133:0/1876556061 conn(0x7f8a9c066eb0 msgr2=0x7f8a9c067290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:11.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.573+0000 7f8aa4600640 1 -- 172.21.15.133:0/1876556061 shutdown_connections 2024-03-07T06:54:11.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.573+0000 7f8aa4600640 1 -- 172.21.15.133:0/1876556061 wait complete. 2024-03-07T06:54:11.574 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.574+0000 7f8aa4600640 1 Processor -- start 2024-03-07T06:54:11.575 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.575+0000 7f8aa4600640 1 -- start start 2024-03-07T06:54:11.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.575+0000 7f8aa4600640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a9c0b1c50 0x7f8a9c0aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.575+0000 7f8aa4600640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a9c0b52a0 0x7f8a9c15c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.575+0000 7f8aa4600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9c15e8f0 0x7f8a9c15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.575+0000 7f8aa4600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c15afd0 con 0x7f8a9c15e8f0 2024-03-07T06:54:11.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.575+0000 7f8aa4600640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c15b6f0 con 0x7f8a9c0b52a0 2024-03-07T06:54:11.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.575+0000 7f8aa4600640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c15b150 con 0x7f8a9c0b1c50 2024-03-07T06:54:11.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.576+0000 7f8aa2600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9c15e8f0 0x7f8a9c15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.576+0000 7f8aa2600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9c15e8f0 0x7f8a9c15cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60864/0 (socket says 172.21.15.133:60864) 2024-03-07T06:54:11.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.576+0000 7f8aa2600640 1 -- 172.21.15.133:0/19951808 learned_addr learned my addr 172.21.15.133:0/19951808 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:11.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.576+0000 7f8aa1200640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a9c0b52a0 0x7f8a9c15c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.576+0000 7f8aa1c00640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a9c0b1c50 0x7f8a9c0aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.576+0000 7f8aa2600640 1 -- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a9c0b1c50 msgr2=0x7f8a9c0aaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.576+0000 7f8aa2600640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a9c0b1c50 0x7f8a9c0aaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.576+0000 7f8aa2600640 1 -- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a9c0b52a0 msgr2=0x7f8a9c15c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.576+0000 7f8aa2600640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a9c0b52a0 0x7f8a9c15c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.576+0000 7f8aa2600640 1 -- 172.21.15.133:0/19951808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c0b0d10 con 0x7f8a9c15e8f0 2024-03-07T06:54:11.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.576+0000 7f8aa1c00640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8a9c0b1c50 0x7f8a9c0aaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:11.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.576+0000 7f8aa1200640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8a9c0b52a0 0x7f8a9c15c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:11.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.577+0000 7f8aa2600640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9c15e8f0 0x7f8a9c15cba0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400cca0 tx=0x7f8a9400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:11.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.577+0000 7f8a99600640 1 -- 172.21.15.133:0/19951808 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9400f680 con 0x7f8a9c15e8f0 2024-03-07T06:54:11.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.577+0000 7f8a99600640 1 -- 172.21.15.133:0/19951808 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9400f840 con 0x7f8a9c15e8f0 2024-03-07T06:54:11.579 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.577+0000 7f8a99600640 1 -- 172.21.15.133:0/19951808 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9400fc30 con 0x7f8a9c15e8f0 2024-03-07T06:54:11.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.577+0000 7f8aa4600640 1 -- 172.21.15.133:0/19951808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9c0b0ff0 con 0x7f8a9c15e8f0 2024-03-07T06:54:11.580 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.577+0000 7f8aa4600640 1 -- 172.21.15.133:0/19951808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9c0ae1a0 con 0x7f8a9c15e8f0 2024-03-07T06:54:11.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.578+0000 7f8aa4600640 1 -- 172.21.15.133:0/19951808 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8a9c06eca0 con 0x7f8a9c15e8f0 2024-03-07T06:54:11.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.579+0000 7f8a99600640 1 -- 172.21.15.133:0/19951808 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f8a9400e3e0 con 0x7f8a9c15e8f0 2024-03-07T06:54:11.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.580+0000 7f8a99600640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8a7c06b4d0 0x7f8a7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.580+0000 7f8a99600640 1 -- 172.21.15.133:0/19951808 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8a9408ac00 con 0x7f8a9c15e8f0 2024-03-07T06:54:11.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.580+0000 7f8a99600640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:6818/1085452100,v1:172.21.15.161:6819/1085452100] conn(0x7f8a7c071390 0x7f8a7c0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.580+0000 7f8a99600640 1 -- 172.21.15.133:0/19951808 --> [v2:172.21.15.161:6818/1085452100,v1:172.21.15.161:6819/1085452100] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a7c073e60 con 0x7f8a7c071390 2024-03-07T06:54:11.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.580+0000 7f8a99600640 1 -- 172.21.15.133:0/19951808 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a9408d300 con 0x7f8a9c15e8f0 2024-03-07T06:54:11.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.581+0000 7f8aa1c00640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8a7c06b4d0 0x7f8a7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.581+0000 7f8aa1200640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:6818/1085452100,v1:172.21.15.161:6819/1085452100] conn(0x7f8a7c071390 0x7f8a7c0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.582+0000 7f8aa1c00640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8a7c06b4d0 0x7f8a7c06d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8a90036000 tx=0x7f8a9003b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:11.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.582+0000 7f8aa1200640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:6818/1085452100,v1:172.21.15.161:6819/1085452100] conn(0x7f8a7c071390 0x7f8a7c0737b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:11.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.584+0000 7f8a99600640 1 -- 172.21.15.133:0/19951808 <== osd.5 v2:172.21.15.161:6818/1085452100 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8a7c073e60 con 0x7f8a7c071390 2024-03-07T06:54:11.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.598+0000 7f8aa4600640 1 -- 172.21.15.133:0/19951808 --> [v2:172.21.15.161:6818/1085452100,v1:172.21.15.161:6819/1085452100] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8a9c062c90 con 0x7f8a7c071390 2024-03-07T06:54:11.599 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.600+0000 7f8a99600640 1 -- 172.21.15.133:0/19951808 <== osd.5 v2:172.21.15.161:6818/1085452100 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8a9c062c90 con 0x7f8a7c071390 2024-03-07T06:54:11.599 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:54:11.600 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T06:54:11.600 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:54:11.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.600+0000 7f8aa4600640 1 -- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:6818/1085452100,v1:172.21.15.161:6819/1085452100] conn(0x7f8a7c071390 msgr2=0x7f8a7c0737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.600+0000 7f8aa4600640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:6818/1085452100,v1:172.21.15.161:6819/1085452100] conn(0x7f8a7c071390 0x7f8a7c0737b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.600+0000 7f8aa4600640 1 -- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8a7c06b4d0 msgr2=0x7f8a7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.600+0000 7f8aa4600640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f8a7c06b4d0 0x7f8a7c06d990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8a90036000 tx=0x7f8a9003b040 comp rx=0 tx=0).stop 2024-03-07T06:54:11.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.600+0000 7f8aa4600640 1 -- 172.21.15.133:0/19951808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9c15e8f0 msgr2=0x7f8a9c15cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.600+0000 7f8aa4600640 1 --2- 172.21.15.133:0/19951808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9c15e8f0 0x7f8a9c15cba0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400cca0 tx=0x7f8a9400a650 comp rx=0 tx=0).stop 2024-03-07T06:54:11.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.601+0000 7f8aa2600640 1 -- 172.21.15.133:0/19951808 reap_dead start 2024-03-07T06:54:11.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.601+0000 7f8aa4600640 1 -- 172.21.15.133:0/19951808 shutdown_connections 2024-03-07T06:54:11.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.601+0000 7f8aa4600640 1 -- 172.21.15.133:0/19951808 >> 172.21.15.133:0/19951808 conn(0x7f8a9c066eb0 msgr2=0x7f8a9c0b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:11.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.601+0000 7f8aa4600640 1 -- 172.21.15.133:0/19951808 shutdown_connections 2024-03-07T06:54:11.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.601+0000 7f8aa4600640 1 -- 172.21.15.133:0/19951808 wait complete. 2024-03-07T06:54:11.619 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-07T06:54:11.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.825+0000 7f7c56800640 1 Processor -- start 2024-03-07T06:54:11.825 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.826+0000 7f7c56800640 1 -- start start 2024-03-07T06:54:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.827+0000 7f7c56800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7c50147bb0 0x7f7c500d0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.827+0000 7f7c56800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7c50073020 0x7f7c500d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.827+0000 7f7c56800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c5015fc30 con 0x7f7c500b5090 2024-03-07T06:54:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.827+0000 7f7c56800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c50161ce0 con 0x7f7c50147bb0 2024-03-07T06:54:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.827+0000 7f7c56800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5015ff10 con 0x7f7c50073020 2024-03-07T06:54:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.828+0000 7f7c4f400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7c50147bb0 0x7f7c500d0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.828+0000 7f7c4f400640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7c50147bb0 0x7f7c500d0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60872/0 (socket says 172.21.15.133:60872) 2024-03-07T06:54:11.828 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.828+0000 7f7c4f400640 1 -- 172.21.15.133:0/2806214525 learned_addr learned my addr 172.21.15.133:0/2806214525 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.828+0000 7f7c4f400640 1 -- 172.21.15.133:0/2806214525 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7c50073020 msgr2=0x7f7c500d1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.828+0000 7f7c4f400640 1 --2- 172.21.15.133:0/2806214525 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7c50073020 0x7f7c500d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.828+0000 7f7c4f400640 1 -- 172.21.15.133:0/2806214525 >> v1:172.21.15.161:6789/0 conn(0x7f7c500b5090 legacy=0x7f7c50072910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:11.829 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.828+0000 7f7c4f400640 1 -- 172.21.15.133:0/2806214525 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c501601f0 con 0x7f7c50147bb0 2024-03-07T06:54:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.829+0000 7f7c4f400640 1 --2- 172.21.15.133:0/2806214525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c50147bb0 0x7f7c500d0b50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009870 tx=0x7f7c4002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ef31e65db0dc79 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.829+0000 7f7c4ea00640 1 -- 172.21.15.133:0/2806214525 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c400317c0 con 0x7f7c50147bb0 2024-03-07T06:54:11.830 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.829+0000 7f7c4ea00640 1 -- 172.21.15.133:0/2806214525 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c40037070 con 0x7f7c50147bb0 2024-03-07T06:54:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.829+0000 7f7c4ea00640 1 -- 172.21.15.133:0/2806214525 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c40031b80 con 0x7f7c50147bb0 2024-03-07T06:54:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.830+0000 7f7c56800640 1 -- 172.21.15.133:0/2806214525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c50147bb0 msgr2=0x7f7c500d0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.830+0000 7f7c56800640 1 --2- 172.21.15.133:0/2806214525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c50147bb0 0x7f7c500d0b50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009870 tx=0x7f7c4002b860 comp rx=0 tx=0).stop 2024-03-07T06:54:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.830+0000 7f7c56800640 1 -- 172.21.15.133:0/2806214525 shutdown_connections 2024-03-07T06:54:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.830+0000 7f7c56800640 1 --2- 172.21.15.133:0/2806214525 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7c50073020 0x7f7c500d1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.830+0000 7f7c56800640 1 --2- 172.21.15.133:0/2806214525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c50147bb0 0x7f7c500d0b50 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.830+0000 7f7c56800640 1 -- 172.21.15.133:0/2806214525 >> 172.21.15.133:0/2806214525 conn(0x7f7c5006b480 msgr2=0x7f7c5006b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:11.831 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.830+0000 7f7c56800640 1 -- 172.21.15.133:0/2806214525 shutdown_connections 2024-03-07T06:54:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.830+0000 7f7c56800640 1 -- 172.21.15.133:0/2806214525 wait complete. 2024-03-07T06:54:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.832+0000 7f7c56800640 1 Processor -- start 2024-03-07T06:54:11.832 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.832+0000 7f7c56800640 1 -- start start 2024-03-07T06:54:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.833+0000 7f7c56800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7c50073020 0x7f7c501f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.833+0000 7f7c56800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c500b5090 0x7f7c500d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.833+0000 7f7c56800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c50147bb0 0x7f7c500d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.833+0000 7f7c56800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c50147f90 con 0x7f7c50147bb0 2024-03-07T06:54:11.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.833+0000 7f7c56800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c501607b0 con 0x7f7c500b5090 2024-03-07T06:54:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.833+0000 7f7c56800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c50160cd0 con 0x7f7c50073020 2024-03-07T06:54:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.833+0000 7f7c4fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c50147bb0 0x7f7c500d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.833+0000 7f7c4fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c50147bb0 0x7f7c500d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60884/0 (socket says 172.21.15.133:60884) 2024-03-07T06:54:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.833+0000 7f7c4fe00640 1 -- 172.21.15.133:0/3865033521 learned_addr learned my addr 172.21.15.133:0/3865033521 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.834+0000 7f7c4f400640 1 --2- 172.21.15.133:0/3865033521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7c50073020 0x7f7c501f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.834+0000 7f7c4fe00640 1 -- 172.21.15.133:0/3865033521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7c50073020 msgr2=0x7f7c501f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.834+0000 7f7c4fe00640 1 --2- 172.21.15.133:0/3865033521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7c50073020 0x7f7c501f39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.834+0000 7f7c4fe00640 1 -- 172.21.15.133:0/3865033521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c500b5090 msgr2=0x7f7c500d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.834+0000 7f7c4fe00640 1 --2- 172.21.15.133:0/3865033521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c500b5090 0x7f7c500d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.834+0000 7f7c4fe00640 1 -- 172.21.15.133:0/3865033521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5013b5b0 con 0x7f7c50147bb0 2024-03-07T06:54:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.834+0000 7f7c4fe00640 1 --2- 172.21.15.133:0/3865033521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c50147bb0 0x7f7c500d0cf0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800b9e0 tx=0x7f7c4800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.834+0000 7f7c47400640 1 -- 172.21.15.133:0/3865033521 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48019070 con 0x7f7c50147bb0 2024-03-07T06:54:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.835+0000 7f7c47400640 1 -- 172.21.15.133:0/3865033521 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4800cb40 con 0x7f7c50147bb0 2024-03-07T06:54:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.835+0000 7f7c47400640 1 -- 172.21.15.133:0/3865033521 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48014440 con 0x7f7c50147bb0 2024-03-07T06:54:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.835+0000 7f7c56800640 1 -- 172.21.15.133:0/3865033521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5013b890 con 0x7f7c50147bb0 2024-03-07T06:54:11.836 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.835+0000 7f7c56800640 1 -- 172.21.15.133:0/3865033521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5013be50 con 0x7f7c50147bb0 2024-03-07T06:54:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.836+0000 7f7c56800640 1 -- 172.21.15.133:0/3865033521 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c50062c90 con 0x7f7c50147bb0 2024-03-07T06:54:11.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.837+0000 7f7c47400640 1 -- 172.21.15.133:0/3865033521 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f7c4801c070 con 0x7f7c50147bb0 2024-03-07T06:54:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.837+0000 7f7c47400640 1 --2- 172.21.15.133:0/3865033521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7c2c06b4d0 0x7f7c2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.838+0000 7f7c47400640 1 -- 172.21.15.133:0/3865033521 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7c48089cd0 con 0x7f7c50147bb0 2024-03-07T06:54:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.838+0000 7f7c4f400640 1 --2- 172.21.15.133:0/3865033521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7c2c06b4d0 0x7f7c2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:11.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.839+0000 7f7c4f400640 1 --2- 172.21.15.133:0/3865033521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7c2c06b4d0 0x7f7c2c06d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7c40036000 tx=0x7f7c40008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:11.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:11.842+0000 7f7c47400640 1 -- 172.21.15.133:0/3865033521 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f7c48058db0 con 0x7f7c50147bb0 2024-03-07T06:54:12.035 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.036+0000 7f7c56800640 1 -- 172.21.15.133:0/3865033521 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c5015e8d0 con 0x7f7c2c06b4d0 2024-03-07T06:54:12.036 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.037+0000 7f7c47400640 1 -- 172.21.15.133:0/3865033521 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7c5015e8d0 con 0x7f7c2c06b4d0 2024-03-07T06:54:12.036 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 5 to deep-scrub 2024-03-07T06:54:12.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.039+0000 7f7c56800640 1 -- 172.21.15.133:0/3865033521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7c2c06b4d0 msgr2=0x7f7c2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.039+0000 7f7c56800640 1 --2- 172.21.15.133:0/3865033521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7c2c06b4d0 0x7f7c2c06d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7c40036000 tx=0x7f7c40008030 comp rx=0 tx=0).stop 2024-03-07T06:54:12.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.039+0000 7f7c56800640 1 -- 172.21.15.133:0/3865033521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c50147bb0 msgr2=0x7f7c500d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.039 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.039+0000 7f7c56800640 1 --2- 172.21.15.133:0/3865033521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c50147bb0 0x7f7c500d0cf0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800b9e0 tx=0x7f7c4800bea0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.039+0000 7f7c56800640 1 -- 172.21.15.133:0/3865033521 shutdown_connections 2024-03-07T06:54:12.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.039+0000 7f7c56800640 1 --2- 172.21.15.133:0/3865033521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c50147bb0 0x7f7c500d0cf0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.039+0000 7f7c56800640 1 --2- 172.21.15.133:0/3865033521 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f7c2c06b4d0 0x7f7c2c06d990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.039+0000 7f7c56800640 1 --2- 172.21.15.133:0/3865033521 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7c500b5090 0x7f7c500d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.039+0000 7f7c56800640 1 --2- 172.21.15.133:0/3865033521 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7c50073020 0x7f7c501f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.039+0000 7f7c56800640 1 -- 172.21.15.133:0/3865033521 >> 172.21.15.133:0/3865033521 conn(0x7f7c5006b480 msgr2=0x7f7c5015edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:12.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.040+0000 7f7c56800640 1 -- 172.21.15.133:0/3865033521 shutdown_connections 2024-03-07T06:54:12.040 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.040+0000 7f7c56800640 1 -- 172.21.15.133:0/3865033521 wait complete. 2024-03-07T06:54:12.059 INFO:tasks.ceph:Scrubbing osd.6 2024-03-07T06:54:12.059 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-07T06:54:12.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.262+0000 7f7925e00640 1 Processor -- start 2024-03-07T06:54:12.262 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.262+0000 7f7925e00640 1 -- start start 2024-03-07T06:54:12.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.264+0000 7f7925e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7920147800 0x7f792015f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.264+0000 7f7925e00640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f792015f5b0 con 0x7f7920070820 2024-03-07T06:54:12.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.264+0000 7f7925e00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79200b2a70 con 0x7f7920070cb0 2024-03-07T06:54:12.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.264+0000 7f7925e00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7920162bb0 con 0x7f7920147800 2024-03-07T06:54:12.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.264+0000 7f791ea00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7920147800 0x7f792015f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.264 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.264+0000 7f791ea00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f7920147800 0x7f792015f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60892/0 (socket says 172.21.15.133:60892) 2024-03-07T06:54:12.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.264+0000 7f791ea00640 1 -- 172.21.15.133:0/2502685164 learned_addr learned my addr 172.21.15.133:0/2502685164 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:12.265 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.265+0000 7f791ea00640 1 -- 172.21.15.133:0/2502685164 >> v1:172.21.15.161:6789/0 conn(0x7f7920070cb0 legacy=0x7f79200b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:12.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.265+0000 7f791ea00640 1 -- 172.21.15.133:0/2502685164 >> v1:172.21.15.133:6789/0 conn(0x7f7920070820 legacy=0x7f79200b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.265+0000 7f791ea00640 1 -- 172.21.15.133:0/2502685164 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200b3f20 con 0x7f7920147800 2024-03-07T06:54:12.266 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.266+0000 7f791ea00640 1 --2- 172.21.15.133:0/2502685164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7920147800 0x7f792015f070 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7910009870 tx=0x7f791002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845e1264669cd039 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:12.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.266+0000 7f791e000640 1 -- 172.21.15.133:0/2502685164 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910034070 con 0x7f7920147800 2024-03-07T06:54:12.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.266+0000 7f791e000640 1 -- 172.21.15.133:0/2502685164 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7910030860 con 0x7f7920147800 2024-03-07T06:54:12.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.267+0000 7f791e000640 1 -- 172.21.15.133:0/2502685164 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910030c30 con 0x7f7920147800 2024-03-07T06:54:12.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.267+0000 7f7925e00640 1 -- 172.21.15.133:0/2502685164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7920147800 msgr2=0x7f792015f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.267 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.267+0000 7f7925e00640 1 --2- 172.21.15.133:0/2502685164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7920147800 0x7f792015f070 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7910009870 tx=0x7f791002b9a0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.267+0000 7f7925e00640 1 -- 172.21.15.133:0/2502685164 shutdown_connections 2024-03-07T06:54:12.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.267+0000 7f7925e00640 1 --2- 172.21.15.133:0/2502685164 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7920147800 0x7f792015f070 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.267+0000 7f7925e00640 1 -- 172.21.15.133:0/2502685164 >> 172.21.15.133:0/2502685164 conn(0x7f7920066eb0 msgr2=0x7f7920067290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:12.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.267+0000 7f7925e00640 1 -- 172.21.15.133:0/2502685164 shutdown_connections 2024-03-07T06:54:12.268 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.267+0000 7f7925e00640 1 -- 172.21.15.133:0/2502685164 wait complete. 2024-03-07T06:54:12.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.269+0000 7f7925e00640 1 Processor -- start 2024-03-07T06:54:12.269 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.269+0000 7f7925e00640 1 -- start start 2024-03-07T06:54:12.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.270+0000 7f7925e00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7920070820 0x7f79200b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.271 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.270+0000 7f7925e00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7920070cb0 0x7f792013fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.270+0000 7f7925e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7920147800 0x7f7920140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.270+0000 7f7925e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7920162030 con 0x7f7920147800 2024-03-07T06:54:12.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.270+0000 7f7925e00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79200aea70 con 0x7f7920070820 2024-03-07T06:54:12.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.270+0000 7f7925e00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7920162310 con 0x7f7920070cb0 2024-03-07T06:54:12.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.270+0000 7f791f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7920147800 0x7f7920140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.271+0000 7f791f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7920147800 0x7f7920140100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60902/0 (socket says 172.21.15.133:60902) 2024-03-07T06:54:12.272 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.271+0000 7f791f400640 1 -- 172.21.15.133:0/398871315 learned_addr learned my addr 172.21.15.133:0/398871315 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:12.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.271+0000 7f791ea00640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7920070820 0x7f79200b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.271+0000 7f7917e00640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7920070cb0 0x7f792013fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.271+0000 7f791f400640 1 -- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7920070cb0 msgr2=0x7f792013fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.271+0000 7f791f400640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7920070cb0 0x7f792013fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.271+0000 7f791f400640 1 -- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7920070820 msgr2=0x7f79200b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.271+0000 7f791f400640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7920070820 0x7f79200b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.271+0000 7f791f400640 1 -- 172.21.15.133:0/398871315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200b14f0 con 0x7f7920147800 2024-03-07T06:54:12.273 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.271+0000 7f791ea00640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7920070820 0x7f79200b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:12.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.271+0000 7f791f400640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7920147800 0x7f7920140100 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f791800c7b0 tx=0x7f791800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:12.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.272+0000 7f7917400640 1 -- 172.21.15.133:0/398871315 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791800ce60 con 0x7f7920147800 2024-03-07T06:54:12.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.272+0000 7f7917400640 1 -- 172.21.15.133:0/398871315 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7918004b10 con 0x7f7920147800 2024-03-07T06:54:12.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.272+0000 7f7917400640 1 -- 172.21.15.133:0/398871315 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791800a850 con 0x7f7920147800 2024-03-07T06:54:12.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.272+0000 7f7925e00640 1 -- 172.21.15.133:0/398871315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79200b17d0 con 0x7f7920147800 2024-03-07T06:54:12.274 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.272+0000 7f7925e00640 1 -- 172.21.15.133:0/398871315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79200b1d90 con 0x7f7920147800 2024-03-07T06:54:12.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.273+0000 7f7925e00640 1 -- 172.21.15.133:0/398871315 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7920062c90 con 0x7f7920147800 2024-03-07T06:54:12.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.274+0000 7f7917400640 1 -- 172.21.15.133:0/398871315 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f791800a9f0 con 0x7f7920147800 2024-03-07T06:54:12.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.275+0000 7f7917400640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f78fc06b500 0x7f78fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.275+0000 7f7917400640 1 -- 172.21.15.133:0/398871315 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f791808a240 con 0x7f7920147800 2024-03-07T06:54:12.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.275+0000 7f7917400640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:6802/2969599292,v1:172.21.15.161:6803/2969599292] conn(0x7f78fc0713c0 0x7f78fc0737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.275+0000 7f7917400640 1 -- 172.21.15.133:0/398871315 --> [v2:172.21.15.161:6802/2969599292,v1:172.21.15.161:6803/2969599292] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7918014e70 con 0x7f78fc0713c0 2024-03-07T06:54:12.276 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.275+0000 7f7917400640 1 -- 172.21.15.133:0/398871315 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7918091050 con 0x7f7920147800 2024-03-07T06:54:12.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.275+0000 7f791ea00640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f78fc06b500 0x7f78fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.276+0000 7f7917e00640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:6802/2969599292,v1:172.21.15.161:6803/2969599292] conn(0x7f78fc0713c0 0x7f78fc0737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.277 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.277+0000 7f7917e00640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:6802/2969599292,v1:172.21.15.161:6803/2969599292] conn(0x7f78fc0713c0 0x7f78fc0737e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:12.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.277+0000 7f791ea00640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f78fc06b500 0x7f78fc06d9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7910036000 tx=0x7f791003b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:12.278 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.279+0000 7f7917400640 1 -- 172.21.15.133:0/398871315 <== osd.6 v2:172.21.15.161:6802/2969599292 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7918014e70 con 0x7f78fc0713c0 2024-03-07T06:54:12.292 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.293+0000 7f7925e00640 1 -- 172.21.15.133:0/398871315 --> [v2:172.21.15.161:6802/2969599292,v1:172.21.15.161:6803/2969599292] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f79200ae5a0 con 0x7f78fc0713c0 2024-03-07T06:54:12.293 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.294+0000 7f7917400640 1 -- 172.21.15.133:0/398871315 <== osd.6 v2:172.21.15.161:6802/2969599292 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f79200ae5a0 con 0x7f78fc0713c0 2024-03-07T06:54:12.294 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:54:12.294 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T06:54:12.294 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:54:12.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.294+0000 7f7925e00640 1 -- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:6802/2969599292,v1:172.21.15.161:6803/2969599292] conn(0x7f78fc0713c0 msgr2=0x7f78fc0737e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.294+0000 7f7925e00640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:6802/2969599292,v1:172.21.15.161:6803/2969599292] conn(0x7f78fc0713c0 0x7f78fc0737e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.294+0000 7f7925e00640 1 -- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f78fc06b500 msgr2=0x7f78fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.295 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.294+0000 7f7925e00640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f78fc06b500 0x7f78fc06d9c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7910036000 tx=0x7f791003b040 comp rx=0 tx=0).stop 2024-03-07T06:54:12.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.294+0000 7f7925e00640 1 -- 172.21.15.133:0/398871315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7920147800 msgr2=0x7f7920140100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.294+0000 7f7925e00640 1 --2- 172.21.15.133:0/398871315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7920147800 0x7f7920140100 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f791800c7b0 tx=0x7f791800cc70 comp rx=0 tx=0).stop 2024-03-07T06:54:12.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.295+0000 7f791f400640 1 -- 172.21.15.133:0/398871315 reap_dead start 2024-03-07T06:54:12.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.295+0000 7f7925e00640 1 -- 172.21.15.133:0/398871315 shutdown_connections 2024-03-07T06:54:12.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.295+0000 7f7925e00640 1 -- 172.21.15.133:0/398871315 >> 172.21.15.133:0/398871315 conn(0x7f7920066eb0 msgr2=0x7f792015ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:12.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.295+0000 7f7925e00640 1 -- 172.21.15.133:0/398871315 shutdown_connections 2024-03-07T06:54:12.296 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.295+0000 7f7925e00640 1 -- 172.21.15.133:0/398871315 wait complete. 2024-03-07T06:54:12.313 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-07T06:54:12.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.526+0000 7fa465800640 1 Processor -- start 2024-03-07T06:54:12.526 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.526+0000 7fa465800640 1 -- start start 2024-03-07T06:54:12.527 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.528+0000 7fa465800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa46015b1b0 0x7fa4600b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.528+0000 7fa465800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa4600af950 0x7fa4600cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.528+0000 7fa465800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa4600ce420 0x7fa46006de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.528+0000 7fa465800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa460071610 con 0x7fa4600af950 2024-03-07T06:54:12.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.528+0000 7fa465800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4600ce850 con 0x7fa4600ce420 2024-03-07T06:54:12.528 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.528+0000 7fa465800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46006f140 con 0x7fa46015b1b0 2024-03-07T06:54:12.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.528+0000 7fa456000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa4600af950 0x7fa4600cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.528+0000 7fa456000640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fa4600af950 0x7fa4600cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60910/0 (socket says 172.21.15.133:60910) 2024-03-07T06:54:12.529 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.528+0000 7fa456000640 1 -- 172.21.15.133:0/95346454 learned_addr learned my addr 172.21.15.133:0/95346454 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:12.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.529+0000 7fa45ea00640 1 --2- 172.21.15.133:0/95346454 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa46015b1b0 0x7fa4600b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.529+0000 7fa45f400640 1 --2- 172.21.15.133:0/95346454 >> v2:172.21.15.161:3300/0 conn(0x7fa4600ce420 0x7fa46006de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.529+0000 7fa456000640 1 -- 172.21.15.133:0/95346454 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa46015b1b0 msgr2=0x7fa4600b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:12.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.529+0000 7fa456000640 1 --2- 172.21.15.133:0/95346454 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa46015b1b0 0x7fa4600b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.529+0000 7fa456000640 1 -- 172.21.15.133:0/95346454 >> v2:172.21.15.161:3300/0 conn(0x7fa4600ce420 msgr2=0x7fa46006de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:12.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.529+0000 7fa456000640 1 --2- 172.21.15.133:0/95346454 >> v2:172.21.15.161:3300/0 conn(0x7fa4600ce420 0x7fa46006de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.530 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.529+0000 7fa456000640 1 -- 172.21.15.133:0/95346454 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46006f420 con 0x7fa4600af950 2024-03-07T06:54:12.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.529+0000 7fa45ea00640 1 --2- 172.21.15.133:0/95346454 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa46015b1b0 0x7fa4600b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:12.531 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.530+0000 7fa456000640 1 --2- 172.21.15.133:0/95346454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4600af950 0x7fa4600cdee0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa4600b0450 tx=0x7fa44802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1113bb67089a33f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:12.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.530+0000 7fa45e000640 1 -- 172.21.15.133:0/95346454 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44802e070 con 0x7fa4600af950 2024-03-07T06:54:12.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.530+0000 7fa45e000640 1 -- 172.21.15.133:0/95346454 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa448037070 con 0x7fa4600af950 2024-03-07T06:54:12.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.530+0000 7fa45e000640 1 -- 172.21.15.133:0/95346454 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448031a00 con 0x7fa4600af950 2024-03-07T06:54:12.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.531+0000 7fa465800640 1 -- 172.21.15.133:0/95346454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4600af950 msgr2=0x7fa4600cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.532 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.531+0000 7fa465800640 1 --2- 172.21.15.133:0/95346454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4600af950 0x7fa4600cdee0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa4600b0450 tx=0x7fa44802b880 comp rx=0 tx=0).stop 2024-03-07T06:54:12.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.531+0000 7fa465800640 1 -- 172.21.15.133:0/95346454 shutdown_connections 2024-03-07T06:54:12.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.531+0000 7fa465800640 1 --2- 172.21.15.133:0/95346454 >> v2:172.21.15.161:3300/0 conn(0x7fa4600ce420 0x7fa46006de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.531+0000 7fa465800640 1 --2- 172.21.15.133:0/95346454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa4600af950 0x7fa4600cdee0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.531+0000 7fa465800640 1 --2- 172.21.15.133:0/95346454 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa46015b1b0 0x7fa4600b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.531+0000 7fa465800640 1 -- 172.21.15.133:0/95346454 >> 172.21.15.133:0/95346454 conn(0x7fa4600b3120 msgr2=0x7fa4600b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:12.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.531+0000 7fa465800640 1 -- 172.21.15.133:0/95346454 shutdown_connections 2024-03-07T06:54:12.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.531+0000 7fa465800640 1 -- 172.21.15.133:0/95346454 wait complete. 2024-03-07T06:54:12.533 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.533+0000 7fa465800640 1 Processor -- start 2024-03-07T06:54:12.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.533+0000 7fa465800640 1 -- start start 2024-03-07T06:54:12.535 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.534+0000 7fa465800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4600af950 0x7fa4601f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.534+0000 7fa465800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4600ce420 0x7fa4601488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.534+0000 7fa465800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa46015b1b0 0x7fa46015df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.534+0000 7fa465800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46015b590 con 0x7fa46015b1b0 2024-03-07T06:54:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.534+0000 7fa465800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4600cc880 con 0x7fa4600af950 2024-03-07T06:54:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.534+0000 7fa465800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa46015b710 con 0x7fa4600ce420 2024-03-07T06:54:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.534+0000 7fa45f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa46015b1b0 0x7fa46015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.534+0000 7fa45f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa46015b1b0 0x7fa46015df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60918/0 (socket says 172.21.15.133:60918) 2024-03-07T06:54:12.536 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.534+0000 7fa45f400640 1 -- 172.21.15.133:0/632167688 learned_addr learned my addr 172.21.15.133:0/632167688 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:12.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.535+0000 7fa456000640 1 --2- 172.21.15.133:0/632167688 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4600ce420 0x7fa4601488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.535+0000 7fa45ea00640 1 --2- 172.21.15.133:0/632167688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4600af950 0x7fa4601f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.535+0000 7fa45f400640 1 -- 172.21.15.133:0/632167688 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4600ce420 msgr2=0x7fa4601488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.535+0000 7fa45f400640 1 --2- 172.21.15.133:0/632167688 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4600ce420 0x7fa4601488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.535+0000 7fa45f400640 1 -- 172.21.15.133:0/632167688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4600af950 msgr2=0x7fa4601f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.535+0000 7fa45f400640 1 --2- 172.21.15.133:0/632167688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4600af950 0x7fa4601f4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.535+0000 7fa45f400640 1 -- 172.21.15.133:0/632167688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4600b4570 con 0x7fa46015b1b0 2024-03-07T06:54:12.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.535+0000 7fa45f400640 1 --2- 172.21.15.133:0/632167688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa46015b1b0 0x7fa46015df80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa45800ccf0 tx=0x7fa45800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:12.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.535+0000 7fa457400640 1 -- 172.21.15.133:0/632167688 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458003b90 con 0x7fa46015b1b0 2024-03-07T06:54:12.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.535+0000 7fa457400640 1 -- 172.21.15.133:0/632167688 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa458003d30 con 0x7fa46015b1b0 2024-03-07T06:54:12.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.535+0000 7fa465800640 1 -- 172.21.15.133:0/632167688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4600b4850 con 0x7fa46015b1b0 2024-03-07T06:54:12.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.535+0000 7fa457400640 1 -- 172.21.15.133:0/632167688 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458004240 con 0x7fa46015b1b0 2024-03-07T06:54:12.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.536+0000 7fa465800640 1 -- 172.21.15.133:0/632167688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4600b4e10 con 0x7fa46015b1b0 2024-03-07T06:54:12.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.537+0000 7fa465800640 1 -- 172.21.15.133:0/632167688 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa460062c90 con 0x7fa46015b1b0 2024-03-07T06:54:12.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.538+0000 7fa457400640 1 -- 172.21.15.133:0/632167688 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fa4580043e0 con 0x7fa46015b1b0 2024-03-07T06:54:12.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.538+0000 7fa457400640 1 --2- 172.21.15.133:0/632167688 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa43c06b4d0 0x7fa43c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.539+0000 7fa457400640 1 -- 172.21.15.133:0/632167688 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa45808aaa0 con 0x7fa46015b1b0 2024-03-07T06:54:12.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.539+0000 7fa45ea00640 1 --2- 172.21.15.133:0/632167688 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa43c06b4d0 0x7fa43c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.540 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.540+0000 7fa45ea00640 1 --2- 172.21.15.133:0/632167688 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa43c06b4d0 0x7fa43c06d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa45000a950 tx=0x7fa450008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:12.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.543+0000 7fa457400640 1 -- 172.21.15.133:0/632167688 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fa458059b80 con 0x7fa46015b1b0 2024-03-07T06:54:12.740 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.740+0000 7fa465800640 1 -- 172.21.15.133:0/632167688 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fa460155ff0 con 0x7fa43c06b4d0 2024-03-07T06:54:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.741+0000 7fa457400640 1 -- 172.21.15.133:0/632167688 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa460155ff0 con 0x7fa43c06b4d0 2024-03-07T06:54:12.741 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 6 to deep-scrub 2024-03-07T06:54:12.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.745+0000 7fa465800640 1 -- 172.21.15.133:0/632167688 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa43c06b4d0 msgr2=0x7fa43c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.745+0000 7fa465800640 1 --2- 172.21.15.133:0/632167688 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa43c06b4d0 0x7fa43c06d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa45000a950 tx=0x7fa450008040 comp rx=0 tx=0).stop 2024-03-07T06:54:12.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.745+0000 7fa465800640 1 -- 172.21.15.133:0/632167688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa46015b1b0 msgr2=0x7fa46015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.745 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.745+0000 7fa465800640 1 --2- 172.21.15.133:0/632167688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa46015b1b0 0x7fa46015df80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa45800ccf0 tx=0x7fa45800a660 comp rx=0 tx=0).stop 2024-03-07T06:54:12.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.745+0000 7fa465800640 1 -- 172.21.15.133:0/632167688 shutdown_connections 2024-03-07T06:54:12.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.745+0000 7fa465800640 1 --2- 172.21.15.133:0/632167688 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa46015b1b0 0x7fa46015df80 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.745+0000 7fa465800640 1 --2- 172.21.15.133:0/632167688 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4600ce420 0x7fa4601488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.745+0000 7fa465800640 1 --2- 172.21.15.133:0/632167688 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fa43c06b4d0 0x7fa43c06d990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.745+0000 7fa465800640 1 --2- 172.21.15.133:0/632167688 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4600af950 0x7fa4601f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.745+0000 7fa465800640 1 -- 172.21.15.133:0/632167688 >> 172.21.15.133:0/632167688 conn(0x7fa4600b3120 msgr2=0x7fa4600b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:12.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.745+0000 7fa465800640 1 -- 172.21.15.133:0/632167688 shutdown_connections 2024-03-07T06:54:12.746 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.745+0000 7fa465800640 1 -- 172.21.15.133:0/632167688 wait complete. 2024-03-07T06:54:12.764 INFO:tasks.ceph:Scrubbing osd.7 2024-03-07T06:54:12.764 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-07T06:54:12.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.976+0000 7f28ff800640 1 Processor -- start 2024-03-07T06:54:12.976 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.976+0000 7f28ff800640 1 -- start start 2024-03-07T06:54:12.977 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.978+0000 7f28ff800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f28f80cdf00 0x7f28f815ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.978+0000 7f28ff800640 1 -- --> v1:172.21.15.133:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f80b2430 con 0x7f28f8070820 2024-03-07T06:54:12.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.978+0000 7f28ff800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f80b2710 con 0x7f28f80cc6f0 2024-03-07T06:54:12.978 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.978+0000 7f28ff800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f80b29f0 con 0x7f28f80cdf00 2024-03-07T06:54:12.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.978+0000 7f28fd800640 1 --1- >> v1:172.21.15.133:6789/0 conn(0x7f28f8070820 0x7f28f815fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.133:6789/0 says I am v1:172.21.15.133:34014/0 (socket says 172.21.15.133:34014) 2024-03-07T06:54:12.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.978+0000 7f28fd800640 1 -- 172.21.15.133:0/293745711 learned_addr learned my addr 172.21.15.133:0/293745711 (peer_addr_for_me v1:172.21.15.133:0/0) 2024-03-07T06:54:12.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.978+0000 7f28f7e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f28f80cdf00 0x7f28f815ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.979+0000 7f28f7e00640 1 -- 172.21.15.133:0/293745711 >> v1:172.21.15.161:6789/0 conn(0x7f28f80cc6f0 legacy=0x7f28f80b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:12.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.979+0000 7f28f7e00640 1 -- 172.21.15.133:0/293745711 >> v1:172.21.15.133:6789/0 conn(0x7f28f8070820 legacy=0x7f28f815fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.979 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.979+0000 7f28f7e00640 1 -- 172.21.15.133:0/293745711 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f815c7b0 con 0x7f28f80cdf00 2024-03-07T06:54:12.980 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.979+0000 7f28f7400640 1 -- 172.21.15.133:0/293745711 <== mon.0 v1:172.21.15.133:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115834211 0 0) 0x7f28f80b2430 con 0x7f28f8070820 2024-03-07T06:54:12.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.980+0000 7f28f7e00640 1 --2- 172.21.15.133:0/293745711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28f80cdf00 0x7f28f815ccd0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f28e80097a0 tx=0x7f28e802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef11895714755c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:12.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.980+0000 7f28f7400640 1 -- 172.21.15.133:0/293745711 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8035020 con 0x7f28f80cdf00 2024-03-07T06:54:12.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.980+0000 7f28f7400640 1 -- 172.21.15.133:0/293745711 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e8031830 con 0x7f28f80cdf00 2024-03-07T06:54:12.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.980+0000 7f28f7400640 1 -- 172.21.15.133:0/293745711 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8031c00 con 0x7f28f80cdf00 2024-03-07T06:54:12.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.981+0000 7f28ff800640 1 -- 172.21.15.133:0/293745711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28f80cdf00 msgr2=0x7f28f815ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.981 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.981+0000 7f28ff800640 1 --2- 172.21.15.133:0/293745711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28f80cdf00 0x7f28f815ccd0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f28e80097a0 tx=0x7f28e802b8d0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.981+0000 7f28ff800640 1 -- 172.21.15.133:0/293745711 shutdown_connections 2024-03-07T06:54:12.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.981+0000 7f28ff800640 1 --2- 172.21.15.133:0/293745711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28f80cdf00 0x7f28f815ccd0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.981+0000 7f28ff800640 1 -- 172.21.15.133:0/293745711 >> 172.21.15.133:0/293745711 conn(0x7f28f8066eb0 msgr2=0x7f28f8067290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:12.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.981+0000 7f28ff800640 1 -- 172.21.15.133:0/293745711 shutdown_connections 2024-03-07T06:54:12.982 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.981+0000 7f28ff800640 1 -- 172.21.15.133:0/293745711 wait complete. 2024-03-07T06:54:12.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.983+0000 7f28ff800640 1 Processor -- start 2024-03-07T06:54:12.983 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.983+0000 7f28ff800640 1 -- start start 2024-03-07T06:54:12.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.984+0000 7f28ff800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28f8070820 0x7f28f81422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.984+0000 7f28ff800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28f80cc6f0 0x7f28f81e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.984+0000 7f28ff800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28f80cdf00 0x7f28f81f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.984+0000 7f28ff800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f815d210 con 0x7f28f80cdf00 2024-03-07T06:54:12.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.984+0000 7f28ff800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f8160160 con 0x7f28f80cc6f0 2024-03-07T06:54:12.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.984+0000 7f28ff800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f81e69a0 con 0x7f28f8070820 2024-03-07T06:54:12.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.984+0000 7f28fd800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28f80cdf00 0x7f28f81f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.985 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.984+0000 7f28fd800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28f80cdf00 0x7f28f81f68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60940/0 (socket says 172.21.15.133:60940) 2024-03-07T06:54:12.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.984+0000 7f28fd800640 1 -- 172.21.15.133:0/3824013253 learned_addr learned my addr 172.21.15.133:0/3824013253 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:12.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.985+0000 7f28fd800640 1 -- 172.21.15.133:0/3824013253 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28f8070820 msgr2=0x7f28f81422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.985+0000 7f28fd800640 1 --2- 172.21.15.133:0/3824013253 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28f8070820 0x7f28f81422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.985+0000 7f28fd800640 1 -- 172.21.15.133:0/3824013253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28f80cc6f0 msgr2=0x7f28f81e58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:12.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.985+0000 7f28fd800640 1 --2- 172.21.15.133:0/3824013253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f28f80cc6f0 0x7f28f81e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:12.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.985+0000 7f28fd800640 1 -- 172.21.15.133:0/3824013253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f81e66c0 con 0x7f28f80cdf00 2024-03-07T06:54:12.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.985+0000 7f28fd800640 1 --2- 172.21.15.133:0/3824013253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28f80cdf00 0x7f28f81f68c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f28f000c800 tx=0x7f28f000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:12.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.985+0000 7f28f4c00640 1 -- 172.21.15.133:0/3824013253 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0018070 con 0x7f28f80cdf00 2024-03-07T06:54:12.986 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.985+0000 7f28f4c00640 1 -- 172.21.15.133:0/3824013253 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f0004d80 con 0x7f28f80cdf00 2024-03-07T06:54:12.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.985+0000 7f28f4c00640 1 -- 172.21.15.133:0/3824013253 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f000a880 con 0x7f28f80cdf00 2024-03-07T06:54:12.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.985+0000 7f28ff800640 1 -- 172.21.15.133:0/3824013253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f81f5de0 con 0x7f28f80cdf00 2024-03-07T06:54:12.987 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.985+0000 7f28ff800640 1 -- 172.21.15.133:0/3824013253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f81f63a0 con 0x7f28f80cdf00 2024-03-07T06:54:12.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.986+0000 7f28ff800640 1 -- 172.21.15.133:0/3824013253 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f28f8062c90 con 0x7f28f80cdf00 2024-03-07T06:54:12.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.987+0000 7f28f4c00640 1 -- 172.21.15.133:0/3824013253 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f28f000aa20 con 0x7f28f80cdf00 2024-03-07T06:54:12.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.988+0000 7f28f4c00640 1 --2- 172.21.15.133:0/3824013253 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f28dc06b4d0 0x7f28dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.988 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.988+0000 7f28f4c00640 1 -- 172.21.15.133:0/3824013253 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f28f00894b0 con 0x7f28f80cdf00 2024-03-07T06:54:12.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.988+0000 7f28f4c00640 1 --2- 172.21.15.133:0/3824013253 >> [v2:172.21.15.161:6810/3665789028,v1:172.21.15.161:6811/3665789028] conn(0x7f28dc071390 0x7f28dc0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:12.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.989+0000 7f28f4c00640 1 -- 172.21.15.133:0/3824013253 --> [v2:172.21.15.161:6810/3665789028,v1:172.21.15.161:6811/3665789028] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f28dc073e60 con 0x7f28dc071390 2024-03-07T06:54:12.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.989+0000 7f28f4c00640 1 -- 172.21.15.133:0/3824013253 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28f0090050 con 0x7f28f80cdf00 2024-03-07T06:54:12.989 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.989+0000 7f28fce00640 1 --2- 172.21.15.133:0/3824013253 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f28dc06b4d0 0x7f28dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.989+0000 7f28f7e00640 1 --2- 172.21.15.133:0/3824013253 >> [v2:172.21.15.161:6810/3665789028,v1:172.21.15.161:6811/3665789028] conn(0x7f28dc071390 0x7f28dc0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:12.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.990+0000 7f28fce00640 1 --2- 172.21.15.133:0/3824013253 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f28dc06b4d0 0x7f28dc06d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f28ec003cc0 tx=0x7f28ec0079b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:12.990 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.990+0000 7f28f7e00640 1 --2- 172.21.15.133:0/3824013253 >> [v2:172.21.15.161:6810/3665789028,v1:172.21.15.161:6811/3665789028] conn(0x7f28dc071390 0x7f28dc0737b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:12.992 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:12.992+0000 7f28f4c00640 1 -- 172.21.15.133:0/3824013253 <== osd.7 v2:172.21.15.161:6810/3665789028 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f28dc073e60 con 0x7f28dc071390 2024-03-07T06:54:13.006 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.007+0000 7f28ff800640 1 -- 172.21.15.133:0/3824013253 --> [v2:172.21.15.161:6810/3665789028,v1:172.21.15.161:6811/3665789028] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f28f806dcb0 con 0x7f28dc071390 2024-03-07T06:54:13.007 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.008+0000 7f28f4c00640 1 -- 172.21.15.133:0/3824013253 <== osd.7 v2:172.21.15.161:6810/3665789028 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f28f806dcb0 con 0x7f28dc071390 2024-03-07T06:54:13.007 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2024-03-07T06:54:13.008 INFO:teuthology.orchestra.run.smithi133.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-07T06:54:13.008 INFO:teuthology.orchestra.run.smithi133.stdout:} 2024-03-07T06:54:13.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.008+0000 7f28ff800640 1 -- 172.21.15.133:0/3824013253 >> [v2:172.21.15.161:6810/3665789028,v1:172.21.15.161:6811/3665789028] conn(0x7f28dc071390 msgr2=0x7f28dc0737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:13.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.008+0000 7f28ff800640 1 --2- 172.21.15.133:0/3824013253 >> [v2:172.21.15.161:6810/3665789028,v1:172.21.15.161:6811/3665789028] conn(0x7f28dc071390 0x7f28dc0737b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.008+0000 7f28ff800640 1 -- 172.21.15.133:0/3824013253 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f28dc06b4d0 msgr2=0x7f28dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:13.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.008+0000 7f28ff800640 1 --2- 172.21.15.133:0/3824013253 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f28dc06b4d0 0x7f28dc06d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f28ec003cc0 tx=0x7f28ec0079b0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.008+0000 7f28ff800640 1 -- 172.21.15.133:0/3824013253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28f80cdf00 msgr2=0x7f28f81f68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:13.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.008+0000 7f28ff800640 1 --2- 172.21.15.133:0/3824013253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28f80cdf00 0x7f28f81f68c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f28f000c800 tx=0x7f28f000ccc0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.008+0000 7f28fd800640 1 -- 172.21.15.133:0/3824013253 reap_dead start 2024-03-07T06:54:13.012 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.008+0000 7f28ff800640 1 -- 172.21.15.133:0/3824013253 shutdown_connections 2024-03-07T06:54:13.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.008+0000 7f28ff800640 1 -- 172.21.15.133:0/3824013253 >> 172.21.15.133:0/3824013253 conn(0x7f28f8066eb0 msgr2=0x7f28f815d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:13.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.009+0000 7f28ff800640 1 -- 172.21.15.133:0/3824013253 shutdown_connections 2024-03-07T06:54:13.013 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.009+0000 7f28ff800640 1 -- 172.21.15.133:0/3824013253 wait complete. 2024-03-07T06:54:13.026 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-07T06:54:13.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.242+0000 7fe869c00640 1 Processor -- start 2024-03-07T06:54:13.242 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.242+0000 7fe869c00640 1 -- start start 2024-03-07T06:54:13.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.244+0000 7fe869c00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe8640b6120 0x7fe864070e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:13.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.244+0000 7fe869c00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe8640b43f0 0x7fe8640b6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:13.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.244+0000 7fe869c00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe864147ea0 con 0x7fe86406e3c0 2024-03-07T06:54:13.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.244+0000 7fe869c00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe864071390 con 0x7fe8640b6120 2024-03-07T06:54:13.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.244+0000 7fe869c00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8640b6a40 con 0x7fe8640b43f0 2024-03-07T06:54:13.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.244+0000 7fe862a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe8640b6120 0x7fe864070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:13.244 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.244+0000 7fe862a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7fe8640b6120 0x7fe864070e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60956/0 (socket says 172.21.15.133:60956) 2024-03-07T06:54:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.244+0000 7fe862a00640 1 -- 172.21.15.133:0/593606300 learned_addr learned my addr 172.21.15.133:0/593606300 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.244+0000 7fe862a00640 1 -- 172.21.15.133:0/593606300 >> v2:172.21.15.161:3300/0 conn(0x7fe8640b43f0 msgr2=0x7fe8640b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.244+0000 7fe862a00640 1 --2- 172.21.15.133:0/593606300 >> v2:172.21.15.161:3300/0 conn(0x7fe8640b43f0 0x7fe8640b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.244+0000 7fe862a00640 1 -- 172.21.15.133:0/593606300 >> v1:172.21.15.161:6789/0 conn(0x7fe86406e3c0 legacy=0x7fe86406e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:13.245 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.244+0000 7fe862a00640 1 -- 172.21.15.133:0/593606300 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8640b4960 con 0x7fe8640b6120 2024-03-07T06:54:13.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.245+0000 7fe862a00640 1 --2- 172.21.15.133:0/593606300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8640b6120 0x7fe864070e50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe86406ffa0 tx=0x7fe85402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba93f325a433878f server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:13.246 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.246+0000 7fe862000640 1 -- 172.21.15.133:0/593606300 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854035020 con 0x7fe8640b6120 2024-03-07T06:54:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.246+0000 7fe862000640 1 -- 172.21.15.133:0/593606300 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe854031870 con 0x7fe8640b6120 2024-03-07T06:54:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.246+0000 7fe862000640 1 -- 172.21.15.133:0/593606300 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854031c40 con 0x7fe8640b6120 2024-03-07T06:54:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.246+0000 7fe869c00640 1 -- 172.21.15.133:0/593606300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8640b6120 msgr2=0x7fe864070e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.246+0000 7fe869c00640 1 --2- 172.21.15.133:0/593606300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8640b6120 0x7fe864070e50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fe86406ffa0 tx=0x7fe85402b910 comp rx=0 tx=0).stop 2024-03-07T06:54:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.246+0000 7fe869c00640 1 -- 172.21.15.133:0/593606300 shutdown_connections 2024-03-07T06:54:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.246+0000 7fe869c00640 1 --2- 172.21.15.133:0/593606300 >> v2:172.21.15.161:3300/0 conn(0x7fe8640b43f0 0x7fe8640b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.246+0000 7fe869c00640 1 --2- 172.21.15.133:0/593606300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8640b6120 0x7fe864070e50 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.246+0000 7fe869c00640 1 -- 172.21.15.133:0/593606300 >> 172.21.15.133:0/593606300 conn(0x7fe864071550 msgr2=0x7fe864071960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:13.247 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.247+0000 7fe869c00640 1 -- 172.21.15.133:0/593606300 shutdown_connections 2024-03-07T06:54:13.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.247+0000 7fe869c00640 1 -- 172.21.15.133:0/593606300 wait complete. 2024-03-07T06:54:13.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.248+0000 7fe869c00640 1 Processor -- start 2024-03-07T06:54:13.248 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.248+0000 7fe869c00640 1 -- start start 2024-03-07T06:54:13.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.249+0000 7fe869c00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe86406e3c0 0x7fe8640cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:13.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.249+0000 7fe869c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8640b43f0 0x7fe8640c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:13.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.249+0000 7fe869c00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8640b6120 0x7fe8640c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:13.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.249+0000 7fe869c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8640b5200 con 0x7fe8640b43f0 2024-03-07T06:54:13.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.249+0000 7fe869c00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8640b4f20 con 0x7fe8640b6120 2024-03-07T06:54:13.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.249+0000 7fe869c00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86406ef40 con 0x7fe86406e3c0 2024-03-07T06:54:13.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.249+0000 7fe85a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8640b43f0 0x7fe8640c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:13.250 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.250+0000 7fe85a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8640b43f0 0x7fe8640c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60964/0 (socket says 172.21.15.133:60964) 2024-03-07T06:54:13.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.250+0000 7fe85a000640 1 -- 172.21.15.133:0/783522139 learned_addr learned my addr 172.21.15.133:0/783522139 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:13.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.250+0000 7fe862a00640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe86406e3c0 0x7fe8640cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:13.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.250+0000 7fe85a000640 1 -- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe86406e3c0 msgr2=0x7fe8640cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:13.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.250+0000 7fe85a000640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe86406e3c0 0x7fe8640cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.250+0000 7fe85a000640 1 -- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8640b6120 msgr2=0x7fe8640c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:13.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.250+0000 7fe863400640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8640b6120 0x7fe8640c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:13.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.250+0000 7fe85a000640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8640b6120 0x7fe8640c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.251 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.250+0000 7fe85a000640 1 -- 172.21.15.133:0/783522139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8640cea70 con 0x7fe8640b43f0 2024-03-07T06:54:13.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.250+0000 7fe863400640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8640b6120 0x7fe8640c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:13.252 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.250+0000 7fe85a000640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8640b43f0 0x7fe8640c6110 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00b4a0 tx=0x7fe84c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:13.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.251+0000 7fe85b400640 1 -- 172.21.15.133:0/783522139 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c014070 con 0x7fe8640b43f0 2024-03-07T06:54:13.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.251+0000 7fe85b400640 1 -- 172.21.15.133:0/783522139 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c010610 con 0x7fe8640b43f0 2024-03-07T06:54:13.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.251+0000 7fe869c00640 1 -- 172.21.15.133:0/783522139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8640ced50 con 0x7fe8640b43f0 2024-03-07T06:54:13.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.251+0000 7fe85b400640 1 -- 172.21.15.133:0/783522139 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c010a00 con 0x7fe8640b43f0 2024-03-07T06:54:13.253 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.251+0000 7fe869c00640 1 -- 172.21.15.133:0/783522139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8640cf310 con 0x7fe8640b43f0 2024-03-07T06:54:13.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.252+0000 7fe869c00640 1 -- 172.21.15.133:0/783522139 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe864062c90 con 0x7fe8640b43f0 2024-03-07T06:54:13.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.253+0000 7fe85b400640 1 -- 172.21.15.133:0/783522139 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7fe84c017070 con 0x7fe8640b43f0 2024-03-07T06:54:13.254 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.254+0000 7fe85b400640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe84006b500 0x7fe84006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:13.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.254+0000 7fe85b400640 1 -- 172.21.15.133:0/783522139 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe84c0899c0 con 0x7fe8640b43f0 2024-03-07T06:54:13.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.254+0000 7fe862a00640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe84006b500 0x7fe84006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:13.255 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.256+0000 7fe862a00640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe84006b500 0x7fe84006d9c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe854036000 tx=0x7fe85403c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:13.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.258+0000 7fe85b400640 1 -- 172.21.15.133:0/783522139 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7fe84c058aa0 con 0x7fe8640b43f0 2024-03-07T06:54:13.455 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.455+0000 7fe869c00640 1 -- 172.21.15.133:0/783522139 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8640cd5a0 con 0x7fe84006b500 2024-03-07T06:54:13.456 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.457+0000 7fe85b400640 1 -- 172.21.15.133:0/783522139 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe8640cd5a0 con 0x7fe84006b500 2024-03-07T06:54:13.456 INFO:teuthology.orchestra.run.smithi133.stderr:instructed osd(s) 7 to deep-scrub 2024-03-07T06:54:13.459 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.459+0000 7fe869c00640 1 -- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe84006b500 msgr2=0x7fe84006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:13.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.459+0000 7fe869c00640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe84006b500 0x7fe84006d9c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe854036000 tx=0x7fe85403c040 comp rx=0 tx=0).stop 2024-03-07T06:54:13.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.459+0000 7fe869c00640 1 -- 172.21.15.133:0/783522139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8640b43f0 msgr2=0x7fe8640c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:13.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.459+0000 7fe869c00640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8640b43f0 0x7fe8640c6110 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fe84c00b4a0 tx=0x7fe84c00b960 comp rx=0 tx=0).stop 2024-03-07T06:54:13.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.460+0000 7fe869c00640 1 -- 172.21.15.133:0/783522139 shutdown_connections 2024-03-07T06:54:13.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.460+0000 7fe869c00640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7fe84006b500 0x7fe84006d9c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.460+0000 7fe869c00640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8640b6120 0x7fe8640c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.460+0000 7fe869c00640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8640b43f0 0x7fe8640c6110 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.460 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.460+0000 7fe869c00640 1 --2- 172.21.15.133:0/783522139 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe86406e3c0 0x7fe8640cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.460+0000 7fe869c00640 1 -- 172.21.15.133:0/783522139 >> 172.21.15.133:0/783522139 conn(0x7fe864071550 msgr2=0x7fe8641e6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:13.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.460+0000 7fe869c00640 1 -- 172.21.15.133:0/783522139 shutdown_connections 2024-03-07T06:54:13.461 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.460+0000 7fe869c00640 1 -- 172.21.15.133:0/783522139 wait complete. 2024-03-07T06:54:13.478 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:54:13.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.693+0000 7f9eff800640 1 Processor -- start 2024-03-07T06:54:13.693 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.693+0000 7f9eff800640 1 -- start start 2024-03-07T06:54:13.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.695+0000 7f9eff800640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9ef80b3c10 0x7f9ef8071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:13.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.695+0000 7f9eff800640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9ef8071b50 0x7f9ef8162f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:13.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.695+0000 7f9eff800640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef8071fe0 con 0x7f9ef8165ed0 2024-03-07T06:54:13.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.695+0000 7f9eff800640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef80734a0 con 0x7f9ef80b3c10 2024-03-07T06:54:13.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.695+0000 7f9eff800640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef80ce540 con 0x7f9ef8071b50 2024-03-07T06:54:13.695 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.695+0000 7f9ef7e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9ef80b3c10 0x7f9ef8071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:13.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.695+0000 7f9ef7e00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f9ef80b3c10 0x7f9ef8071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60968/0 (socket says 172.21.15.133:60968) 2024-03-07T06:54:13.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.695+0000 7f9ef7e00640 1 -- 172.21.15.133:0/1781267358 learned_addr learned my addr 172.21.15.133:0/1781267358 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:13.696 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.696+0000 7f9efd800640 1 --2- 172.21.15.133:0/1781267358 >> v2:172.21.15.161:3300/0 conn(0x7f9ef8071b50 0x7f9ef8162f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:13.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.696+0000 7f9ef7e00640 1 -- 172.21.15.133:0/1781267358 >> v2:172.21.15.161:3300/0 conn(0x7f9ef8071b50 msgr2=0x7f9ef8162f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:13.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.696+0000 7f9ef7e00640 1 --2- 172.21.15.133:0/1781267358 >> v2:172.21.15.161:3300/0 conn(0x7f9ef8071b50 0x7f9ef8162f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.696+0000 7f9ef7e00640 1 -- 172.21.15.133:0/1781267358 >> v1:172.21.15.161:6789/0 conn(0x7f9ef8165ed0 legacy=0x7f9ef806ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:13.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.696+0000 7f9ef7e00640 1 -- 172.21.15.133:0/1781267358 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef80cdf80 con 0x7f9ef80b3c10 2024-03-07T06:54:13.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.697+0000 7f9ef7e00640 1 --2- 172.21.15.133:0/1781267358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef80b3c10 0x7f9ef8071610 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8068cd0 tx=0x7f9ee002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e47c8429c43559 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:13.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.697+0000 7f9ef7400640 1 -- 172.21.15.133:0/1781267358 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee002ca80 con 0x7f9ef80b3c10 2024-03-07T06:54:13.697 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.697+0000 7f9ef7400640 1 -- 172.21.15.133:0/1781267358 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee002cc20 con 0x7f9ef80b3c10 2024-03-07T06:54:13.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.697+0000 7f9ef7400640 1 -- 172.21.15.133:0/1781267358 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00343f0 con 0x7f9ef80b3c10 2024-03-07T06:54:13.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.698+0000 7f9eff800640 1 -- 172.21.15.133:0/1781267358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef80b3c10 msgr2=0x7f9ef8071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:13.698 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.698+0000 7f9eff800640 1 --2- 172.21.15.133:0/1781267358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef80b3c10 0x7f9ef8071610 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8068cd0 tx=0x7f9ee002bee0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.698+0000 7f9eff800640 1 -- 172.21.15.133:0/1781267358 shutdown_connections 2024-03-07T06:54:13.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.698+0000 7f9eff800640 1 --2- 172.21.15.133:0/1781267358 >> v2:172.21.15.161:3300/0 conn(0x7f9ef8071b50 0x7f9ef8162f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.698+0000 7f9eff800640 1 --2- 172.21.15.133:0/1781267358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef80b3c10 0x7f9ef8071610 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.698+0000 7f9eff800640 1 -- 172.21.15.133:0/1781267358 >> 172.21.15.133:0/1781267358 conn(0x7f9ef80b3120 msgr2=0x7f9ef80b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:13.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.698+0000 7f9eff800640 1 -- 172.21.15.133:0/1781267358 shutdown_connections 2024-03-07T06:54:13.699 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.698+0000 7f9eff800640 1 -- 172.21.15.133:0/1781267358 wait complete. 2024-03-07T06:54:13.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.700+0000 7f9eff800640 1 Processor -- start 2024-03-07T06:54:13.700 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.701+0000 7f9eff800640 1 -- start start 2024-03-07T06:54:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.702+0000 7f9eff800640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ef8071b50 0x7f9ef8158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:13.702 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.702+0000 7f9eff800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef8165ed0 0x7f9ef815bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.702+0000 7f9eff800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ef815daf0 0x7f9ef815f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.702+0000 7f9eff800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef8072550 con 0x7f9ef8165ed0 2024-03-07T06:54:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.702+0000 7f9eff800640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef8072270 con 0x7f9ef8071b50 2024-03-07T06:54:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.702+0000 7f9eff800640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef81634d0 con 0x7f9ef815daf0 2024-03-07T06:54:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.702+0000 7f9ef7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef8165ed0 0x7f9ef815bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.702+0000 7f9ef7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef8165ed0 0x7f9ef815bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60970/0 (socket says 172.21.15.133:60970) 2024-03-07T06:54:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.702+0000 7f9ef7e00640 1 -- 172.21.15.133:0/861210479 learned_addr learned my addr 172.21.15.133:0/861210479 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:13.703 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.702+0000 7f9efce00640 1 --2- 172.21.15.133:0/861210479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ef8071b50 0x7f9ef8158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.702+0000 7f9ef7e00640 1 -- 172.21.15.133:0/861210479 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ef815daf0 msgr2=0x7f9ef815f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.702+0000 7f9ef7e00640 1 --2- 172.21.15.133:0/861210479 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ef815daf0 0x7f9ef815f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.703+0000 7f9ef7e00640 1 -- 172.21.15.133:0/861210479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ef8071b50 msgr2=0x7f9ef8158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.703+0000 7f9ef7e00640 1 --2- 172.21.15.133:0/861210479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ef8071b50 0x7f9ef8158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.703+0000 7f9ef7e00640 1 -- 172.21.15.133:0/861210479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef815c020 con 0x7f9ef8165ed0 2024-03-07T06:54:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.703+0000 7f9efce00640 1 --2- 172.21.15.133:0/861210479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ef8071b50 0x7f9ef8158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:13.704 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.703+0000 7f9ef7e00640 1 --2- 172.21.15.133:0/861210479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef8165ed0 0x7f9ef815bae0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009f40 tx=0x7f9ee0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.703+0000 7f9ef4c00640 1 -- 172.21.15.133:0/861210479 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0043020 con 0x7f9ef8165ed0 2024-03-07T06:54:13.705 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.703+0000 7f9ef4c00640 1 -- 172.21.15.133:0/861210479 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee003e450 con 0x7f9ef8165ed0 2024-03-07T06:54:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.703+0000 7f9ef4c00640 1 -- 172.21.15.133:0/861210479 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0009510 con 0x7f9ef8165ed0 2024-03-07T06:54:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.703+0000 7f9eff800640 1 -- 172.21.15.133:0/861210479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef815f550 con 0x7f9ef8165ed0 2024-03-07T06:54:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.703+0000 7f9eff800640 1 -- 172.21.15.133:0/861210479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef815e250 con 0x7f9ef8165ed0 2024-03-07T06:54:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.705+0000 7f9eff800640 1 -- 172.21.15.133:0/861210479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef8062c90 con 0x7f9ef8165ed0 2024-03-07T06:54:13.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.705+0000 7f9ef4c00640 1 -- 172.21.15.133:0/861210479 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9ee0046020 con 0x7f9ef8165ed0 2024-03-07T06:54:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.706+0000 7f9ef4c00640 1 --2- 172.21.15.133:0/861210479 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9ed406b4d0 0x7f9ed406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.706+0000 7f9ef4c00640 1 -- 172.21.15.133:0/861210479 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9ee00ac710 con 0x7f9ef8165ed0 2024-03-07T06:54:13.707 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.707+0000 7f9efce00640 1 --2- 172.21.15.133:0/861210479 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9ed406b4d0 0x7f9ed406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:13.708 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.708+0000 7f9efce00640 1 --2- 172.21.15.133:0/861210479 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9ed406b4d0 0x7f9ed406d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800a830 tx=0x7f9ee8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:13.710 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.711+0000 7f9ef4c00640 1 -- 172.21.15.133:0/861210479 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9ee004d030 con 0x7f9ef8165ed0 2024-03-07T06:54:13.899 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.899+0000 7f9eff800640 1 -- 172.21.15.133:0/861210479 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ef81617b0 con 0x7f9ed406b4d0 2024-03-07T06:54:13.901 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.902+0000 7f9ef4c00640 1 -- 172.21.15.133:0/861210479 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13353 (secure 0 0 0) 0x7f9ef81617b0 con 0x7f9ed406b4d0 2024-03-07T06:54:13.902 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:54:13.902 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:54:13.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.904+0000 7f9eff800640 1 -- 172.21.15.133:0/861210479 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9ed406b4d0 msgr2=0x7f9ed406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:13.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.904+0000 7f9eff800640 1 --2- 172.21.15.133:0/861210479 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9ed406b4d0 0x7f9ed406d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800a830 tx=0x7f9ee8008040 comp rx=0 tx=0).stop 2024-03-07T06:54:13.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.904+0000 7f9eff800640 1 -- 172.21.15.133:0/861210479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef8165ed0 msgr2=0x7f9ef815bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:13.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.904+0000 7f9eff800640 1 --2- 172.21.15.133:0/861210479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef8165ed0 0x7f9ef815bae0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009f40 tx=0x7f9ee0004a40 comp rx=0 tx=0).stop 2024-03-07T06:54:13.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.904+0000 7f9eff800640 1 -- 172.21.15.133:0/861210479 shutdown_connections 2024-03-07T06:54:13.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.904+0000 7f9eff800640 1 --2- 172.21.15.133:0/861210479 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ef815daf0 0x7f9ef815f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.904+0000 7f9eff800640 1 --2- 172.21.15.133:0/861210479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9ef8165ed0 0x7f9ef815bae0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.904+0000 7f9eff800640 1 --2- 172.21.15.133:0/861210479 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9ed406b4d0 0x7f9ed406d990 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.904+0000 7f9eff800640 1 --2- 172.21.15.133:0/861210479 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ef8071b50 0x7f9ef8158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:13.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.904+0000 7f9eff800640 1 -- 172.21.15.133:0/861210479 >> 172.21.15.133:0/861210479 conn(0x7f9ef80b3120 msgr2=0x7f9ef8162260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:13.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.905+0000 7f9eff800640 1 -- 172.21.15.133:0/861210479 shutdown_connections 2024-03-07T06:54:13.906 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:13.905+0000 7f9eff800640 1 -- 172.21.15.133:0/861210479 wait complete. 2024-03-07T06:54:13.923 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-03-07T06:54:12.534159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":255376,"kb_used_data":5928,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719344,"statfs":{"total":773094113280,"available":772832608256,"internally_reserved":0,"allocated":6070272,"data_stored":3177544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.200915"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":228,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-07T06:54:01.852422+0000","last_change":"2024-03-07T06:51:06.316448+0000","last_active":"2024-03-07T06:54:01.852422+0000","last_peered":"2024-03-07T06:54:01.852422+0000","last_clean":"2024-03-07T06:54:01.852422+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:54:01.852422+0000","last_undegraded":"2024-03-07T06:54:01.852422+0000","last_fullsized":"2024-03-07T06:54:01.852422+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-07T06:51:04.803542+0000","last_clean_scrub_stamp":"2024-03-07T06:51:04.803542+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T17:15:32.354066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":839969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1204224,"data_stored":839969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:54:13.927 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-07T06:51:04.803542+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=6, tm_min=51, tm_sec=4, tm_wday=3, tm_yday=67, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=7, tm_hour=6, tm_min=54, tm_sec=6, tm_wday=3, tm_yday=67, tm_isdst=0) 2024-03-07T06:54:13.927 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-07T06:54:30.528 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:54:30.537 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:54:33.928 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-07T06:54:34.148 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.149+0000 7f207dc00640 1 Processor -- start 2024-03-07T06:54:34.149 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.149+0000 7f207dc00640 1 -- start start 2024-03-07T06:54:34.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.150+0000 7f207dc00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f207815fbb0 0x7f20780cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:34.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.150+0000 7f207dc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20780cdee0 0x7f207815c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:34.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.150+0000 7f207dc00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20780731b0 con 0x7f2078072950 2024-03-07T06:54:34.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.150+0000 7f207dc00640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2078072f60 con 0x7f207815fbb0 2024-03-07T06:54:34.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.150+0000 7f207dc00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2078073490 con 0x7f20780cdee0 2024-03-07T06:54:34.151 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.151+0000 7f2076a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f207815fbb0 0x7f20780cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.151+0000 7f2076a00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7f207815fbb0 0x7f20780cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54260/0 (socket says 172.21.15.133:54260) 2024-03-07T06:54:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.151+0000 7f2076a00640 1 -- 172.21.15.133:0/3563495001 learned_addr learned my addr 172.21.15.133:0/3563495001 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.151+0000 7f2077400640 1 --2- 172.21.15.133:0/3563495001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20780cdee0 0x7f207815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.151+0000 7f2076a00640 1 -- 172.21.15.133:0/3563495001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20780cdee0 msgr2=0x7f207815c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.151+0000 7f2076a00640 1 --2- 172.21.15.133:0/3563495001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20780cdee0 0x7f207815c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.151+0000 7f2076a00640 1 -- 172.21.15.133:0/3563495001 >> v1:172.21.15.161:6789/0 conn(0x7f2078072950 legacy=0x7f207815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:34.152 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.151+0000 7f2076a00640 1 -- 172.21.15.133:0/3563495001 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078072150 con 0x7f207815fbb0 2024-03-07T06:54:34.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.152+0000 7f2076a00640 1 --2- 172.21.15.133:0/3563495001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f207815fbb0 0x7f20780cf7d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2068009870 tx=0x7f206802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c8d88a26ded82e server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:34.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.153+0000 7f2076000640 1 -- 172.21.15.133:0/3563495001 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20680317c0 con 0x7f207815fbb0 2024-03-07T06:54:34.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.153+0000 7f2076000640 1 -- 172.21.15.133:0/3563495001 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2068037070 con 0x7f207815fbb0 2024-03-07T06:54:34.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.153+0000 7f2076000640 1 -- 172.21.15.133:0/3563495001 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068031b80 con 0x7f207815fbb0 2024-03-07T06:54:34.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.153+0000 7f207dc00640 1 -- 172.21.15.133:0/3563495001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f207815fbb0 msgr2=0x7f20780cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:34.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.153+0000 7f207dc00640 1 --2- 172.21.15.133:0/3563495001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f207815fbb0 0x7f20780cf7d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2068009870 tx=0x7f206802b860 comp rx=0 tx=0).stop 2024-03-07T06:54:34.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.153+0000 7f207dc00640 1 -- 172.21.15.133:0/3563495001 shutdown_connections 2024-03-07T06:54:34.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.153+0000 7f207dc00640 1 --2- 172.21.15.133:0/3563495001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20780cdee0 0x7f207815c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.153+0000 7f207dc00640 1 --2- 172.21.15.133:0/3563495001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f207815fbb0 0x7f20780cf7d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.153+0000 7f207dc00640 1 -- 172.21.15.133:0/3563495001 >> 172.21.15.133:0/3563495001 conn(0x7f20780b3120 msgr2=0x7f20780b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:34.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.153+0000 7f207dc00640 1 -- 172.21.15.133:0/3563495001 shutdown_connections 2024-03-07T06:54:34.155 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.153+0000 7f207dc00640 1 -- 172.21.15.133:0/3563495001 wait complete. 2024-03-07T06:54:34.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.155+0000 7f207dc00640 1 Processor -- start 2024-03-07T06:54:34.158 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.155+0000 7f207dc00640 1 -- start start 2024-03-07T06:54:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.156+0000 7f207dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20780cdee0 0x7f207815c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.156+0000 7f207dc00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f207815fbb0 0x7f20780aed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.156+0000 7f207dc00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20780af250 0x7f2078153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.156+0000 7f207dc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20780ce2c0 con 0x7f20780cdee0 2024-03-07T06:54:34.159 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.156+0000 7f207dc00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2078160980 con 0x7f207815fbb0 2024-03-07T06:54:34.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.156+0000 7f207dc00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20780b1a00 con 0x7f20780af250 2024-03-07T06:54:34.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.156+0000 7f2076a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20780cdee0 0x7f207815c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:34.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.157+0000 7f2076a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20780cdee0 0x7f207815c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54270/0 (socket says 172.21.15.133:54270) 2024-03-07T06:54:34.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.157+0000 7f2076a00640 1 -- 172.21.15.133:0/3945707685 learned_addr learned my addr 172.21.15.133:0/3945707685 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:34.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.157+0000 7f2077400640 1 --2- 172.21.15.133:0/3945707685 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20780af250 0x7f2078153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:34.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.157+0000 7f206fe00640 1 --2- 172.21.15.133:0/3945707685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f207815fbb0 0x7f20780aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:34.160 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.157+0000 7f2076a00640 1 -- 172.21.15.133:0/3945707685 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20780af250 msgr2=0x7f2078153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:34.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.157+0000 7f2076a00640 1 --2- 172.21.15.133:0/3945707685 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20780af250 0x7f2078153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.157+0000 7f2076a00640 1 -- 172.21.15.133:0/3945707685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f207815fbb0 msgr2=0x7f20780aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:34.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.157+0000 7f2076a00640 1 --2- 172.21.15.133:0/3945707685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f207815fbb0 0x7f20780aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.157+0000 7f2076a00640 1 -- 172.21.15.133:0/3945707685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20780afa40 con 0x7f20780cdee0 2024-03-07T06:54:34.161 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.157+0000 7f2076a00640 1 --2- 172.21.15.133:0/3945707685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20780cdee0 0x7f207815c590 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2068009870 tx=0x7f20680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:34.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.158+0000 7f206ea00640 1 -- 172.21.15.133:0/3945707685 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068032860 con 0x7f20780cdee0 2024-03-07T06:54:34.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.158+0000 7f206ea00640 1 -- 172.21.15.133:0/3945707685 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2068032a00 con 0x7f20780cdee0 2024-03-07T06:54:34.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.158+0000 7f207dc00640 1 -- 172.21.15.133:0/3945707685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207809ac20 con 0x7f20780cdee0 2024-03-07T06:54:34.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.158+0000 7f206ea00640 1 -- 172.21.15.133:0/3945707685 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068032df0 con 0x7f20780cdee0 2024-03-07T06:54:34.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.158+0000 7f207dc00640 1 -- 172.21.15.133:0/3945707685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207809b1e0 con 0x7f20780cdee0 2024-03-07T06:54:34.162 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.159+0000 7f207dc00640 1 -- 172.21.15.133:0/3945707685 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2078062c90 con 0x7f20780cdee0 2024-03-07T06:54:34.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.160+0000 7f206ea00640 1 -- 172.21.15.133:0/3945707685 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f206804c060 con 0x7f20780cdee0 2024-03-07T06:54:34.163 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.161+0000 7f206ea00640 1 --2- 172.21.15.133:0/3945707685 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f205406b4d0 0x7f205406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:34.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.161+0000 7f206ea00640 1 -- 172.21.15.133:0/3945707685 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f20680ab8a0 con 0x7f20780cdee0 2024-03-07T06:54:34.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.161+0000 7f206fe00640 1 --2- 172.21.15.133:0/3945707685 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f205406b4d0 0x7f205406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:34.164 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.162+0000 7f206fe00640 1 --2- 172.21.15.133:0/3945707685 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f205406b4d0 0x7f205406d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2060006fd0 tx=0x7f2060008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:34.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.168+0000 7f206ea00640 1 -- 172.21.15.133:0/3945707685 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f2068040070 con 0x7f20780cdee0 2024-03-07T06:54:34.363 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.363+0000 7f207dc00640 1 -- 172.21.15.133:0/3945707685 --> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f207815b250 con 0x7f205406b4d0 2024-03-07T06:54:34.365 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.365+0000 7f206ea00640 1 -- 172.21.15.133:0/3945707685 <== mgr.4100 v2:172.21.15.161:6800/187879951 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13360 (secure 0 0 0) 0x7f207815b250 con 0x7f205406b4d0 2024-03-07T06:54:34.365 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:54:34.366 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2024-03-07T06:54:34.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.368+0000 7f207dc00640 1 -- 172.21.15.133:0/3945707685 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f205406b4d0 msgr2=0x7f205406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:34.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.368+0000 7f207dc00640 1 --2- 172.21.15.133:0/3945707685 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f205406b4d0 0x7f205406d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2060006fd0 tx=0x7f2060008040 comp rx=0 tx=0).stop 2024-03-07T06:54:34.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.368+0000 7f207dc00640 1 -- 172.21.15.133:0/3945707685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20780cdee0 msgr2=0x7f207815c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:34.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.368+0000 7f207dc00640 1 --2- 172.21.15.133:0/3945707685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20780cdee0 0x7f207815c590 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2068009870 tx=0x7f20680046c0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.368+0000 7f207dc00640 1 -- 172.21.15.133:0/3945707685 shutdown_connections 2024-03-07T06:54:34.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.368+0000 7f207dc00640 1 --2- 172.21.15.133:0/3945707685 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20780af250 0x7f2078153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.368 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.368+0000 7f207dc00640 1 --2- 172.21.15.133:0/3945707685 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f205406b4d0 0x7f205406d990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.368+0000 7f207dc00640 1 --2- 172.21.15.133:0/3945707685 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f207815fbb0 0x7f20780aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.368+0000 7f207dc00640 1 --2- 172.21.15.133:0/3945707685 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f20780cdee0 0x7f207815c590 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.368+0000 7f207dc00640 1 -- 172.21.15.133:0/3945707685 >> 172.21.15.133:0/3945707685 conn(0x7f20780b3120 msgr2=0x7f207815ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:34.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.368+0000 7f207dc00640 1 -- 172.21.15.133:0/3945707685 shutdown_connections 2024-03-07T06:54:34.369 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.368+0000 7f207dc00640 1 -- 172.21.15.133:0/3945707685 wait complete. 2024-03-07T06:54:34.387 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-03-07T06:54:32.539900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":255360,"kb_used_data":5928,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719360,"statfs":{"total":773094113280,"available":772832624640,"internally_reserved":0,"allocated":6070272,"data_stored":3177544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003402"},"pg_stats":[{"pgid":"1.0","version":"51'119","reported_seq":239,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-03-07T06:54:14.384755+0000","last_change":"2024-03-07T06:54:14.384832+0000","last_active":"2024-03-07T06:54:14.384755+0000","last_peered":"2024-03-07T06:54:14.384755+0000","last_clean":"2024-03-07T06:54:14.384755+0000","last_became_active":"2024-03-07T06:51:06.314342+0000","last_became_peered":"2024-03-07T06:51:06.314342+0000","last_unstale":"2024-03-07T06:54:14.384755+0000","last_undegraded":"2024-03-07T06:54:14.384755+0000","last_fullsized":"2024-03-07T06:54:14.384755+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"51'119","last_scrub_stamp":"2024-03-07T06:54:14.384565+0000","last_deep_scrub":"51'119","last_deep_scrub_stamp":"2024-03-07T06:54:14.384565+0000","last_clean_scrub_stamp":"2024-03-07T06:54:14.384565+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-08T14:34:32.556521+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1204224,"data_stored":839969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31776,"kb_used_data":596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340064,"statfs":{"total":96636764160,"available":96604225536,"internally_reserved":0,"allocated":610304,"data_stored":249601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1204224,"data_stored":839969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-07T06:54:34.388 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-07T06:54:34.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.591+0000 7ff332200640 1 Processor -- start 2024-03-07T06:54:34.591 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.591+0000 7ff332200640 1 -- start start 2024-03-07T06:54:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.593+0000 7ff332200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c145670 0x7ff32c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.593+0000 7ff332200640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff32c15cf90 0x7ff32c0cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:34.592 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.593+0000 7ff332200640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32c06e150 con 0x7ff32c0cda20 2024-03-07T06:54:34.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.593+0000 7ff332200640 1 -- --> v2:172.21.15.133:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32c06ed60 con 0x7ff32c15cf90 2024-03-07T06:54:34.593 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.593+0000 7ff332200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c06e430 con 0x7ff32c145670 2024-03-07T06:54:34.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.593+0000 7ff32aa00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff32c15cf90 0x7ff32c0cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:34.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.593+0000 7ff32aa00640 1 --2- >> v2:172.21.15.133:3300/0 conn(0x7ff32c15cf90 0x7ff32c0cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54286/0 (socket says 172.21.15.133:54286) 2024-03-07T06:54:34.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.593+0000 7ff32aa00640 1 -- 172.21.15.133:0/1157088125 learned_addr learned my addr 172.21.15.133:0/1157088125 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:34.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.594+0000 7ff32b400640 1 --2- 172.21.15.133:0/1157088125 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c145670 0x7ff32c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:34.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.594+0000 7ff32aa00640 1 -- 172.21.15.133:0/1157088125 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c145670 msgr2=0x7ff32c0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:34.594 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.594+0000 7ff32aa00640 1 --2- 172.21.15.133:0/1157088125 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c145670 0x7ff32c0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.594+0000 7ff32aa00640 1 -- 172.21.15.133:0/1157088125 >> v1:172.21.15.161:6789/0 conn(0x7ff32c0cda20 legacy=0x7ff32c15fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:34.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.594+0000 7ff32aa00640 1 -- 172.21.15.133:0/1157088125 --> v2:172.21.15.133:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c06e710 con 0x7ff32c15cf90 2024-03-07T06:54:34.595 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.594+0000 7ff32b400640 1 --2- 172.21.15.133:0/1157088125 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c145670 0x7ff32c0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:34.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.595+0000 7ff32aa00640 1 --2- 172.21.15.133:0/1157088125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c15cf90 0x7ff32c0cd4e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff318009870 tx=0x7ff31802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858c6a95f523e5e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:34.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.595+0000 7ff32a000640 1 -- 172.21.15.133:0/1157088125 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3180317c0 con 0x7ff32c15cf90 2024-03-07T06:54:34.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.595+0000 7ff32a000640 1 -- 172.21.15.133:0/1157088125 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff318037070 con 0x7ff32c15cf90 2024-03-07T06:54:34.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.595+0000 7ff32a000640 1 -- 172.21.15.133:0/1157088125 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318031b80 con 0x7ff32c15cf90 2024-03-07T06:54:34.596 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.596+0000 7ff332200640 1 -- 172.21.15.133:0/1157088125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c15cf90 msgr2=0x7ff32c0cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.596+0000 7ff332200640 1 --2- 172.21.15.133:0/1157088125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c15cf90 0x7ff32c0cd4e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff318009870 tx=0x7ff31802b860 comp rx=0 tx=0).stop 2024-03-07T06:54:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.596+0000 7ff332200640 1 -- 172.21.15.133:0/1157088125 shutdown_connections 2024-03-07T06:54:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.596+0000 7ff332200640 1 --2- 172.21.15.133:0/1157088125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c15cf90 0x7ff32c0cd4e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.596+0000 7ff332200640 1 --2- 172.21.15.133:0/1157088125 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c145670 0x7ff32c0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.596+0000 7ff332200640 1 -- 172.21.15.133:0/1157088125 >> 172.21.15.133:0/1157088125 conn(0x7ff32c0b1d30 msgr2=0x7ff32c0b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.596+0000 7ff332200640 1 -- 172.21.15.133:0/1157088125 shutdown_connections 2024-03-07T06:54:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.596+0000 7ff332200640 1 -- 172.21.15.133:0/1157088125 wait complete. 2024-03-07T06:54:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.598+0000 7ff332200640 1 Processor -- start 2024-03-07T06:54:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.598+0000 7ff332200640 1 -- start start 2024-03-07T06:54:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.599+0000 7ff332200640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff32c0cda20 0x7ff32c135620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.599+0000 7ff332200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c145670 0x7ff32c0c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.599+0000 7ff332200640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c15cf90 0x7ff32c131f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.599+0000 7ff332200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c145a50 con 0x7ff32c145670 2024-03-07T06:54:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.599+0000 7ff332200640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c15d3f0 con 0x7ff32c0cda20 2024-03-07T06:54:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.599+0000 7ff332200640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c145bd0 con 0x7ff32c15cf90 2024-03-07T06:54:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.599+0000 7ff32aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c145670 0x7ff32c0c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.599+0000 7ff32aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c145670 0x7ff32c0c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54302/0 (socket says 172.21.15.133:54302) 2024-03-07T06:54:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.599+0000 7ff32aa00640 1 -- 172.21.15.133:0/2718519957 learned_addr learned my addr 172.21.15.133:0/2718519957 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.599+0000 7ff32aa00640 1 -- 172.21.15.133:0/2718519957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c15cf90 msgr2=0x7ff32c131f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.599+0000 7ff32aa00640 1 --2- 172.21.15.133:0/2718519957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c15cf90 0x7ff32c131f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.600+0000 7ff32aa00640 1 -- 172.21.15.133:0/2718519957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff32c0cda20 msgr2=0x7ff32c135620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.600+0000 7ff32b400640 1 --2- 172.21.15.133:0/2718519957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff32c0cda20 0x7ff32c135620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:34.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.600+0000 7ff32aa00640 1 --2- 172.21.15.133:0/2718519957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff32c0cda20 0x7ff32c135620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.600+0000 7ff32aa00640 1 -- 172.21.15.133:0/2718519957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c132480 con 0x7ff32c145670 2024-03-07T06:54:34.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.600+0000 7ff32b400640 1 --2- 172.21.15.133:0/2718519957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff32c0cda20 0x7ff32c135620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-07T06:54:34.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.600+0000 7ff32aa00640 1 --2- 172.21.15.133:0/2718519957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c145670 0x7ff32c0c91d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff318000c00 tx=0x7ff3180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:34.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.600+0000 7ff323400640 1 -- 172.21.15.133:0/2718519957 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318032860 con 0x7ff32c145670 2024-03-07T06:54:34.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.600+0000 7ff323400640 1 -- 172.21.15.133:0/2718519957 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff318032a20 con 0x7ff32c145670 2024-03-07T06:54:34.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.600+0000 7ff332200640 1 -- 172.21.15.133:0/2718519957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff32c12f540 con 0x7ff32c145670 2024-03-07T06:54:34.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.600+0000 7ff323400640 1 -- 172.21.15.133:0/2718519957 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318032e10 con 0x7ff32c145670 2024-03-07T06:54:34.602 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.601+0000 7ff332200640 1 -- 172.21.15.133:0/2718519957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c12fb00 con 0x7ff32c145670 2024-03-07T06:54:34.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.602+0000 7ff332200640 1 -- 172.21.15.133:0/2718519957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32c062c90 con 0x7ff32c145670 2024-03-07T06:54:34.603 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.603+0000 7ff323400640 1 -- 172.21.15.133:0/2718519957 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7ff31804c060 con 0x7ff32c145670 2024-03-07T06:54:34.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.603+0000 7ff323400640 1 --2- 172.21.15.133:0/2718519957 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff30c06b4d0 0x7ff30c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:34.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.604+0000 7ff323400640 1 -- 172.21.15.133:0/2718519957 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff3180abdc0 con 0x7ff32c145670 2024-03-07T06:54:34.604 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.604+0000 7ff32b400640 1 --2- 172.21.15.133:0/2718519957 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff30c06b4d0 0x7ff30c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:34.605 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.605+0000 7ff32b400640 1 --2- 172.21.15.133:0/2718519957 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff30c06b4d0 0x7ff30c06d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff31c006fd0 tx=0x7ff31c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.608+0000 7ff323400640 1 -- 172.21.15.133:0/2718519957 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7ff32c132480 con 0x7ff32c145670 2024-03-07T06:54:34.805 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.806+0000 7ff332200640 1 -- 172.21.15.133:0/2718519957 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7ff32c15f390 con 0x7ff32c145670 2024-03-07T06:54:34.810 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.811+0000 7ff323400640 1 -- 172.21.15.133:0/2718519957 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7ff318040070 con 0x7ff32c145670 2024-03-07T06:54:34.811 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.812+0000 7ff323400640 1 -- 172.21.15.133:0/2718519957 <== mon.0 v2:172.21.15.133:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff318037020 con 0x7ff32c145670 2024-03-07T06:54:34.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.815+0000 7ff332200640 1 -- 172.21.15.133:0/2718519957 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff30c06b4d0 msgr2=0x7ff30c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:34.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.815+0000 7ff332200640 1 --2- 172.21.15.133:0/2718519957 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff30c06b4d0 0x7ff30c06d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff31c006fd0 tx=0x7ff31c008040 comp rx=0 tx=0).stop 2024-03-07T06:54:34.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.815+0000 7ff332200640 1 -- 172.21.15.133:0/2718519957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c145670 msgr2=0x7ff32c0c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:34.815 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.815+0000 7ff332200640 1 --2- 172.21.15.133:0/2718519957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c145670 0x7ff32c0c91d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff318000c00 tx=0x7ff3180046c0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.815+0000 7ff332200640 1 -- 172.21.15.133:0/2718519957 shutdown_connections 2024-03-07T06:54:34.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.815+0000 7ff332200640 1 --2- 172.21.15.133:0/2718519957 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff32c15cf90 0x7ff32c131f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.815+0000 7ff332200640 1 --2- 172.21.15.133:0/2718519957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff32c145670 0x7ff32c0c91d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.815+0000 7ff332200640 1 --2- 172.21.15.133:0/2718519957 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7ff30c06b4d0 0x7ff30c06d990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.816 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.815+0000 7ff332200640 1 --2- 172.21.15.133:0/2718519957 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff32c0cda20 0x7ff32c135620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:34.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.815+0000 7ff332200640 1 -- 172.21.15.133:0/2718519957 >> 172.21.15.133:0/2718519957 conn(0x7ff32c0b1d30 msgr2=0x7ff32c15c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:34.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.815+0000 7ff332200640 1 -- 172.21.15.133:0/2718519957 shutdown_connections 2024-03-07T06:54:34.817 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:34.816+0000 7ff332200640 1 -- 172.21.15.133:0/2718519957 wait complete. 2024-03-07T06:54:34.834 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-07T06:54:34.835 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-07T06:54:35.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.048+0000 7f9a2ce00640 1 Processor -- start 2024-03-07T06:54:35.048 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.048+0000 7f9a2ce00640 1 -- start start 2024-03-07T06:54:35.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.050+0000 7f9a2ce00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9a2815fbb0 0x7f9a280cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:35.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.050+0000 7f9a2ce00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a280cdee0 0x7f9a2815c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:35.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.050+0000 7f9a2ce00640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a280731b0 con 0x7f9a28072950 2024-03-07T06:54:35.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.050+0000 7f9a2ce00640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a28072f60 con 0x7f9a2815fbb0 2024-03-07T06:54:35.050 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.050+0000 7f9a2ce00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a28073490 con 0x7f9a280cdee0 2024-03-07T06:54:35.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.051+0000 7f9a25e00640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9a2815fbb0 0x7f9a280cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:35.051 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.051+0000 7f9a26800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a280cdee0 0x7f9a2815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:35.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.051+0000 7f9a26800640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a280cdee0 0x7f9a2815c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.133:58976/0 (socket says 172.21.15.133:58976) 2024-03-07T06:54:35.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.051+0000 7f9a26800640 1 -- 172.21.15.133:0/3967911192 learned_addr learned my addr 172.21.15.133:0/3967911192 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:35.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.051+0000 7f9a25400640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f9a28072950 0x7f9a2815ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.133:48434/0 (socket says 172.21.15.133:48434) 2024-03-07T06:54:35.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.052+0000 7f9a26800640 1 -- 172.21.15.133:0/3967911192 >> v2:172.21.15.161:3300/0 conn(0x7f9a2815fbb0 msgr2=0x7f9a280cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-07T06:54:35.052 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.052+0000 7f9a26800640 1 --2- 172.21.15.133:0/3967911192 >> v2:172.21.15.161:3300/0 conn(0x7f9a2815fbb0 0x7f9a280cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:35.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.052+0000 7f9a26800640 1 -- 172.21.15.133:0/3967911192 >> v1:172.21.15.161:6789/0 conn(0x7f9a28072950 legacy=0x7f9a2815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:35.053 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.052+0000 7f9a26800640 1 -- 172.21.15.133:0/3967911192 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a28072150 con 0x7f9a280cdee0 2024-03-07T06:54:35.054 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.053+0000 7f9a26800640 1 --2- 172.21.15.133:0/3967911192 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a280cdee0 0x7f9a2815c550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009870 tx=0x7f9a2002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daf0d8aa08811035 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:35.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.053+0000 7f9a24a00640 1 -- 172.21.15.133:0/3967911192 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a20035020 con 0x7f9a280cdee0 2024-03-07T06:54:35.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.054+0000 7f9a24a00640 1 -- 172.21.15.133:0/3967911192 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9a20031640 con 0x7f9a280cdee0 2024-03-07T06:54:35.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.054+0000 7f9a24a00640 1 -- 172.21.15.133:0/3967911192 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2002d040 con 0x7f9a280cdee0 2024-03-07T06:54:35.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.054+0000 7f9a2ce00640 1 -- 172.21.15.133:0/3967911192 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a280cdee0 msgr2=0x7f9a2815c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:35.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.054+0000 7f9a2ce00640 1 --2- 172.21.15.133:0/3967911192 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a280cdee0 0x7f9a2815c550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009870 tx=0x7f9a2002b8f0 comp rx=0 tx=0).stop 2024-03-07T06:54:35.055 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.054+0000 7f9a2ce00640 1 -- 172.21.15.133:0/3967911192 shutdown_connections 2024-03-07T06:54:35.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.054+0000 7f9a2ce00640 1 --2- 172.21.15.133:0/3967911192 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a280cdee0 0x7f9a2815c550 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:35.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.054+0000 7f9a2ce00640 1 --2- 172.21.15.133:0/3967911192 >> v2:172.21.15.161:3300/0 conn(0x7f9a2815fbb0 0x7f9a280cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:35.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.054+0000 7f9a2ce00640 1 -- 172.21.15.133:0/3967911192 >> 172.21.15.133:0/3967911192 conn(0x7f9a280b3120 msgr2=0x7f9a280b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:35.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.054+0000 7f9a2ce00640 1 -- 172.21.15.133:0/3967911192 shutdown_connections 2024-03-07T06:54:35.056 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.055+0000 7f9a2ce00640 1 -- 172.21.15.133:0/3967911192 wait complete. 2024-03-07T06:54:35.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.056+0000 7f9a2ce00640 1 Processor -- start 2024-03-07T06:54:35.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.057+0000 7f9a2ce00640 1 -- start start 2024-03-07T06:54:35.057 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.058+0000 7f9a2ce00640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a280cdee0 0x7f9a2814dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:35.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.058+0000 7f9a2ce00640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a2815fbb0 0x7f9a280aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:35.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.058+0000 7f9a2ce00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a28159cc0 0x7f9a2815a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:35.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.058+0000 7f9a2ce00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a280ce2c0 con 0x7f9a28159cc0 2024-03-07T06:54:35.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.058+0000 7f9a2ce00640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a28160980 con 0x7f9a2815fbb0 2024-03-07T06:54:35.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.058+0000 7f9a2ce00640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a280b1a00 con 0x7f9a280cdee0 2024-03-07T06:54:35.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.058+0000 7f9a26800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a28159cc0 0x7f9a2815a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:35.058 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.058+0000 7f9a26800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a28159cc0 0x7f9a2815a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54318/0 (socket says 172.21.15.133:54318) 2024-03-07T06:54:35.059 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.058+0000 7f9a26800640 1 -- 172.21.15.133:0/1992002439 learned_addr learned my addr 172.21.15.133:0/1992002439 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-03-07T06:54:35.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.058+0000 7f9a25400640 1 --2- 172.21.15.133:0/1992002439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a2815fbb0 0x7f9a280aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:35.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.058+0000 7f9a25e00640 1 --2- 172.21.15.133:0/1992002439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a280cdee0 0x7f9a2814dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:35.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.059+0000 7f9a26800640 1 -- 172.21.15.133:0/1992002439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a280cdee0 msgr2=0x7f9a2814dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:35.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.059+0000 7f9a26800640 1 --2- 172.21.15.133:0/1992002439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a280cdee0 0x7f9a2814dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:35.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.059+0000 7f9a26800640 1 -- 172.21.15.133:0/1992002439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a2815fbb0 msgr2=0x7f9a280aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:35.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.059+0000 7f9a26800640 1 --2- 172.21.15.133:0/1992002439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a2815fbb0 0x7f9a280aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:35.060 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.059+0000 7f9a26800640 1 -- 172.21.15.133:0/1992002439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2806e240 con 0x7f9a28159cc0 2024-03-07T06:54:35.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.059+0000 7f9a26800640 1 --2- 172.21.15.133:0/1992002439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a28159cc0 0x7f9a2815a150 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009870 tx=0x7f9a200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:35.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.059+0000 7f9a1e000640 1 -- 172.21.15.133:0/1992002439 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a20040070 con 0x7f9a28159cc0 2024-03-07T06:54:35.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.059+0000 7f9a1e000640 1 -- 172.21.15.133:0/1992002439 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9a200399f0 con 0x7f9a28159cc0 2024-03-07T06:54:35.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.059+0000 7f9a1e000640 1 -- 172.21.15.133:0/1992002439 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a20035040 con 0x7f9a28159cc0 2024-03-07T06:54:35.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.059+0000 7f9a2ce00640 1 -- 172.21.15.133:0/1992002439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2806e520 con 0x7f9a28159cc0 2024-03-07T06:54:35.061 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.060+0000 7f9a2ce00640 1 -- 172.21.15.133:0/1992002439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2814e690 con 0x7f9a28159cc0 2024-03-07T06:54:35.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.061+0000 7f9a2ce00640 1 -- 172.21.15.133:0/1992002439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a28062c90 con 0x7f9a28159cc0 2024-03-07T06:54:35.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.062+0000 7f9a1e000640 1 -- 172.21.15.133:0/1992002439 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 89778+0+0 (secure 0 0 0) 0x7f9a2004b060 con 0x7f9a28159cc0 2024-03-07T06:54:35.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.062+0000 7f9a1e000640 1 --2- 172.21.15.133:0/1992002439 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9a0406b4d0 0x7f9a0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-07T06:54:35.062 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.063+0000 7f9a1e000640 1 -- 172.21.15.133:0/1992002439 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9a200aba30 con 0x7f9a28159cc0 2024-03-07T06:54:35.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.063+0000 7f9a25e00640 1 --2- 172.21.15.133:0/1992002439 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9a0406b4d0 0x7f9a0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-07T06:54:35.063 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.064+0000 7f9a25e00640 1 --2- 172.21.15.133:0/1992002439 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9a0406b4d0 0x7f9a0406d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009a10 tx=0x7f9a10008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-07T06:54:35.066 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.067+0000 7f9a1e000640 1 -- 172.21.15.133:0/1992002439 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183808 (secure 0 0 0) 0x7f9a2002f300 con 0x7f9a28159cc0 2024-03-07T06:54:35.300 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.300+0000 7f9a2ce00640 1 -- 172.21.15.133:0/1992002439 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a280b0e80 con 0x7f9a28159cc0 2024-03-07T06:54:35.301 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.301+0000 7f9a1e000640 1 -- 172.21.15.133:0/1992002439 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f9a2007aa00 con 0x7f9a28159cc0 2024-03-07T06:54:35.301 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-03-07T06:54:35.301 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4775,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/44975892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":44975892},{"type":"v1","addr":"172.21.15.161:6835","nonce":44975892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":4972,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6833/906115542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":906115542},{"type":"v1","addr":"172.21.15.133:6833","nonce":906115542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6837/1998424684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6836","nonce":1998424684},{"type":"v1","addr":"172.21.15.161:6837","nonce":1998424684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22},{"gid":5877,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/3665155599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3665155599},{"type":"v1","addr":"172.21.15.133:6835","nonce":3665155599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[]} 2024-03-07T06:54:35.302 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 29 2024-03-07T06:54:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.304+0000 7f9a2ce00640 1 -- 172.21.15.133:0/1992002439 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9a0406b4d0 msgr2=0x7f9a0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.304+0000 7f9a2ce00640 1 --2- 172.21.15.133:0/1992002439 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9a0406b4d0 0x7f9a0406d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9a10009a10 tx=0x7f9a10008040 comp rx=0 tx=0).stop 2024-03-07T06:54:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.304+0000 7f9a2ce00640 1 -- 172.21.15.133:0/1992002439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a28159cc0 msgr2=0x7f9a2815a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-07T06:54:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.304+0000 7f9a2ce00640 1 --2- 172.21.15.133:0/1992002439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a28159cc0 0x7f9a2815a150 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9a20009870 tx=0x7f9a200046c0 comp rx=0 tx=0).stop 2024-03-07T06:54:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.304+0000 7f9a2ce00640 1 -- 172.21.15.133:0/1992002439 shutdown_connections 2024-03-07T06:54:35.304 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.304+0000 7f9a2ce00640 1 --2- 172.21.15.133:0/1992002439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a28159cc0 0x7f9a2815a150 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:35.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.304+0000 7f9a2ce00640 1 --2- 172.21.15.133:0/1992002439 >> [v2:172.21.15.161:6800/187879951,v1:172.21.15.161:6801/187879951] conn(0x7f9a0406b4d0 0x7f9a0406d990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:35.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.304+0000 7f9a2ce00640 1 --2- 172.21.15.133:0/1992002439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9a2815fbb0 0x7f9a280aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:35.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.304+0000 7f9a2ce00640 1 --2- 172.21.15.133:0/1992002439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9a280cdee0 0x7f9a2814dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-07T06:54:35.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.304+0000 7f9a2ce00640 1 -- 172.21.15.133:0/1992002439 >> 172.21.15.133:0/1992002439 conn(0x7f9a280b3120 msgr2=0x7f9a2815ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-07T06:54:35.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.304+0000 7f9a2ce00640 1 -- 172.21.15.133:0/1992002439 shutdown_connections 2024-03-07T06:54:35.305 INFO:teuthology.orchestra.run.smithi133.stderr:2024-03-07T06:54:35.304+0000 7f9a2ce00640 1 -- 172.21.15.133:0/1992002439 wait complete. 2024-03-07T06:54:35.322 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-07T06:54:35.323 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-07T06:54:35.323 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-07T06:54:35.323 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:35.348 INFO:tasks.ceph.mds.a:Stopped 2024-03-07T06:54:35.348 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-07T06:54:35.348 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:35.417 INFO:tasks.ceph.mds.c:Stopped 2024-03-07T06:54:35.418 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-07T06:54:35.418 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:35.449 INFO:tasks.ceph.mds.b:Stopped 2024-03-07T06:54:35.449 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-07T06:54:35.449 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:35.516 INFO:tasks.ceph.mds.d:Stopped 2024-03-07T06:54:35.517 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-07T06:54:35.517 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-07T06:54:35.517 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:35.571 INFO:tasks.ceph.osd.0:Stopped 2024-03-07T06:54:35.572 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-07T06:54:35.572 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:35.622 INFO:tasks.ceph.osd.1:Stopped 2024-03-07T06:54:35.623 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-07T06:54:35.623 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:35.681 INFO:tasks.ceph.osd.2:Stopped 2024-03-07T06:54:35.682 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-07T06:54:35.682 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:35.751 INFO:tasks.ceph.osd.3:Stopped 2024-03-07T06:54:35.751 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-07T06:54:35.751 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:35.798 INFO:tasks.ceph.osd.4:Stopped 2024-03-07T06:54:35.798 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-07T06:54:35.798 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:35.867 INFO:tasks.ceph.osd.5:Stopped 2024-03-07T06:54:35.867 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-07T06:54:35.867 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:35.938 INFO:tasks.ceph.osd.6:Stopped 2024-03-07T06:54:35.938 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-07T06:54:35.939 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:36.002 INFO:tasks.ceph.osd.7:Stopped 2024-03-07T06:54:36.003 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-07T06:54:36.003 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-07T06:54:36.003 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:40.121 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-03-07T06:54:42.105 INFO:tasks.ceph.mgr.y:Stopped 2024-03-07T06:54:42.105 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-07T06:54:42.105 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:42.189 INFO:tasks.ceph.mgr.x:Stopped 2024-03-07T06:54:42.189 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-07T06:54:42.189 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-07T06:54:42.189 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:42.236 INFO:tasks.ceph.mon.a:Stopped 2024-03-07T06:54:42.236 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-07T06:54:42.237 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:42.270 INFO:tasks.ceph.mon.b:Stopped 2024-03-07T06:54:42.270 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-07T06:54:42.270 INFO:teuthology.orchestra.run:waiting for 300 2024-03-07T06:54:42.345 INFO:tasks.ceph.mon.c:Stopped 2024-03-07T06:54:42.345 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-07T06:54:42.345 DEBUG:teuthology.orchestra.run.smithi133:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-07T06:54:42.451 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:54:42.452 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-07T06:54:42.736 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:54:42.736 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-07T06:54:42.889 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:54:42.890 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-07T06:54:43.063 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:54:43.064 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-07T06:54:43.228 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi161.front.sepia.ceph.com 2024-03-07T06:54:43.228 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-07T06:54:43.492 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi161.front.sepia.ceph.com 2024-03-07T06:54:43.493 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-07T06:54:43.630 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi161.front.sepia.ceph.com 2024-03-07T06:54:43.630 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-07T06:54:43.808 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi161.front.sepia.ceph.com 2024-03-07T06:54:43.808 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-07T06:54:44.010 INFO:tasks.ceph:Archiving mon data... 2024-03-07T06:54:44.011 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926/data/mon.a.tgz 2024-03-07T06:54:44.011 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-03-07T06:54:44.065 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.xB684zjDZ8 2024-03-07T06:54:44.066 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.xB684zjDZ8 2024-03-07T06:54:44.243 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.xB684zjDZ8 2024-03-07T06:54:44.386 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.xB684zjDZ8 is 336KB 2024-03-07T06:54:44.495 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.xB684zjDZ8 2024-03-07T06:54:44.550 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926/data/mon.b.tgz 2024-03-07T06:54:44.551 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2024-03-07T06:54:44.605 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.fwyaJdZ1TO 2024-03-07T06:54:44.606 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.fwyaJdZ1TO 2024-03-07T06:54:44.808 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.fwyaJdZ1TO 2024-03-07T06:54:44.952 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.fwyaJdZ1TO is 444KB 2024-03-07T06:54:45.063 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.fwyaJdZ1TO 2024-03-07T06:54:45.120 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926/data/mon.c.tgz 2024-03-07T06:54:45.120 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2024-03-07T06:54:45.122 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-03-07T06:54:45.169 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.rsNxh3B5aE 2024-03-07T06:54:45.169 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.rsNxh3B5aE 2024-03-07T06:54:45.358 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.rsNxh3B5aE 2024-03-07T06:54:45.504 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.rsNxh3B5aE is 444KB 2024-03-07T06:54:45.611 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.rsNxh3B5aE 2024-03-07T06:54:45.667 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-07T06:54:45.667 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-07T06:54:45.675 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-07T06:54:45.828 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-07T06:54:45.887 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-07T06:54:45.887 INFO:tasks.ceph:Archiving crash dumps... 2024-03-07T06:54:45.888 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926/remote/smithi133/crash 2024-03-07T06:54:45.889 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-07T06:54:45.988 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926/remote/smithi161/crash 2024-03-07T06:54:45.989 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-07T06:54:46.084 INFO:tasks.ceph:Shutting down logrotate 2024-03-07T06:54:46.085 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:54:46.092 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-07T06:54:46.240 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-07T06:54:46.330 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-07T06:54:46.429 INFO:tasks.ceph:Compressing logs... 2024-03-07T06:54:46.430 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-07T06:54:46.440 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-07T06:54:46.524 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54666.log 2024-03-07T06:54:46.525 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49099.log 2024-03-07T06:54:46.526 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54666.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54666.log.gz 2024-03-07T06:54:46.527 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52129.log 2024-03-07T06:54:46.528 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44114.log 2024-03-07T06:54:46.528 INFO:teuthology.orchestra.run.smithi133.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49099.log.gz 2024-03-07T06:54:46.529 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52129.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52129.log.gz 2024-03-07T06:54:46.529 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49448.log 2024-03-07T06:54:46.530 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44114.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44114.log.gz 2024-03-07T06:54:46.531 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46966.log 2024-03-07T06:54:46.531 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-07T06:54:46.532 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51963.log 2024-03-07T06:54:46.533 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49448.log.gz 2024-03-07T06:54:46.533 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46966.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-07T06:54:46.533 INFO:teuthology.orchestra.run.smithi133.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46966.log.gz 2024-03-07T06:54:46.541 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.43425.log 2024-03-07T06:54:46.541 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43018.log 2024-03-07T06:54:46.541 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.3.43425.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.43425.log.gz 2024-03-07T06:54:46.541 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37192.log 2024-03-07T06:54:46.542 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.1.43018.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-07T06:54:46.542 INFO:teuthology.orchestra.run.smithi161.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43018.log.gz 2024-03-07T06:54:46.542 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.37192.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37192.log.gz 2024-03-07T06:54:46.542 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-07T06:54:46.542 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37112.log 2024-03-07T06:54:46.543 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-07T06:54:46.543 INFO:teuthology.orchestra.run.smithi161.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-07T06:54:46.543 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.37112.log: 92.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-07T06:54:46.543 INFO:teuthology.orchestra.run.smithi161.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37112.log.gz 2024-03-07T06:54:46.543 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37232.log 2024-03-07T06:54:46.543 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37072.log 2024-03-07T06:54:46.546 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51963.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51963.log.gz 2024-03-07T06:54:46.547 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44431.log 2024-03-07T06:54:46.547 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50793.log 2024-03-07T06:54:46.547 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44431.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44431.log.gz 2024-03-07T06:54:46.547 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42519.log 2024-03-07T06:54:46.547 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50793.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50793.log.gz 2024-03-07T06:54:46.548 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45115.log 2024-03-07T06:54:46.548 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47869.log 2024-03-07T06:54:46.548 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42519.log.gz 2024-03-07T06:54:46.548 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49313.log 2024-03-07T06:54:46.548 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45115.log.gz 2024-03-07T06:54:46.548 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55422.log 2024-03-07T06:54:46.549 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47869.log.gz 2024-03-07T06:54:46.549 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43461.log 2024-03-07T06:54:46.549 INFO:teuthology.orchestra.run.smithi133.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49313.log.gz 2024-03-07T06:54:46.549 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46738.log 2024-03-07T06:54:46.549 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55422.log.gz 2024-03-07T06:54:46.549 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43461.log: gzip -5 87.8% --verbose -- replaced with /var/log/ceph/ceph-client.admin.43461.log.gz 2024-03-07T06:54:46.550 INFO:teuthology.orchestra.run.smithi133.stderr: -- /var/log/ceph/ceph-client.admin.53935.log 2024-03-07T06:54:46.550 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45894.log 2024-03-07T06:54:46.550 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46738.log.gz 2024-03-07T06:54:46.550 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51578.log 2024-03-07T06:54:46.550 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53935.log.gz 2024-03-07T06:54:46.550 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.2.45894.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45894.log.gz 2024-03-07T06:54:46.550 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43111.log 2024-03-07T06:54:46.558 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.37232.log: gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-osd.7.log -- replaced with /var/log/ceph/ceph-client.admin.37232.log.gz 2024-03-07T06:54:46.558 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-03-07T06:54:46.559 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.37072.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-07T06:54:46.559 INFO:teuthology.orchestra.run.smithi161.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37072.log.gz 2024-03-07T06:54:46.559 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37152.log 2024-03-07T06:54:46.559 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37363.log 2024-03-07T06:54:46.559 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.37152.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-07T06:54:46.559 INFO:teuthology.orchestra.run.smithi161.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37152.log.gz 2024-03-07T06:54:46.559 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.37363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37272.log 2024-03-07T06:54:46.560 INFO:teuthology.orchestra.run.smithi161.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37363.log.gz 2024-03-07T06:54:46.560 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-07T06:54:46.560 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.37272.log: gzip -5 87.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.37272.log.gz 2024-03-07T06:54:46.560 INFO:teuthology.orchestra.run.smithi161.stderr: /var/log/ceph/ceph-client.admin.37312.log 2024-03-07T06:54:46.560 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-07T06:54:46.560 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.37312.log: 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-07T06:54:46.561 INFO:teuthology.orchestra.run.smithi161.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37312.log.gz 2024-03-07T06:54:46.561 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-07T06:54:46.561 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-07T06:54:46.566 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51578.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51578.log.gzgzip 2024-03-07T06:54:46.566 INFO:teuthology.orchestra.run.smithi133.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43457.log 2024-03-07T06:54:46.566 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48163.log 2024-03-07T06:54:46.567 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43111.log.gz 2024-03-07T06:54:46.567 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46612.log 2024-03-07T06:54:46.567 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43457.log.gz 2024-03-07T06:54:46.567 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45594.log 2024-03-07T06:54:46.567 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48163.log.gz 2024-03-07T06:54:46.567 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46612.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46612.log.gz 2024-03-07T06:54:46.568 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44347.log 2024-03-07T06:54:46.568 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.45594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46528.log 2024-03-07T06:54:46.568 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45594.log.gz 2024-03-07T06:54:46.568 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44347.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44347.log.gz 2024-03-07T06:54:46.568 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49494.log 2024-03-07T06:54:46.568 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46528.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46528.log.gz 2024-03-07T06:54:46.568 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44809.log 2024-03-07T06:54:46.568 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49494.log: gzip 89.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.49494.log.gz --verbose 2024-03-07T06:54:46.569 INFO:teuthology.orchestra.run.smithi133.stderr: -- /var/log/ceph/ceph-client.admin.55128.log 2024-03-07T06:54:46.569 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44809.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.46150.log -- replaced with /var/log/ceph/ceph-client.admin.44809.log.gz 2024-03-07T06:54:46.569 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-03-07T06:54:46.569 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55128.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55128.log.gz 2024-03-07T06:54:46.569 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48373.log 2024-03-07T06:54:46.569 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43458.log 2024-03-07T06:54:46.569 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46150.log.gz 2024-03-07T06:54:46.570 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48373.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48373.log.gz 2024-03-07T06:54:46.570 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55086.log 2024-03-07T06:54:46.570 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46924.log 2024-03-07T06:54:46.570 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43458.log.gz 2024-03-07T06:54:46.570 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50835.log 89.2% 2024-03-07T06:54:46.570 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55086.log.gz 2024-03-07T06:54:46.570 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50196.log 2024-03-07T06:54:46.570 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46924.log.gz 2024-03-07T06:54:46.571 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52352.log 89.4% 2024-03-07T06:54:46.571 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50835.log.gz 2024-03-07T06:54:46.571 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47008.log 2024-03-07T06:54:46.571 INFO:teuthology.orchestra.run.smithi133.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.50196.log.gz 2024-03-07T06:54:46.571 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52352.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52352.log.gz 2024-03-07T06:54:46.571 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54163.log 2024-03-07T06:54:46.575 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47008.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47008.log.gz 2024-03-07T06:54:46.575 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44767.log 2024-03-07T06:54:46.576 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51536.log 2024-03-07T06:54:46.576 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54163.log.gz 2024-03-07T06:54:46.576 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43797.log 2024-03-07T06:54:46.576 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44767.log.gz 2024-03-07T06:54:46.576 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51536.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30901.log 2024-03-07T06:54:46.576 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51536.log.gz 2024-03-07T06:54:46.577 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43797.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43797.log.gz 2024-03-07T06:54:46.577 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54392.log 2024-03-07T06:54:46.577 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.tmp-client.admin.30901.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30901.log.gz 2024-03-07T06:54:46.577 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55254.log 2024-03-07T06:54:46.577 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54392.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54392.log.gz -5 2024-03-07T06:54:46.577 INFO:teuthology.orchestra.run.smithi133.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52836.log 2024-03-07T06:54:46.577 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37242.log 2024-03-07T06:54:46.578 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55254.log.gz 2024-03-07T06:54:46.578 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52836.log: gzip -5 --verbose -- 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52836.log.gz /var/log/ceph/ceph-client.admin.50751.log 2024-03-07T06:54:46.578 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-03-07T06:54:46.581 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51349.log 2024-03-07T06:54:46.581 INFO:teuthology.orchestra.run.smithi133.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37242.log.gz 2024-03-07T06:54:46.581 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50751.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50751.log.gz 2024-03-07T06:54:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49704.log 2024-03-07T06:54:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49578.log 2024-03-07T06:54:46.582 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51349.log.gz 2024-03-07T06:54:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51139.log 2024-03-07T06:54:46.582 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49704.log.gz 2024-03-07T06:54:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49578.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49578.log.gz 2024-03-07T06:54:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47365.log 2024-03-07T06:54:46.584 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51139.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51139.log.gz 2024-03-07T06:54:46.584 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52794.log 2024-03-07T06:54:46.584 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53293.log 2024-03-07T06:54:46.584 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47365.log.gz 2024-03-07T06:54:46.584 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49406.log 2024-03-07T06:54:46.584 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52794.log.gz 2024-03-07T06:54:46.585 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52086.log 2024-03-07T06:54:46.586 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53293.log.gz 2024-03-07T06:54:46.586 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42943.log 2024-03-07T06:54:46.586 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49406.log.gz 2024-03-07T06:54:46.587 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51494.log 2024-03-07T06:54:46.587 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52086.log.gz 2024-03-07T06:54:46.588 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48541.log 2024-03-07T06:54:46.588 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42943.log.gz 2024-03-07T06:54:46.588 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44132.log 2024-03-07T06:54:46.589 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51494.log.gz 2024-03-07T06:54:46.589 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43279.log 2024-03-07T06:54:46.590 INFO:teuthology.orchestra.run.smithi133.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48541.log.gz 2024-03-07T06:54:46.591 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44132.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44132.log.gz 2024-03-07T06:54:46.591 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50614.log 2024-03-07T06:54:46.592 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46780.log 2024-03-07T06:54:46.592 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43279.log.gz 2024-03-07T06:54:46.593 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42605.log 2024-03-07T06:54:46.593 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50614.log.gz 2024-03-07T06:54:46.594 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46780.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46780.log.gz 2024-03-07T06:54:46.594 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48205.log 2024-03-07T06:54:46.595 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44935.log 2024-03-07T06:54:46.595 INFO:teuthology.orchestra.run.smithi133.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42605.log.gz 2024-03-07T06:54:46.596 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54582.log 2024-03-07T06:54:46.596 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48205.log.gz 2024-03-07T06:54:46.597 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47287.log 2024-03-07T06:54:46.597 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44935.log.gz 2024-03-07T06:54:46.598 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37369.log 2024-03-07T06:54:46.598 INFO:teuthology.orchestra.run.smithi133.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54582.log.gz 2024-03-07T06:54:46.599 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48037.log 2024-03-07T06:54:46.599 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47287.log.gz 2024-03-07T06:54:46.600 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.37369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46822.log 2024-03-07T06:54:46.600 INFO:teuthology.orchestra.run.smithi133.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37369.log.gz 2024-03-07T06:54:46.601 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48583.log 2024-03-07T06:54:46.601 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48037.log.gz 2024-03-07T06:54:46.601 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54750.log 2024-03-07T06:54:46.602 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46822.log.gz 2024-03-07T06:54:46.602 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43445.log 2024-03-07T06:54:46.603 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48583.log.gz 2024-03-07T06:54:46.604 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46402.log 2024-03-07T06:54:46.604 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54750.log.gz 2024-03-07T06:54:46.605 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50498.log 2024-03-07T06:54:46.605 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43445.log.gz 2024-03-07T06:54:46.606 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52436.log 2024-03-07T06:54:46.606 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46402.log.gz 2024-03-07T06:54:46.607 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46108.log 2024-03-07T06:54:46.607 INFO:teuthology.orchestra.run.smithi133.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.50498.log.gz 2024-03-07T06:54:46.611 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52436.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-07T06:54:46.611 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52436.log.gz 2024-03-07T06:54:46.613 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44893.log 2024-03-07T06:54:46.614 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46108.log.gz 2024-03-07T06:54:46.614 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42092.log 2024-03-07T06:54:46.614 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47491.log 2024-03-07T06:54:46.614 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44893.log.gz 2024-03-07T06:54:46.614 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-07T06:54:46.614 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47161.log 2024-03-07T06:54:46.614 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42092.log.gz 2024-03-07T06:54:46.614 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54266.log 2024-03-07T06:54:46.614 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47491.log.gz 2024-03-07T06:54:46.616 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 94.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-07T06:54:46.617 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52482.log 2024-03-07T06:54:46.617 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47161.log.gz 2024-03-07T06:54:46.617 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42733.log 2024-03-07T06:54:46.617 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54266.log.gz 2024-03-07T06:54:46.618 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48871.log 2024-03-07T06:54:46.618 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52482.log.gz 2024-03-07T06:54:46.618 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50554.log 2024-03-07T06:54:46.618 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42733.log.gz 2024-03-07T06:54:46.619 INFO:teuthology.orchestra.run.smithi161.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-07T06:54:46.620 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53683.log 2024-03-07T06:54:46.620 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48871.log.gz 2024-03-07T06:54:46.621 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44725.log 2024-03-07T06:54:46.621 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50554.log.gz 2024-03-07T06:54:46.621 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52257.log 2024-03-07T06:54:46.621 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53683.log.gz 2024-03-07T06:54:46.623 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42350.log 2024-03-07T06:54:46.623 INFO:teuthology.orchestra.run.smithi133.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44725.log.gz 2024-03-07T06:54:46.623 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54834.log 2024-03-07T06:54:46.623 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52257.log.gz 2024-03-07T06:54:46.624 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46696.log 2024-03-07T06:54:46.624 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42350.log.gz 2024-03-07T06:54:46.625 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46318.log 2024-03-07T06:54:46.625 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54834.log.gz 2024-03-07T06:54:46.625 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49364.log 2024-03-07T06:54:46.626 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46696.log.gz 2024-03-07T06:54:46.626 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43027.log 2024-03-07T06:54:46.626 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46318.log.gz 2024-03-07T06:54:46.627 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52301.log 2024-03-07T06:54:46.627 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49364.log.gz 2024-03-07T06:54:46.628 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47911.log 2024-03-07T06:54:46.628 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43027.log.gz 2024-03-07T06:54:46.629 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52301.log: gzip -5 --verbose -- 89.0% /var/log/ceph/ceph-client.admin.47407.log -- replaced with /var/log/ceph/ceph-client.admin.52301.log.gz 2024-03-07T06:54:46.629 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-03-07T06:54:46.630 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53463.log 2024-03-07T06:54:46.631 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47911.log.gz 2024-03-07T06:54:46.631 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47407.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47407.log.gz 2024-03-07T06:54:46.632 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47995.log 2024-03-07T06:54:46.632 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53463.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53463.log.gz 2024-03-07T06:54:46.632 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49015.log 2024-03-07T06:54:46.633 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50455.log 2024-03-07T06:54:46.634 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47995.log.gz 2024-03-07T06:54:46.634 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49015.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49015.log.gz 2024-03-07T06:54:46.635 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49854.log 2024-03-07T06:54:46.636 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51097.log 2024-03-07T06:54:46.636 INFO:teuthology.orchestra.run.smithi133.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50455.log.gz 2024-03-07T06:54:46.637 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49854.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49854.log.gz 2024-03-07T06:54:46.637 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44125.log 2024-03-07T06:54:46.638 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51097.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51097.log.gz 2024-03-07T06:54:46.638 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49269.log 2024-03-07T06:54:46.639 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54876.log 2024-03-07T06:54:46.640 INFO:teuthology.orchestra.run.smithi133.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44125.log.gz 2024-03-07T06:54:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49269.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49269.log.gz 2024-03-07T06:54:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53725.log 2024-03-07T06:54:46.641 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47665.log 2024-03-07T06:54:46.641 INFO:teuthology.orchestra.run.smithi133.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54876.log.gz 2024-03-07T06:54:46.642 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53725.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53725.log.gz 2024-03-07T06:54:46.643 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46192.log 2024-03-07T06:54:46.644 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47665.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-07T06:54:46.644 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46192.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47665.log.gz 2024-03-07T06:54:46.644 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46192.log.gz 2024-03-07T06:54:46.645 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53851.log 2024-03-07T06:54:46.646 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53028.log 2024-03-07T06:54:46.647 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53851.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53851.log.gz 2024-03-07T06:54:46.647 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54061.log 2024-03-07T06:54:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53028.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53028.log.gz 2024-03-07T06:54:46.648 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52171.log 2024-03-07T06:54:46.649 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54061.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54061.log.gz 2024-03-07T06:54:46.649 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53893.log 2024-03-07T06:54:46.650 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52171.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52171.log.gz 2024-03-07T06:54:46.650 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47203.log 2024-03-07T06:54:46.651 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53893.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53893.log.gz 2024-03-07T06:54:46.651 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54019.log 2024-03-07T06:54:46.652 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47203.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47203.log.gz 2024-03-07T06:54:46.652 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43789.log 2024-03-07T06:54:46.653 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54019.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54019.log.gz 2024-03-07T06:54:46.653 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48745.log 2024-03-07T06:54:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43789.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43789.log.gz 2024-03-07T06:54:46.654 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52394.log 2024-03-07T06:54:46.655 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48745.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48745.log.gz 2024-03-07T06:54:46.655 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42859.log 2024-03-07T06:54:46.656 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52394.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52394.log.gz 2024-03-07T06:54:46.656 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51307.log 2024-03-07T06:54:46.657 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42859.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42859.log.gz 2024-03-07T06:54:46.658 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52710.log 2024-03-07T06:54:46.658 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51307.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51307.log.gz 2024-03-07T06:54:46.659 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51223.log 2024-03-07T06:54:46.659 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52710.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52710.log.gz 2024-03-07T06:54:46.660 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50877.log 2024-03-07T06:54:46.660 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51223.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51223.log.gz 2024-03-07T06:54:46.661 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44641.log 2024-03-07T06:54:46.661 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50877.log: gzip -5 --verbose 89.3% -- -- replaced with /var/log/ceph/ceph-client.admin.50877.log.gz 2024-03-07T06:54:46.662 INFO:teuthology.orchestra.run.smithi133.stderr: /var/log/ceph/ceph-client.admin.51620.log 2024-03-07T06:54:46.663 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43460.log 2024-03-07T06:54:46.663 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44641.log.gz 2024-03-07T06:54:46.663 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51620.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.51620.log.gz --verbose 2024-03-07T06:54:46.664 INFO:teuthology.orchestra.run.smithi133.stderr: -- /var/log/ceph/ceph-client.admin.55296.log 2024-03-07T06:54:46.665 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43460.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43460.log.gz 2024-03-07T06:54:46.665 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54960.log 2024-03-07T06:54:46.666 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55296.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55296.log.gz 2024-03-07T06:54:46.666 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52524.log 2024-03-07T06:54:46.667 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53589.log 2024-03-07T06:54:46.667 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54960.log.gz 2024-03-07T06:54:46.668 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44977.log 2024-03-07T06:54:46.668 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52524.log.gz 2024-03-07T06:54:46.669 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53589.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53589.log.gz 2024-03-07T06:54:46.669 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50041.log 2024-03-07T06:54:46.670 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42477.log 2024-03-07T06:54:46.670 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44977.log.gz 2024-03-07T06:54:46.670 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.50041.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50041.log.gz 2024-03-07T06:54:46.671 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47449.log 2024-03-07T06:54:46.672 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42477.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42477.log.gz 2024-03-07T06:54:46.672 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42392.log 2024-03-07T06:54:46.673 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42435.log 2024-03-07T06:54:46.673 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47449.log.gz 2024-03-07T06:54:46.674 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42392.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42392.log.gz 2024-03-07T06:54:46.674 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51391.log 2024-03-07T06:54:46.675 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55482.log 2024-03-07T06:54:46.676 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42435.log.gz 2024-03-07T06:54:46.677 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51391.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51391.log.gz 2024-03-07T06:54:46.677 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48331.log 2024-03-07T06:54:46.677 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53977.log 2024-03-07T06:54:46.678 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55482.log.gz 2024-03-07T06:54:46.679 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48331.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48331.log.gz 2024-03-07T06:54:46.679 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55002.log 2024-03-07T06:54:46.680 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51055.log 2024-03-07T06:54:46.680 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53977.log.gz 2024-03-07T06:54:46.681 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55002.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55002.log.gz 2024-03-07T06:54:46.681 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55338.log 2024-03-07T06:54:46.682 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53505.log 2024-03-07T06:54:46.683 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51055.log.gz 2024-03-07T06:54:46.683 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55338.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.55338.log.gz 2024-03-07T06:54:46.684 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42561.log 2024-03-07T06:54:46.684 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42901.log 2024-03-07T06:54:46.685 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53505.log.gz 2024-03-07T06:54:46.686 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44124.log 2024-03-07T06:54:46.686 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42561.log.gz 2024-03-07T06:54:46.687 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49536.log 2024-03-07T06:54:46.687 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42901.log.gz 2024-03-07T06:54:46.688 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44124.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44124.log.gz -5 2024-03-07T06:54:46.688 INFO:teuthology.orchestra.run.smithi133.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46444.log 2024-03-07T06:54:46.689 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44515.log 2024-03-07T06:54:46.690 INFO:teuthology.orchestra.run.smithi133.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49536.log.gz 2024-03-07T06:54:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46444.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-07T06:54:46.691 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46444.log.gz 2024-03-07T06:54:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48829.log 2024-03-07T06:54:46.692 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44515.log.gz 2024-03-07T06:54:46.693 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53767.log 2024-03-07T06:54:46.694 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48829.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48829.log.gz 2024-03-07T06:54:46.695 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46654.log 2024-03-07T06:54:46.696 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52657.log 2024-03-07T06:54:46.696 INFO:teuthology.orchestra.run.smithi133.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53767.log.gz 2024-03-07T06:54:46.697 INFO:teuthology.orchestra.run.smithi161.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-07T06:54:46.698 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43459.log 2024-03-07T06:54:46.698 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46654.log.gz 2024-03-07T06:54:46.699 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44073.log 2024-03-07T06:54:46.699 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52657.log.gz 2024-03-07T06:54:46.700 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48457.log 2024-03-07T06:54:46.701 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43459.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43459.log.gz 2024-03-07T06:54:46.701 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42817.log 2024-03-07T06:54:46.702 INFO:teuthology.orchestra.run.smithi133.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44073.log.gz 2024-03-07T06:54:46.703 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47623.log 2024-03-07T06:54:46.703 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48457.log.gz 2024-03-07T06:54:46.704 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53547.log 2024-03-07T06:54:46.704 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42817.log.gz 2024-03-07T06:54:46.706 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48247.log 2024-03-07T06:54:46.706 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47623.log.gz 2024-03-07T06:54:46.707 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53421.log 2024-03-07T06:54:46.707 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53547.log.gz 2024-03-07T06:54:46.708 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42985.log 2024-03-07T06:54:46.708 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48247.log.gz 2024-03-07T06:54:46.710 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54308.log 2024-03-07T06:54:46.710 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53421.log.gz 2024-03-07T06:54:46.711 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50971.log 2024-03-07T06:54:46.711 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42985.log.gz 2024-03-07T06:54:46.712 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47245.log 2024-03-07T06:54:46.713 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54308.log.gz 2024-03-07T06:54:46.714 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44683.log 2024-03-07T06:54:46.714 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50971.log.gz 2024-03-07T06:54:46.715 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55380.log 2024-03-07T06:54:46.715 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47245.log.gz 2024-03-07T06:54:46.716 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55522.log 2024-03-07T06:54:46.717 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44683.log.gz 2024-03-07T06:54:46.718 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44851.log 2024-03-07T06:54:46.718 INFO:teuthology.orchestra.run.smithi133.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55380.log.gz 2024-03-07T06:54:46.719 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43447.log 2024-03-07T06:54:46.719 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55522.log.gz 2024-03-07T06:54:46.720 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54350.log 2024-03-07T06:54:46.721 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44851.log.gz 2024-03-07T06:54:46.722 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43447.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43447.log.gz 2024-03-07T06:54:46.722 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43195.log 2024-03-07T06:54:46.723 INFO:teuthology.orchestra.run.smithi161.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-07T06:54:46.723 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46066.log 2024-03-07T06:54:46.723 INFO:teuthology.orchestra.run.smithi133.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54350.log.gz 2024-03-07T06:54:46.724 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43796.log 2024-03-07T06:54:46.725 INFO:teuthology.orchestra.run.smithi133.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43195.log.gz 2024-03-07T06:54:46.726 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51704.log 2024-03-07T06:54:46.726 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46066.log.gz 2024-03-07T06:54:46.727 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42775.log 2024-03-07T06:54:46.727 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43796.log.gz 2024-03-07T06:54:46.728 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43451.log 2024-03-07T06:54:46.729 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51704.log.gz 2024-03-07T06:54:46.730 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46486.log 2024-03-07T06:54:46.730 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42775.log.gz 2024-03-07T06:54:46.731 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44557.log 2024-03-07T06:54:46.731 INFO:teuthology.orchestra.run.smithi133.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43451.log.gz 2024-03-07T06:54:46.732 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42173.log 2024-03-07T06:54:46.733 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46486.log.gz 2024-03-07T06:54:46.734 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52566.log 2024-03-07T06:54:46.734 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44557.log.gz 2024-03-07T06:54:46.735 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47749.log 2024-03-07T06:54:46.735 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42173.log.gz 2024-03-07T06:54:46.736 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49183.log 2024-03-07T06:54:46.737 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52566.log.gz 2024-03-07T06:54:46.738 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46276.log 2024-03-07T06:54:46.738 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47749.log.gz 2024-03-07T06:54:46.739 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46360.log 2024-03-07T06:54:46.739 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49183.log.gz 2024-03-07T06:54:46.740 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42269.log 2024-03-07T06:54:46.741 INFO:teuthology.orchestra.run.smithi133.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46276.log.gz 2024-03-07T06:54:46.742 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51883.log 2024-03-07T06:54:46.742 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46360.log.gz 2024-03-07T06:54:46.743 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43153.log 2024-03-07T06:54:46.743 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42269.log.gz 2024-03-07T06:54:46.744 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47953.log 2024-03-07T06:54:46.746 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51883.log.gz 2024-03-07T06:54:46.746 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48121.log 2024-03-07T06:54:46.746 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43153.log.gz 2024-03-07T06:54:46.747 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47953.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-07T06:54:46.747 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47953.log.gz 2024-03-07T06:54:46.748 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49057.log 2024-03-07T06:54:46.749 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48121.log.gz 2024-03-07T06:54:46.750 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53809.log 2024-03-07T06:54:46.751 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43794.log 2024-03-07T06:54:46.752 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49057.log.gz 2024-03-07T06:54:46.752 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.53809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48625.log 2024-03-07T06:54:46.753 INFO:teuthology.orchestra.run.smithi133.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53809.log.gz 2024-03-07T06:54:46.754 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54624.log 2024-03-07T06:54:46.754 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43794.log.gz 2024-03-07T06:54:46.755 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51265.log 2024-03-07T06:54:46.755 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48625.log.gz 2024-03-07T06:54:46.755 INFO:teuthology.orchestra.run.smithi133.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-07T06:54:46.756 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43781.log 2024-03-07T06:54:46.756 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54624.log.gz 2024-03-07T06:54:46.757 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55564.log 2024-03-07T06:54:46.758 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51265.log.gz 2024-03-07T06:54:46.758 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43781.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43781.log.gz 2024-03-07T06:54:46.758 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43793.log 2024-03-07T06:54:46.759 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50393.log 2024-03-07T06:54:46.760 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55564.log.gz 2024-03-07T06:54:46.761 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43793.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43793.log.gz 2024-03-07T06:54:46.761 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46024.log 2024-03-07T06:54:46.761 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48913.log 2024-03-07T06:54:46.762 INFO:teuthology.orchestra.run.smithi133.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50393.log.gz 2024-03-07T06:54:46.763 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46024.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46024.log.gz 2024-03-07T06:54:46.763 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47119.log 2024-03-07T06:54:46.764 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54792.log 2024-03-07T06:54:46.764 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48913.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48913.log.gz 2024-03-07T06:54:46.765 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47119.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47119.log.gz 2024-03-07T06:54:46.765 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51788.log 2024-03-07T06:54:46.766 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48499.log 2024-03-07T06:54:46.767 INFO:teuthology.orchestra.run.smithi133.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54792.log.gz 2024-03-07T06:54:46.767 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51788.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51788.log.gz 2024-03-07T06:54:46.767 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51451.log 2024-03-07T06:54:46.768 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48499.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-07T06:54:46.769 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48499.log.gz 2024-03-07T06:54:46.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51451.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51451.log.gz 2024-03-07T06:54:46.770 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48079.log 2024-03-07T06:54:46.770 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55044.log 2024-03-07T06:54:46.772 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48079.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48079.log.gz 2024-03-07T06:54:46.772 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54103.log 2024-03-07T06:54:46.773 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44109.log 2024-03-07T06:54:46.773 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55044.log.gz 2024-03-07T06:54:46.774 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44389.log 2024-03-07T06:54:46.774 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54103.log.gz 2024-03-07T06:54:46.775 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49225.log 2024-03-07T06:54:46.775 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44109.log.gz 2024-03-07T06:54:46.777 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44389.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44389.log.gz -5 2024-03-07T06:54:46.777 INFO:teuthology.orchestra.run.smithi133.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52044.log 2024-03-07T06:54:46.778 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53215.log 2024-03-07T06:54:46.778 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49225.log.gz 2024-03-07T06:54:46.779 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52044.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50136.log 2024-03-07T06:54:46.779 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52044.log.gz 2024-03-07T06:54:46.780 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.53215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54918.log 2024-03-07T06:54:46.780 INFO:teuthology.orchestra.run.smithi133.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53215.log.gz 2024-03-07T06:54:46.781 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50136.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50136.log.gz 2024-03-07T06:54:46.781 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-07T06:54:46.782 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54708.log 2024-03-07T06:54:46.783 INFO:teuthology.orchestra.run.smithi133.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54918.log.gz 2024-03-07T06:54:46.783 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48289.log 2024-03-07T06:54:46.784 INFO:teuthology.orchestra.run.smithi133.stderr: 85.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-07T06:54:46.785 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51013.log 2024-03-07T06:54:46.785 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54708.log.gz 2024-03-07T06:54:46.785 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51662.log 2024-03-07T06:54:46.786 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51013.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48289.log.gz 2024-03-07T06:54:46.787 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51013.log.gz 2024-03-07T06:54:46.787 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50709.log 2024-03-07T06:54:46.788 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42691.log 2024-03-07T06:54:46.789 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51662.log.gz 2024-03-07T06:54:46.790 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46570.log 2024-03-07T06:54:46.790 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50709.log.gz 2024-03-07T06:54:46.791 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42691.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42691.log.gz 2024-03-07T06:54:46.791 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-07T06:54:46.792 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43069.log 2024-03-07T06:54:46.792 INFO:teuthology.orchestra.run.smithi133.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46570.log.gz 2024-03-07T06:54:46.793 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50269.log 2024-03-07T06:54:46.794 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54206.log 2024-03-07T06:54:46.795 INFO:teuthology.orchestra.run.smithi133.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43069.log.gz 2024-03-07T06:54:46.796 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43790.log 2024-03-07T06:54:46.796 INFO:teuthology.orchestra.run.smithi133.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50269.log.gz 2024-03-07T06:54:46.797 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.54206.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-07T06:54:46.797 INFO:teuthology.orchestra.run.smithi133.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54206.log.gz 2024-03-07T06:54:46.799 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55212.log 2024-03-07T06:54:46.799 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43790.log.gz 2024-03-07T06:54:46.802 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.55212.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55212.log.gz 2024-03-07T06:54:46.802 INFO:teuthology.orchestra.run.smithi133.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gzgzip 2024-03-07T06:54:46.802 INFO:teuthology.orchestra.run.smithi133.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47533.log 2024-03-07T06:54:46.803 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52878.log 2024-03-07T06:54:46.804 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47533.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47533.log.gz 2024-03-07T06:54:46.808 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50331.log 2024-03-07T06:54:46.812 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52920.log 2024-03-07T06:54:46.812 INFO:teuthology.orchestra.run.smithi133.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52878.log.gz 2024-03-07T06:54:46.812 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.50331.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50331.log.gz 2024-03-07T06:54:46.813 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55170.log 2024-03-07T06:54:46.813 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52920.log.gz 2024-03-07T06:54:46.813 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52608.log 2024-03-07T06:54:46.813 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.55170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44599.log 2024-03-07T06:54:46.813 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55170.log.gz 2024-03-07T06:54:46.816 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47707.log 2024-03-07T06:54:46.817 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52608.log.gz 2024-03-07T06:54:46.817 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44599.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44599.log.gz 2024-03-07T06:54:46.817 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46234.log 2024-03-07T06:54:46.817 INFO:teuthology.orchestra.run.smithi133.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47707.log.gz 2024-03-07T06:54:46.818 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42649.log 2024-03-07T06:54:46.818 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46234.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46234.log.gz 2024-03-07T06:54:46.819 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49620.log 2024-03-07T06:54:46.819 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.42649.log: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-07T06:54:46.819 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44473.log 2024-03-07T06:54:46.820 INFO:teuthology.orchestra.run.smithi133.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42649.log.gz 2024-03-07T06:54:46.821 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49788.log 2024-03-07T06:54:46.821 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44473.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49620.log.gz 2024-03-07T06:54:46.822 INFO:teuthology.orchestra.run.smithi133.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44473.log.gz 2024-03-07T06:54:46.822 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48415.log 2024-03-07T06:54:46.823 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44118.log 2024-03-07T06:54:46.823 INFO:teuthology.orchestra.run.smithi133.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49788.log.gz 2024-03-07T06:54:46.824 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49141.log 2024-03-07T06:54:46.824 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48415.log.gz 2024-03-07T06:54:46.825 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44118.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44118.log.gz 2024-03-07T06:54:46.825 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44133.log 2024-03-07T06:54:46.827 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46864.log 2024-03-07T06:54:46.827 INFO:teuthology.orchestra.run.smithi133.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49141.log.gz 2024-03-07T06:54:46.828 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.44133.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49746.log 2024-03-07T06:54:46.828 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44133.log.gz 2024-03-07T06:54:46.829 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.46864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48787.log 2024-03-07T06:54:46.830 INFO:teuthology.orchestra.run.smithi133.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46864.log.gz 2024-03-07T06:54:46.830 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.49746.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49746.log.gz 2024-03-07T06:54:46.831 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43237.log 2024-03-07T06:54:46.831 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.48787.log: /var/log/ceph/ceph-client.admin.47791.log 2024-03-07T06:54:46.832 INFO:teuthology.orchestra.run.smithi161.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-07T06:54:46.833 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48787.log.gz 2024-03-07T06:54:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43237.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43237.log.gz 2024-03-07T06:54:46.833 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36985.log 2024-03-07T06:54:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.47791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48973.log 2024-03-07T06:54:46.834 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47791.log.gz 2024-03-07T06:54:46.835 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.36985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45445.log 2024-03-07T06:54:46.835 INFO:teuthology.orchestra.run.smithi133.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.36985.log.gz 2024-03-07T06:54:46.836 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.48973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52752.log 2024-03-07T06:54:46.837 INFO:teuthology.orchestra.run.smithi133.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48973.log.gz 2024-03-07T06:54:46.837 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.0.45445.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45445.log.gz 2024-03-07T06:54:46.837 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51181.log 2024-03-07T06:54:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43776.log 2024-03-07T06:54:46.840 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52752.log: /var/log/ceph/ceph-client.admin.51181.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52752.log.gz 2024-03-07T06:54:46.840 INFO:teuthology.orchestra.run.smithi133.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52962.log 2024-03-07T06:54:46.840 INFO:teuthology.orchestra.run.smithi133.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51181.log.gz 2024-03-07T06:54:46.840 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.43776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52213.log 2024-03-07T06:54:46.841 INFO:teuthology.orchestra.run.smithi133.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43776.log.gz 2024-03-07T06:54:46.842 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51746.log 2024-03-07T06:54:46.842 INFO:teuthology.orchestra.run.smithi133.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52962.log.gz 2024-03-07T06:54:46.843 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.52213.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-07T06:54:46.844 INFO:teuthology.orchestra.run.smithi133.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52213.log.gz 2024-03-07T06:54:46.845 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.51746.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51746.log.gz 2024-03-07T06:54:46.845 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49662.log 2024-03-07T06:54:46.846 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.49662.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49662.log.gz 2024-03-07T06:54:46.857 INFO:teuthology.orchestra.run.smithi161.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-07T06:54:46.884 INFO:teuthology.orchestra.run.smithi161.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-07T06:54:46.896 INFO:teuthology.orchestra.run.smithi161.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-07T06:54:46.899 INFO:teuthology.orchestra.run.smithi161.stderr: 2024-03-07T06:54:46.900 INFO:teuthology.orchestra.run.smithi161.stderr:real 0m0.407s 2024-03-07T06:54:46.900 INFO:teuthology.orchestra.run.smithi161.stderr:user 0m1.632s 2024-03-07T06:54:46.900 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m0.248s 2024-03-07T06:54:47.052 INFO:teuthology.orchestra.run.smithi133.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-07T06:54:47.080 INFO:teuthology.orchestra.run.smithi133.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-07T06:54:47.110 INFO:teuthology.orchestra.run.smithi133.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-07T06:54:47.129 INFO:teuthology.orchestra.run.smithi133.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-07T06:54:49.007 INFO:teuthology.orchestra.run.smithi133.stderr: 93.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-07T06:54:49.011 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-03-07T06:54:49.011 INFO:teuthology.orchestra.run.smithi133.stderr:real 0m2.527s 2024-03-07T06:54:49.011 INFO:teuthology.orchestra.run.smithi133.stderr:user 0m3.733s 2024-03-07T06:54:49.011 INFO:teuthology.orchestra.run.smithi133.stderr:sys 0m1.035s 2024-03-07T06:54:49.012 INFO:tasks.ceph:Archiving logs... 2024-03-07T06:54:49.013 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926/remote/smithi133/log 2024-03-07T06:54:49.014 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-07T06:54:49.810 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926/remote/smithi161/log 2024-03-07T06:54:49.812 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-07T06:54:50.110 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-07T06:54:50.120 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-07T06:54:50.120 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-07T06:54:50.128 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-07T06:54:50.294 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-07T06:54:50.303 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-07T06:54:50.304 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-07T06:54:50.311 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-07T06:54:50.358 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-03-07T06:54:50.367 INFO:teuthology.orchestra.run.smithi161.stderr:bash: line 1: ntpq: command not found 2024-03-07T06:54:50.368 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-07T06:54:50.368 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-03-07T06:54:50.368 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 37 +2555us[+2555us] +/- 84ms 2024-03-07T06:54:50.368 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -1024us[-1020us] +/- 51ms 2024-03-07T06:54:50.368 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 -854us[ -849us] +/- 83ms 2024-03-07T06:54:50.368 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T06:54:50.375 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-07T06:54:50.376 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2024-03-07T06:54:50.376 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 38 +2563us[+2563us] +/- 83ms 2024-03-07T06:54:50.376 INFO:teuthology.orchestra.run.smithi161.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -982us[ -975us] +/- 51ms 2024-03-07T06:54:50.376 INFO:teuthology.orchestra.run.smithi161.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 -802us[ -802us] +/- 83ms 2024-03-07T06:54:50.376 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-07T06:54:50.377 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-07T06:54:50.393 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-07T06:54:50.394 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-07T06:54:50.404 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-07T06:54:50.627 DEBUG:teuthology.orchestra.run.smithi161:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-07T06:54:50.841 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-07T06:54:50.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:54:50.943 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-03-07T06:54:51.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:54:51.041 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-07T06:54:51.051 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-07T06:54:51.063 INFO:teuthology.task.internal:Duration was 817.666775 seconds 2024-03-07T06:54:51.063 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-07T06:54:51.073 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-07T06:54:51.073 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-07T06:54:51.081 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-07T06:54:51.235 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-07T06:54:51.244 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-07T06:54:51.640 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-07T06:54:51.640 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-03-07T06:54:51.640 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-07T06:54:51.700 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2024-03-07T06:54:51.700 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-07T06:54:51.771 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-07T06:54:51.771 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-07T06:54:51.779 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-07T06:54:51.869 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-07T06:54:51.869 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-07T06:54:51.874 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-07T06:54:52.055 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-07T06:54:52.066 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-07T06:54:52.066 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-07T06:54:52.160 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-07T06:54:52.257 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-07T06:54:52.270 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-07T06:54:52.277 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-07T06:54:52.380 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-03-07T06:54:52.384 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2024-03-07T06:54:52.436 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-07T06:54:52.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:54:52.484 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-07T06:54:52.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-07T06:54:52.536 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-07T06:54:52.547 INFO:teuthology.task.internal:Transferring archived files... 2024-03-07T06:54:52.548 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926/remote/smithi133 2024-03-07T06:54:52.548 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-07T06:54:52.653 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926/remote/smithi161 2024-03-07T06:54:52.654 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-07T06:54:52.762 INFO:teuthology.task.internal:Removing archive directory... 2024-03-07T06:54:52.762 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-07T06:54:52.770 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-07T06:54:52.828 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-07T06:54:52.840 INFO:teuthology.task.internal:Not uploading archives. 2024-03-07T06:54:52.840 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-07T06:54:52.850 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-07T06:54:52.851 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-07T06:54:52.857 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-07T06:54:52.907 INFO:teuthology.orchestra.run.smithi133.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 7 06:54 /home/ubuntu/cephtest 2024-03-07T06:54:52.913 INFO:teuthology.orchestra.run.smithi161.stdout: 525017 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 7 06:54 /home/ubuntu/cephtest 2024-03-07T06:54:52.916 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-07T06:54:52.927 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-07T06:54:52.949 INFO:teuthology.nuke:Checking targets against current locks 2024-03-07T06:54:52.976 DEBUG:teuthology.nuke:shortname: smithi133 2024-03-07T06:54:52.977 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-07T06:54:53.000 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 06:29:08.180308', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNnQT/fxUBs5Q0elCOxjRtpReucIfdQ7XZT28EvF4NqbJJWKkBq9sNyVNk0kzyXIw+udH5hrcAWgun/AQdgp6tc='} 2024-03-07T06:54:53.004 DEBUG:teuthology.nuke:shortname: smithi161 2024-03-07T06:54:53.004 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-07T06:54:53.029 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-07_05:44:13-fs-wip-lusov-qdb-distro-default-smithi/7583926', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-07 06:29:08.181504', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMTx5NOlXi6N3VtHmq0Qr1XTvtJezbU/EhMYstAczFirOyXNqu+yRMicxrcRDOcv8elfpQq53o9nBxC/te7xEos='} 2024-03-07T06:54:53.061 INFO:teuthology.orchestra.console.smithi133:Power off 2024-03-07T06:54:53.062 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-07T06:54:53.085 DEBUG:teuthology.orchestra.console.smithi133:power off output: Chassis Power Control: Down/Off 2024-03-07T06:54:53.086 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T06:54:53.103 INFO:teuthology.orchestra.console.smithi161:Power off 2024-03-07T06:54:53.103 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-07T06:54:53.114 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-03-07T06:54:53.126 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off 2024-03-07T06:54:53.126 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T06:54:53.149 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-03-07T06:54:57.115 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T06:54:57.150 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T06:54:57.227 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-03-07T06:54:57.261 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-03-07T06:55:01.229 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T06:55:01.262 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T06:55:01.340 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2024-03-07T06:55:01.374 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2024-03-07T06:55:05.341 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T06:55:05.374 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-07T06:55:05.451 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off 2024-03-07T06:55:05.485 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off 2024-03-07T06:55:05.553 INFO:teuthology.orchestra.console.smithi133:Power off completed 2024-03-07T06:55:05.587 INFO:teuthology.orchestra.console.smithi161:Power off completed 2024-03-07T06:55:05.696 INFO:teuthology.run:Summary data: client.0-kernel-sha1: dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 client.1-kernel-sha1: dd5c1cd171679b4e0ceeab3e8d62f25c2b4e6412 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/strays} duration: 817.666775226593 failure_reason: 'Test failure: test_dirfrag_limit (tasks.cephfs.test_strays.TestStrays)' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=466416b0f1624b5eaeac4b2301dc182b status: fail success: false 2024-03-07T06:55:05.697 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-07T06:55:05.775 INFO:teuthology.run:FAIL